marian2js/skill-doctor

Summary

skill-doctor is a CLI linter and static analyzer for 'agent skill' packages — Markdown-based instruction files (SKILL.md) used to route AI agents to specialized capabilities. It scans SKILL.md frontmatter, validates bundle integrity (broken local links, escaped references), scores trigger description quality via regex heuristics, and validates evals/evals.json schema. Output is a 0-100 score with categorized diagnostics, available as CLI, GitHub Action, or Node.js API.

Great for

people interested in building developer tooling for the emerging AI agent skill/tool ecosystem, specifically linting and quality-scoring structured Markdown instruction formats used by systems like Anthropic's Claude agents

Easy wins

• +Add more trigger language patterns to TRIGGER_LANGUAGE_PATTERN in analyze-skill.ts — the current regex covers basic phrases but misses equivalents like 'activate when', 'invoke for', 'call this skill' etc.
• +The PLAIN_PATH_PATTERN in read-skill-document.ts only checks agents/assets/references/scripts subdirs; adding 'prompts/' or 'tools/' as recognized dirs would improve coverage for other skill conventions.
• +Write tests for edge cases in parseFrontmatter (e.g., frontmatter with '...' closer, YAML with multi-line strings, files with Windows CRLF) — the normalizeContent call handles CRLF but there are no tests exercising it.
• +Add a --watch mode that re-runs analysis on file changes, which would be useful for skill authors iterating locally.

Red flags

• !Only 1 commit in the entire repo history — this is a brand-new project with essentially no commit history to evaluate trajectory or maintainer responsiveness from.
• !Zero open issues and 0 good-first-issues/help-wanted labels means there's no structured onboarding path for contributors beyond the CONTRIBUTING.md.
• !The 'skill ecosystem' this targets (agentskills.io, Anthropic skill-creator workflow) is niche and still evolving — if the underlying spec changes, large parts of the rule taxonomy could become stale quickly.
• !TRIGGER_LANGUAGE_PATTERN and GENERIC_DESCRIPTION_PATTERN are hardcoded regex with no configuration surface — strict/pedantic mode changes severity but can't add domain-specific trigger phrases, limiting extensibility.

Code quality

The code is clean and consistent throughout. analyze-skill.ts properly separates diagnostic collection into focused functions (collectReferenceDiagnostics, collectEvalsDiagnostics, addFrontmatterDiagnostics) with no obvious God-function sprawl. Error handling in collectEvalsDiagnostics correctly catches JSON.parse failures and returns early. The render.ts terminal formatting is well-abstracted via helpers like printFramedBox and colorizeByScore rather than inline string manipulation. One minor concern: the regex-based trigger quality detection (TRIGGER_LANGUAGE_PATTERN, GENERIC_DESCRIPTION_PATTERN) is inherently brittle and will produce false positives/negatives, but this appears to be an intentional, documented tradeoff rather than a bug. The test in diagnose.test.ts uses real temp-filesystem fixtures rather than mocks, which is a solid choice for an FS-heavy tool.

What makes it unique

There's no direct equivalent in the npm ecosystem for linting agent skill Markdown packages specifically. The closest analogs are markdownlint (generic) or react-doctor (which this explicitly credits for CLI shape). Within the narrow niche of AI agent skill quality tooling it's genuinely novel, but that niche is small and platform-specific enough that it could be made obsolete if Anthropic ships first-party validation.