NVIDIA's SkillEvaluator is a CLI tool that assesses AI agent skills—defined as instruction folders per the Agent Skills specification—across three independent tiers. Tier 1 (Validation) performs static quality and security checks without an LLM, using bundled SkillSpector and Gitleaks scanners plus optional Semgrep; it produces a deterministic score (the reference text-analyzer skill scores 88.8/100, B grade, against a 70 passing threshold) and exits zero on pass, though publication evidence remains INCOMPLETE without scanners. Tier 2 (Deduplication) detects overlapping guidance using embeddings and optional inter-skill comparison via a skill catalog. Tier 3 (Live Evaluation) measures actual agent impact using Harbor, an open-source sandboxed evaluation framework (Docker by default), with autopilot generating skill-grounded test cases. All tiers share a single CLI installable via pip or uv (uv provisions Python 3.13); CI integration is initiated with `skillevaluator ci init`, and agents like Claude Code or Codex can invoke it via `npx skills add`. Optional NVIDIA Build API keys enable LLM-backed rubric evaluation (Tier 1) and embeddings (Tier 2) using hosted endpoints free for eligible developers. A full validation runs `skillevaluator validate` with `--extra all` and `--env-mode docker`; Tier 1 gates the pipeline, Tier 2 gates unless `--no-block-on-dedup` is set, and Tier 3 gates only with `--block-on-agent-eval`. The project is Apache 2.0 licensed, experimental, and community-supported.

Read original