I Scanned My AI Agent Skills and Found 140 Vulnerabilities

You install a skill for your AI coding agent. It does what it says. You move on.

But what else did it do? What did it read? What did it send somewhere? What could it do next time? What about the other dozen skills you installed last month?

That question kept me up at night. So I scanned everything I run through OpenCode — my primary AI harness these days. 19 skills, 2 plugins, 3 MCP servers. The answer was 140 findings across 17 vulnerability categories. Some were real. Most were noise. All of them taught me something.

Here's what I learned.

What Even Are AI Agent Skills?

If you're using Claude Code, Codex CLI, Gemini CLI, or a BYOK terminal harness like OpenCode, you've probably installed a skill or two. They're like browser extensions for AI agents — small packages that give your agent new abilities.

Some are simple: "write tests" or "format code." Others are complex: run a local server, manage git worktrees, or orchestrate parallel sub-agents. Skills can include markdown instructions, shell scripts, JavaScript, even full server processes.

The problem? When you install a skill, you're giving it trust. Your agent reads the skill's instructions and follows them. If those instructions say "run this shell command" or "read files from ~/.ssh/," your agent does it. No questions asked. No permission dialog. No sandbox.

There's no app store review. No permission prompt. No "this skill wants to access your filesystem" warning. You type install skill-name and hope for the best. Some skills are just markdown files with instructions. Others are full applications with JavaScript servers, shell scripts, and database access. You're trusting all of them equally.

Research backs this up. A large-scale study of over 42,000 skills pulled from major marketplaces found that roughly a quarter contain at least one vulnerability, and about 1 in 20 show likely malicious intent. Those aren't hypothetical risks. Those are skills people actually installed. And the ecosystem is growing fast — hundreds of new skills appear every month across GitHub, npm, and community registries.

Enter SkillSpector

SkillSpector is a security scanner built specifically for this problem, open-sourced by NVIDIA. Think of it as a virus scanner, but for AI agent skills.

It works in two stages:

Stage 1: Static analysis. Fast, no API key needed. Scans skill files against 68 vulnerability patterns across 17 categories — prompt injection, data exfiltration, privilege escalation, supply chain risks, tool misuse, and more. This takes seconds.

Stage 2: LLM semantic analysis. Optional. Sends skill content to an LLM for deeper understanding. Catches things pattern matching misses — subtle intent, context-dependent risks. Requires an API key (SkillSpector supports OpenAI, Anthropic, Bedrock, NVIDIA's own build.nvidia.com, or even your existing Claude CLI/Codex CLI login).

The key insight: SkillSpector doesn't just look for known malware signatures. It understands what skills do. It reads shell scripts and asks "is this rm -rf actually necessary?" It reads JavaScript and asks "why does this need child_process?" It reads markdown instructions and asks "is this telling the agent to ignore safety constraints?"

That's the difference between scanning file extensions and understanding intent. And it's the difference between a tool that catches obvious threats and one that catches subtle ones.

It's also gaining traction fast — as of this writing, the repo sits at roughly 14.4k stars on GitHub, which tells you this isn't a niche side project. People are actually using it.

Try It

This post isn't a how-to — the README already covers installation better than I could rewrite it here. If you want to run this against your own setup, the repo is at github.com/NVIDIA/SkillSpector. Point it at a skill directory, a zip, or a Git URL, and it hands back a risk score in seconds.

What I care about here is what I actually found when I ran it against my own stack — and what it taught me about telling real risk apart from noise.

What I Found: 19 Skills, 140 Findings

I ran SkillSpector against my actual setup — the plugins and skills I've layered into OpenCode over the past few months. Here's what it scanned:

• 14 skills from the superpowers plugin (brainstorming, test-driven-development, systematic-debugging, etc.)
• 5 skills from the caveman plugin (communication compression modes)
• 2 plugin packages (the full plugin directories)

The results:

That looks terrifying. 100 CRITICAL? Time to uninstall everything?

Not so fast. Before you nuke your entire setup, there's an important nuance that separates useful security scanning from panic-inducing noise.

The False Positive Problem

Here's the thing about security scanners: they're paranoid by design. And that's correct — false negatives are dangerous. But it means you get a lot of noise.

Most of those 140 findings were documentation false positives. The superpowers plugin includes docs/plans/, RELEASE-NOTES.md, and CLAUDE.md in its package. Those files reference shell commands, describe agent configurations, and discuss installation procedures. SkillSpector reads them and flags the patterns it sees.

But docs aren't executable. A markdown file that mentions rm -rf isn't the same as a shell script that runs rm -rf.

The solution: baselines.

SkillSpector lets you suppress known findings with glob rules or fingerprints. I created baselines that say "ignore everything in docs/, RELEASE-NOTES.md, and .github/ — those are documentation, not runtime code."

After baselines:

70 findings suppressed. 4 real findings remained. And all 4 were intentional design patterns:

• Two files are agent plugin entry points (TypeScript, Python) — they're supposed to be executables
• One is a documentation example of debug instrumentation
• One is a session lifecycle hook configuration

Zero actual vulnerabilities. The tools were clean.

This is exactly how security scanning should work. Not "everything is fine" out of the box — that would mean the scanner isn't looking hard enough. Instead: flag everything, let you decide what's real, and then track only the new stuff going forward. Baselines turn a firehose of findings into a manageable, incremental workflow.

Who Needs This?

Four use cases where SkillSpector earns its place:

1. Pre-install gate. Before installing a skill from the internet, scan it. Takes 5 seconds with --no-llm. If the score is high, at least read the findings before proceeding.

2. CI/CD pipeline. If you distribute skills to a team, scan them in your build pipeline. Block high-severity findings from reaching production. Gate skill installs on scan results.

3. Supply chain audit. Skill marketplaces and registries can use SkillSpector to vet submissions before publishing. The NVIDIA Verified Skills pipeline does exactly this.

4. Compliance and governance. Regulated industries need audit trails. SkillSpector's JSON and SARIF outputs feed into existing security tooling. If your security team asks "what's running in your AI agents?" — a SkillSpector report is a better answer than "I think it's fine."

Each use case serves a different level of paranoia. Individual developers might just want a quick sanity check before installing something new. Security teams want automated gates in the pipeline. Regulators want documented evidence of due diligence. SkillSpector covers all three.

Do You Really Need an AI Skill Vulnerability Scanner?

Here's the honest answer: it depends.

You definitely need it if:

• You install skills from the internet without reading every line of code
• You distribute skills to other people
• You're in a regulated industry with compliance requirements
• You've ever thought "I wonder if this skill is safe"
• Your team uses AI agents and you're responsible for what they do

You probably don't need it if:

• You write all your own skills from scratch
• You only use skills from publishers you fully trust
• You manually audit every skill before installation (be honest — you don't)
• You're a solo developer who never installs third-party skills

Think of it like a seatbelt. You don't need a seatbelt when you're driving perfectly on an empty road. You need it when something unexpected happens. The difference is that seatbelts cost you nothing to wear, and scanning a skill takes 5 seconds.

The cost of scanning is near-zero. The cost of not scanning is... unknown. That's the scary part. You don't know what you've already installed that might be problematic. You won't know until something goes wrong.

I scanned my setup. It took a few minutes. I learned my skills were clean. That's worth the peace of mind.