AST08: Poor Scanning
Risk Level: Medium
Scanners built for traditional code look for code. A malicious skill's payload can be a sentence of English telling the agent to read a credentials file and post it to a remote host, which contains no suspicious function call, no obfuscated string, and nothing for a regex to match. Running a scanner and getting a clean result is not evidence that a skill is safe.
What the numbers show
Snyk found critical-level security issues in 13.4% of the 3,984 skills it scanned (534 skills), and reported that the majority of those issues were not caught by simple pattern matching. Against Snyk's toxicskills-goof test suite, SpecWeave's pattern-matching scanner caught three of four samples; the fourth used pure natural-language instructions with no detectable code signature at all.
Mitigations
- Scan the natural-language instruction layer separately from the code layer
- Use behavioral analysis that evaluates intent, not signature matching alone
- Detonate skills in an isolated sandbox and observe what they actually do
- Chain multiple techniques: pattern matching, then semantic analysis, then sandbox observation
- Treat any scanner result as advisory, never as the sole gate on installation
- Re-scan already-installed skills as detection models improve