Is AI-generated code safe to run? How to check it first
Not automatically. An AI coding assistant can add a dependency you did not ask for, copy a setup command from an untrusted page, write config that runs on startup, or produce insecure code that calls a model. Before you run it, scan the project with npx am-i-hacked for malicious-code indicators and npx secure-semgrep . for insecure AI-agent, shell and SSRF patterns, then review every new dependency and config file yourself.
What goes wrong with AI-written code
- Dependencies you did not choose. Assistants sometimes suggest package names that do not exist or are near-misses of real ones. Attackers register those names. Check that every new dependency is the package you meant, with a real history.
- Setup steps copied from somewhere else.
curl ... | sh, apostinstallscript, an editor task. An agent that browses can repeat what an untrusted page told it. - Config that runs code. Agent hooks, MCP server definitions,
.claude/settings.json,.mcp.jsonand editor settings all start programs. - Insecure model-calling code. Hardcoded API keys, model output passed to
exec, user input placed in a system prompt, no token limit. - Server code that fetches URLs without checking where they point. See SSRF.
How to check it
- Scan for malicious-code indicators. From the project folder:
It reads the source and the project's own AI-tool config (npx am-i-hacked.claude/settings*.json,.mcp.json) and exits1on findings. Add it in front of the dev server so it runs every time:{ "scripts": { "dev": "am-i-hacked && next dev" } } - Scan for insecure AI-agent code. secure-semgrep runs Semgrep with bundled rules for LLM and agent code, plus Semgrep's own default and security-audit packs. It needs
semgrepinstalled.npx secure-semgrep . npx secure-semgrep -L ts -L node . # add the packs for your stack - Review every new dependency by hand, and run your advisory scanner (
npm audit, OSV-Scanner) over the lockfile. - Read every new config and script file the assistant created or changed:
package.jsonscripts,*.config.*, editor tasks, agent hooks, CI workflows. - Check the machine the agent runs on with
npx am-i-hacked --system, which reads user-level config for Claude Code, Codex, Cursor, Gemini CLI, OpenCode and Kilo Code. See how to check a developer machine.
What the secure-semgrep AI rules cover
| Area | Examples of what is flagged |
|---|---|
| Model provider SDKs (OpenAI, Anthropic, Gemini, Cohere, Mistral, Hugging Face) | Hardcoded API keys, user input in the system prompt, no max-tokens limit, missing refusal, moderation or safety checks, no error handling |
| Agent code | Model output passed to code execution, LangChain dangerous execution, an agent loop with no bound |
| MCP servers | Command injection in tool handlers, credentials returned in responses, hardcoded secrets in config, SSRF, tool poisoning, typosquatted tool names, unsanitized return values |
| Agent and editor settings | Claude Code settings that bypass permissions, auto-enable MCP servers or redirect the API URL; editor settings that point at an executable; hidden Unicode in AI config |
| Agent hooks | wget | bash, DNS exfiltration, path traversal, reading sensitive files, unconditional allow, relative script paths, unquoted variables |
SKILL.md files | Prompt injection, data exfiltration, reading sensitive files, base64 payloads |
Rules cover Python and TypeScript most fully, with Go, Java, Ruby and others for some checks. Every finding is a prompt to look, not a verdict.
Limits
Neither tool reads your intent. A scan finds known patterns; it does not prove that the code does what you asked or that it is safe. Neither tool scans installed dependencies in node_modules. Code review is still the check that matters.