Find shadow AI in your MCP configs

npx toolstile audit is a free, open-source command that reads the MCP server configs of eight AI clients on a machine and reports plaintext tokens, unencrypted remote servers, unpinned packages and tools that run without asking. The audit itself runs locally, makes no network requests and never prints a secret value. Only --fix goes online, to look up package versions on npm or PyPI.

npx toolstile audit

It needs Node.js 18 or later and no Toolstile account. The source is MIT-licensed at github.com/toolstile/cli and published to npm as toolstile. It is about 3,500 lines of TypeScript, so you can read it before you run it.

The part of shadow AI it can see

Shadow AI is AI tooling people use without the security team knowing about it or managing it. Most of it is out of reach for a config check: chat apps in the browser, personal API keys, extensions. MCP servers are a part you can actually inspect. Developers add them to Claude Code, Cursor, Codex and the rest by editing a JSON or TOML file. Each entry either starts a local program with the developer's permissions or connects to a remote service, and often carries a personal access token pasted straight into the file. Nothing central lists them.

The audit reads those files on one machine and tells you which servers are set up, where the credentials sit and which entries are risky. It does not see AI tools that don't use these files, watch network traffic, or judge what a server does once it runs. Treat it as an inventory and hygiene check for MCP configs, not a shadow AI scanner for the whole company.

What it checks

A few details from the source:

• Secrets. It recognizes common token formats (GitHub, GitLab, Anthropic, OpenAI, Slack, AWS, Stripe, Linear, Notion, private keys, JWTs and more), Authorization headers, passwords in connection strings and credentials in URL query strings. It also flags other values of eight or more characters under keys such as token, secret, password or api_key. Values that are already variable references, like ${GITHUB_TOKEN}, ${env:GITHUB_TOKEN} or %GITHUB_TOKEN%, pass.
• Unpinned packages. It reads npx, bunx, pnpx, pnpm dlx, yarn dlx, bun x, uvx, uv tool run, pipx run and docker run, including behind cmd /c or wsl on Windows. A package without an exact version, or one tagged latest or next, counts as unpinned.
• File permissions. A config with secrets is flagged only when the file and every folder above it let another account reach it. This check is skipped on Windows.

When it finds servers or issues, the report includes a score out of 100 and a grade from A to F. One leaked secret scores 83 (B); four score about 56 (D).

Which clients it reads

Project configs come from the current directory. Point it elsewhere with --project <dir>, or skip them with --no-project. In ~/.claude.json it also reads the servers Claude Code stores per project. An existing file that can't be read or parsed is listed with a safe reason. Parse errors include a position when one is available; file contents are never quoted.

Sample output

toolstile audit · local MCP config check · nothing leaves this machine

  Grade C  71/100   2 high · 1 medium
  10 MCP servers in 4 files from Claude Code, Cursor, Codex

HIGH  Plaintext secrets (2)
  Anything that can read these files can use the credentials, including
  backups, dotfile syncs and other tools.

  • github (Cursor)  GitHub token
    ~/.cursor/mcp.json  mcpServers.github.env.GITHUB_PERSONAL_ACCESS_TOKEN
  • linear (Claude Code)  authorization header
    ~/.claude.json  mcpServers.linear.headers.Authorization

  Fix Move them into environment variables, or use the server's OAuth sign-in.
      Rotate any secret in a file that was shared or synced.
…
Fix them: npx toolstile audit --fix (shows every change; you pick which to apply)

Findings name the file and the key path, such as mcpServers.github.env.GITHUB_PERSONAL_ACCESS_TOKEN, and the kind of secret. The value itself is never shown.

What --fix changes

npx toolstile audit --fix

It lists every change it can make as a diff with secret values masked, then lets you pick which to apply (space toggles, enter applies):

  1.  HIGH   github (Cursor): move secret to $GITHUB_PERSONAL_ACCESS_TOKEN
             ~/.cursor/mcp.json  mcpServers.github.env.GITHUB_PERSONAL_ACCESS_TOKEN
             - "••••"
             + "${env:GITHUB_PERSONAL_ACCESS_TOKEN}"
  2.  MEDIUM linear (Claude Code): pin linear-mcp to 1.4.2
             ~/.claude.json  mcpServers.linear.args[1]
             - "linear-mcp"
             + "linear-mcp@1.4.2"

It leaves the rest for you and says what to change: secrets in arguments or URLs, Claude Desktop and Cline (they can't read variables), secrets on Windows, http:// servers, Docker images and auto-approve lists. --fix doesn't list broad filesystem roots, so review those in the read-only audit report.

Before writing a file, it checks that the edited text parses to exactly the planned result; if not, the file is left alone. JSON files keep their comments and layout. The originals are copied to ~/.config/toolstile/backups/ first, and those copies still hold the old secrets, so delete them once your agents work. After moving secrets, open a new terminal, restart your agents and rotate any secret that sat in a shared or synced file.

--dry-run only shows the changes. --yes applies the ones that start selected without asking.

It runs on your machine

• The audit reads config files and writes nothing, unless you ask for a grade card or confirm changes in --fix.
• The read-only audit makes no network requests, and the CLI collects no telemetry. --fix, including a dry run, sends the names of the packages it offers to pin to npm or PyPI to look up their current versions.
• It never prints or sends secret values, and the read-only audit stores none. Applying a secret fix saves the value in the owner-only secrets file and keeps config backups, as described above. Duplicate credentials are matched by hash in memory.
• Its runtime dependencies are smol-toml for Codex's TOML and jsonc-parser for comment-preserving edits, plus an optional PNG renderer for the grade card.
• Releases are published from GitHub Actions with npm provenance, so each version links back to the commit that built it.

Running it across a team

There is no central console for audit results. Each person runs it on their own machine and the report stays there, so collecting results is up to you. What the CLI gives you for that:

• JSON output. npx toolstile audit --json prints the files, server count, findings, score and grade. Findings carry the rule, severity, file and key path, never the value. Collect it with whatever you already use to gather data from laptops.
• A gate for checked-in configs. --fail-on high (or medium, low) exits with code 1 when a finding reaches that level, so you can run it in CI or a pre-commit hook for repositories that commit .mcp.json, .cursor/mcp.json or .vscode/mcp.json. Exit code 2 means invalid options or a failed run.
• Output safe to share. --redact replaces your home folder and project paths with ~ and <project>. --card writes a 1200×630 image with only the grade and counts.
• A library. import { audit } from "toolstile" returns the same result object, if you'd rather wrap it in your own script.

In CI, pin the version you reviewed, as the audit itself would advise:

npx toolstile@0.3.0 audit --fail-on high

Where Toolstile fits

The audit works on its own. It comes from the team behind Toolstile, a managed MCP gateway, because two of the problems it finds, pasted tokens and the same server set up in every client, are what a gateway removes for remote servers.

With Toolstile, each AI client connects to one Toolstile address with its own authorization. Upstream credentials for GitHub, Linear and the rest are stored encrypted in Toolstile and are never handed to the agents, so there is no per-client token to paste into a config and nothing to copy between clients. You choose which tools each agent may use in per-agent permissions, revoke one agent without rotating the account, and see every call in one activity log. Security and data handling covers storage and retention.

The same CLI can connect a machine's agents without writing a token into any config:

npx toolstile auth login
npx toolstile mcp setup

Each enabled agent's config runs a local relay, toolstile mcp proxy, which keeps that agent's token in the CLI's owner-only credentials file. The audit has nothing to flag in those entries. See set up a whole machine from the terminal.

Toolstile reaches public HTTPS servers only. Local stdio servers, such as filesystem or Docker servers, stay configured in each client, so keep running the audit for them. For the wider set of MCP risks and controls, read MCP security risks and best practices.

Start free · Connect your first MCP server