garak
Command line vulnerability scanner for LLMs, maintained by NVIDIA. Fires adversarial probe suites at a model endpoint, scores replies with detectors, writes JSONL plus an HTML digest.
Overview
NVIDIA maintains garak as a command line scanner for large language models. You hand it a generator, meaning an OpenAI key, a Hugging Face model id, a local Ollama endpoint or a WebSocket, then choose probes. It runs adversarial prompts against the target while detectors score the responses. Output is JSONL for machines and an HTML digest for humans. The mental model the project uses is Nessus for LLMs, and that is close enough: a probe library, a detector library and a report.
Probe coverage is the reason to use it. Prompt injection, DAN lineage jailbreaks, encoding smuggling, glitch tokens, GCG adversarial suffixes, package hallucination that leads to slopsquatting, PII leakage and toxicity all have dedicated modules. The 2026 releases pushed into harder ground: 0.15.0 in May added a multi-turn GOAT probe and an agent-breaker probe for tool-using systems, and 0.14.1 added bootstrap confidence intervals so two runs against different checkpoints can actually be compared. Detectors improved too, with a ModernBERT refusal classifier replacing brittle substring checks.
Where it falls down is everything above the model. garak assesses a generator in isolation, so indirect prompt injection through your document store, over-permissive tool scopes and broken session isolation are all outside its view. Promptfoo covers that layer better. The sensible arrangement is garak for model-level regression testing when you swap or fine-tune a checkpoint, and an application-aware harness for the system you actually ship.
Key Features
- ✓ Probe families for prompt injection, DAN style jailbreaks, encoding smuggling, glitch tokens, GCG suffixes, package hallucination and PII leakage
- ✓ Multi-turn GOAT probe and an agent-breaker probe aimed at tool-using systems, added in 0.15.0
- ✓ Generator plugins for OpenAI, Anthropic, Hugging Face, Bedrock, Groq, Replicate, local models and WebSocket endpoints
- ✓ Detectors that go beyond string matching, including a ModernBERT refusal classifier
- ✓ JSON config alongside YAML since 0.14.0, which makes CI invocation programmable
- ✓ Bootstrap confidence intervals on attack success rates, so repeat runs are comparable
Where it holds
- • Genuinely free and Apache 2.0, with NVIDIA paying for maintenance rather than a startup runway
- • Probe library is deep and academically grounded, not a handful of jailbreak strings scraped off Reddit
- • Plugin architecture means adding a generator for an internal model server is a short Python file
- • Reports are machine readable, so results can be diffed between model versions
Where it breaks
- • Model-level only. It knows nothing about your retrieval layer, tool permissions or session state
- • Runs are slow and noisy, and triaging a full sweep by hand takes real hours
- • Token spend against hosted APIs adds up fast and there is no built-in budget cap
- • Documentation lags the code, so probe semantics often have to be read from source
My Take
Point it at an endpoint, walk away, come back to a JSONL file full of the ways your model folded. Version 0.16.0 landed on 4 August 2026 with intent annotation and a native Anthropic generator, and the repo sits around 9.1k stars under NVIDIA's org. The catch is scope. It probes the model, so it will find encoding tricks and glitch tokens, and tell you nothing about whether your RAG pipeline lets an attacker plant instructions in a PDF. Treat it as a weekly baseline scan, not a commit gate, and pair it with something that understands your application.
Quick Info
- Pricing:
- open-source
- Openness:
- Open source
- Licence:
- Apache 2.0
- Starting at:
- Free. Apache 2.0, installed with pip install garak, no paid tier and no hosted service. The only real cost is tokens: a full probe sweep against a commercial API runs to thousands of calls, so budget for it before pointing it at a metered endpoint.
- Added:
- Aug 2026
- Updated:
- Aug 2026
Use Cases
Judge it on your own work
The notes above say where garak holds and where it breaks. The fastest check is your own workload.
Visit website ↗Alternatives to garak
Promptfoo
freemiumOpen source CLI for LLM evaluation and red teaming. Generates adversarial cases against your prompt templates and agents, runs them in CI, reports vulnerabilities. MIT licensed.
HiddenLayer
paidEnterprise AI security platform covering model file scanning, AI asset discovery, attack simulation and runtime detection. Agentless, no access to weights or training data required.