NVIDIA NeMo Guardrails
Open source guardrails runtime from NVIDIA. Wraps an LLM app with input, retrieval, execution and output rails, plus Colang for multi-turn dialogue policy. Apache 2.0, version 0.24.0.
Overview
NeMo Guardrails is a Python toolkit that sits between your users and your model, applying programmable rails at five points: user input, dialog flow, retrieved context, tool execution and model output. Rails are defined in a config directory, some as simple classifier calls and some as Colang flows, a Python-like language for describing what a conversation is and is not allowed to do. It is Apache 2.0, around 7k GitHub stars, and now lives under the NVIDIA-NeMo GitHub organisation rather than the old NVIDIA path.
The 2026 releases have been about performance and production fit. IORails, introduced in 0.21.0, is a rewritten input and output engine that avoids much of the legacy overhead, and 0.22.0 through 0.24.0 built it out with streaming, OpenTelemetry, tool-calling support, Hugging Face classifier rails and GLiNER for open PII detection. The bundled server is OpenAI-compatible, which is the single most useful integration decision here: point an existing client at it and the rails apply without touching application code.
Two things to weigh honestly. Latency is unavoidable when a rail asks a model to judge another model's output, so keep LLM-backed rails to the turns that need them and push the rest onto local classifiers. And Colang is a genuine learning investment. Teams that only need input and output filtering usually find the config surface larger than the problem. NVIDIA is clear in the original paper that the built-in rails are a starting point rather than a validated safety guarantee, which means you still own the testing.
Key Features
- ✓ Five rail types covering user input, dialog flow, retrieved context, tool execution and model output
- ✓ IORails engine introduced in 0.21.0, the faster path, expanded through 0.24.0 with engine-neutral rail outcomes
- ✓ OpenAI-compatible guardrails server, so it can sit in front of an existing client with no SDK change
- ✓ Colang 1.0 and 2.0 for declarative multi-turn dialogue policy rather than single-shot filtering
- ✓ Lightweight Hugging Face classifier rails and GLiNER-based PII detection for local, non-LLM checks
- ✓ Streaming support and OpenTelemetry instrumentation for production observability
Where it holds
- • Apache 2.0 with NVIDIA behind maintenance, and a release cadence that has held steady through 2026
- • Dialog rails handle conversational policy that stateless input and output filters cannot express
- • The OpenAI-compatible server makes retrofitting an existing app a config change rather than a rewrite
- • Local classifier and GLiNER rails avoid a paid model call for the common PII and toxicity checks
Where it breaks
- • Latency is a real cost. NVIDIA's own numbers put typical configurations at 100 to 300ms, and chained rails compound it
- • Colang is a language you have to learn, and teams routinely bounce off 2.0
- • NVIDIA's own paper says it should not be a standalone solution for safety-specific rails, which is a fair warning to heed
- • For a plain toxicity or PII filter it is far more machinery than a Llama Guard call behind a function
My Take
Five rail types, one config directory and a domain specific language called Colang. That is the shape of it. Version 0.24.0 shipped on 26 August 2026 and widened the IORails engine, which is the faster path and the one to build on rather than the legacy runtime. The tax is latency: NVIDIA quotes 100 to 300ms for typical configurations, and chaining input, retrieval and output rails on one turn stacks that up. Worth it when you need multi-turn dialogue policy that a stateless filter cannot express. Overkill when you want a toxicity check, where a single Llama Guard call is cleaner. NVIDIA's own paper says not to treat it as a standalone safety solution, and that caution is well placed.
Quick Info
- Pricing:
- open-source
- Openness:
- Open source
- Licence:
- Apache 2.0
- Starting at:
- The library is free under Apache 2.0, installed from PyPI, Python 3.10 to 3.13. The managed route is the NeMo Guardrails NIM microservice, which requires an NVIDIA AI Enterprise licence. NVIDIA does not publish list prices and the commonly quoted figure is around $4,500 per GPU per year, with a 90 day evaluation available. Note that every LLM-backed rail is an extra model call you pay for on top.
- Added:
- Aug 2026
- Updated:
- Aug 2026
Use Cases
Judge it on your own work
The notes above say where NVIDIA NeMo Guardrails holds and where it breaks. The fastest check is your own workload.
Visit website ↗Alternatives to NVIDIA NeMo Guardrails
HiddenLayer
paidEnterprise AI security platform covering model file scanning, AI asset discovery, attack simulation and runtime detection. Agentless, no access to weights or training data required.
garak
open-sourceCommand line vulnerability scanner for LLMs, maintained by NVIDIA. Fires adversarial probe suites at a model endpoint, scores replies with detectors, writes JSONL plus an HTML digest.
Promptfoo
freemiumOpen source CLI for LLM evaluation and red teaming. Generates adversarial cases against your prompt templates and agents, runs them in CI, reports vulnerabilities. MIT licensed.