Benchmark — August 2026

Independent OWASP LLM Top 10 + content-moderation evaluation of 10 AI guardrail backends.

707 probes · 10 backends · generated 2026-08-04 13:05 UTC · run ac387219 · guardrailprobe v0.1.5

llm_guard

Winner — 85.9% overall

guardrails_ai

Best accuracy / latency ratio

−5.1%

Biggest regression — azure_content_safety

707

Total probes run · 0 backends skipped

Signed PDF JSON Markdown Methodology

Overall comparison

Backend	Overall	vs last month	Best	Worst	Avg latency
openai_moderation	100.0%	+0.0%	LLM01	LLM01	33 422 ms
llm_guard	85.9%	+0.0%	LLM01	LLM10	2 569 ms
lakera	83.3%	+1.3%	LLM01	LLM10	584 ms
nemo	79.5%	−5.1%	LLM01	LLM10	31 454 ms
aws_bedrock	59.0%	+0.0%	LLM01	LLM10	660 ms
azure_prompt_shields	24.4%	+0.0%	LLM01	LLM09	1 613 ms
azure_content_safety	20.5%	−5.1%	LLM02	LLM01	1 620 ms
llama_firewall	11.5%	+0.0%	LLM01	LLM05	8 043 ms
presidio	6.4%	+0.0%	LLM02	LLM01	974 ms
guardrails_ai	2.6%	+0.0%	LLM01	LLM05	0 ms

Low scores are not always failures: Presidio is a PII-detection tool being tested against injection probes, and guardrails_ai's 0 ms latency means its hub validators never actually executed (see the methodology's two-tier categorisation of general-purpose vs specialised tools).

Per-category winners (OWASP LLM Top 10)

Category	Description	Winner	Score	Runner-up	Score
LLM01	Prompt Injection	lakera	100%	aws_bedrock	100%
LLM02	Insecure Output	lakera	81%	llm_guard	81%
LLM03	Training Data Poisoning	lakera	100%	aws_bedrock	100%
LLM04	Model DoS	lakera	67%	llm_guard	67%
LLM05	Supply Chain	llm_guard	100%	lakera	80%
LLM06	Sensitive Info Disclosure	lakera	100%	llm_guard	100%
LLM07	Insecure Plugin	lakera	83%	llm_guard	83%
LLM08	Excessive Agency	lakera	100%	llm_guard	100%
LLM09	Overreliance	llm_guard	100%	nemo	100%
LLM10	Model Theft	lakera	20%	llm_guard	20%

Content moderation

Backend	Hate	Violence	Sexual	Self-harm	Overall CM
nemo	100%	100%	100%	100%	100%
lakera	100%	100%	100%	100%	100%
aws_bedrock	100%	100%	100%	100%	100%
llm_guard	100%	100%	100%	100%	100%
azure_content_safety	80%	100%	60%	80%	80%
llama_firewall	0%	0%	20%	0%	5%
guardrails_ai	0%	0%	0%	0%	0%
presidio	0%	0%	0%	0%	0%
openai_moderation	0%	0%	0%	0%	0%
azure_prompt_shields	0%	0%	0%	0%	0%

Accuracy vs latency

Backend	Overall	Avg latency	Class	Recommended for
guardrails_ai	2.6%	0 ms	Ultra-fast	Real-time, high-throughput pipelines
lakera	83.3%	584 ms	Moderate	Batch processing, async pipelines
aws_bedrock	59.0%	660 ms	Moderate	Batch processing, async pipelines
presidio	6.4%	974 ms	Moderate	Batch processing, async pipelines
azure_prompt_shields	24.4%	1 613 ms	Slow	Offline analysis, compliance audits
azure_content_safety	20.5%	1 620 ms	Slow	Offline analysis, compliance audits
llm_guard	85.9%	2 569 ms	Slow	Offline analysis, compliance audits
llama_firewall	11.5%	8 043 ms	Slow	Offline analysis, compliance audits
nemo	79.5%	31 454 ms	Slow	Offline analysis, compliance audits
openai_moderation	100.0%	33 422 ms	Slow	Offline analysis, compliance audits

Month-over-month

Backend	July	August	Change	Status
lakera	82.0%	83.3%	+1.3%	stable
aws_bedrock	59.0%	59.0%	+0.0%	stable
azure_prompt_shields	24.4%	24.4%	+0.0%	stable
guardrails_ai	2.6%	2.6%	+0.0%	stable
llama_firewall	11.5%	11.5%	+0.0%	stable
llm_guard	85.9%	85.9%	+0.0%	stable
openai_moderation	100.0%	100.0%	+0.0%	stable
presidio	6.4%	6.4%	+0.0%	stable
azure_content_safety	25.6%	20.5%	−5.1%	regression
nemo	84.6%	79.5%	−5.1%	regression

Reproduce this benchmark

Any machine, ~10 minutes with credentials configured

pip install guardrailprobe
guardrailprobe run --year 2026 --month 8

Reports regenerate on the first of every month via GitHub Actions. Signed PDF carries an RFC 3161 timestamp — verify with guardrailprobe cert verify benchmark_2026_08.pdf.