A comprehensive Python framework for evaluating LLM-extracted structured data against ground truth labels. Supports binary classification, scalar values, and list fields with detailed performance metrics, confidence-based evaluation, and statistical uncertainty quantification via non-parametric bootstrap confidence intervals.
pytest for LLM apps - Test for grounding failures, prompt injection, safety violations, and regressions
MCP-native post-generation validation layer for LLM outputs
Deterministic validation firewall that verifies AI-generated proposals against ground-truth state using immutable rules. Zero dependencies. Patent pending.
A new package that helps developers integration-test AI and LLM applications by validating structured outputs. It takes a user's test scenario or prompt as input, sends it to an LLM, and uses pattern
Systematic AI evaluation framework that transforms subjective assessment into objective measurement. Reduce research time by 85% while maintaining 95%+ accuracy through multi-LLM validation.
A unified suite of MCP suppressors that prevent hallucinations, enforce grounding, and stabilise agent behaviour.
A truth filter for AI output. An experiment: I pointed property-based testing (Hegel / Hypothesis lineage) at a specification instead of code. Ran an AI-generated 36 KB research synthesis through the harness — 27 of 28 claims held, 1 was falsified and re-encoded to pass, 6 small structural ingredients surfaced. One case write-up.
C# SDK for the Guardrails AI API -- LLM validation, hallucination detection, PII protection, and content safety
Add a description, image, and links to the llm-validation topic page so that developers can more easily learn about it.
To associate your repository with the llm-validation topic, visit your repo's landing page and select "manage topics."