Ground truth in the age of AI agents

A practitioner on Hacker News recently diagnosed a structural failure in multi-agent systems: "Agent A infers something and calls it near-certain. Agent B receives it as fact. By Agent C, a guess has become ground truth." Metacognitive poisoning happens because traditional evaluation can't track uncertainty across non-deterministic loops. You must rethink how you define and measure correctness when static datasets fail.

TL;DR

• Data silos block agents from synthesizing reality and disrupt 82 percent of enterprise workflows.
• Frontier models achieve only 38 percent accuracy on complex data queries because static datasets can't evaluate non-deterministic tool use.
• Evaluating agents requires separating the transcript of what an agent says from the outcome it executed in the environment.
• Mathematical reasoning scores track the logic paths of agents to prevent confident hallucinations.
• Intermediate validation loops catch silent failures before they cascade across multi-agent handoffs.

The terminology trap: Grounding versus ground truth

You might conflate grounding an agent with establishing ground truth. Grounding provides context through vector search or database queries to keep a model anchored in corporate knowledge. It answers the question of what data the agent can see. Ground truth is the benchmark used to evaluate if the agent's decisions against that knowledge were correct. It answers the question of whether the agent did the right thing.

Data silos make both processes difficult. Data silos disrupt workflows for 82 percent of enterprises, and 68 percent of organizational data remains unanalyzed (IBM Data Differentiator, 2025). When reality is fragmented across disconnected systems, you might assume fixing the data architecture will solve your evaluation problem.

It doesn't.

Connecting an agent to a data catalog gives it the ability to read the data. It provides no way to score whether the agent chose the right tables and joined them correctly. Grounding gives the agent the map, but ground truth verifies if the agent took the correct route. If an agent has access to 50 disconnected databases, the grounding step just feeds it more schemas. The evaluation step needs to confirm the agent didn't hallucinate a join between two incompatible tables.

You can ground an agent in your corporate documentation and still lack any ground truth to measure its reasoning.

Why golden datasets fail for agentic loops

For years, machine learning teams evaluated models by comparing text outputs to golden datasets. If the model's output string matched the expected string, the test passed.

Static testing breaks down when applied to autonomous agents. Agents execute multi-step loops where they select tools, query databases, read errors, and make routing decisions.

There is no single correct path to a result.

The performance gap in agentic reality

The performance gap between static QA and agentic reality is significant. Frontier models like Gemini-3-Pro achieve only 38 percent pass@1 accuracy on complex data queries (UC Berkeley, 2026). The models fail because they get lost in fragmented state. They try to reason across multiple systems without a mechanism to verify their own intermediate steps.

An agent might write a SQL query that executes without syntax errors but pulls from the wrong column. A static dataset just sees a successful query execution. The agent moves forward with bad data. The compounding nature of these errors means the output might look highly confident but be fabricated.

Without verifying intermediate steps, errors compound invisibly across the system. One practitioner diagnosed the failure mode as "metacognitive poisoning."

Agent A makes a speculative inference. Agent B receives that output as a fact. By the time the data reaches Agent C, a guess has solidified into certainty. The uncertainty disappeared across the handoffs.

Static datasets can't catch this failure because they only check the output string, ignoring the flawed logic that produced it.

Evaluating non-deterministic trajectories

Separating transcript from outcome

Evaluating non-deterministic loops requires you to evaluate the environment. Evaluating agents requires separating the transcript from the outcome, a standard recommended by Anthropic.

The transcript is the agent's internal log of what it did and why.

The outcome is the state of the environment. An agent might confidently state in its transcript that it updated a customer record. If you evaluate the text, the agent passes. If you query the SQL database and the record remains unchanged, the outcome is a failure.

Agents often find unconventional paths to success or hallucinate competence while failing to execute. Ground truth must anchor to environmental state changes. You measure success by querying the API or database to confirm the action occurred. You don't ask the LLM if it finished the job.

Consider a flight-booking agent. The agent might say "Your flight is booked" at the end of its run. That statement is just the transcript. The outcome is whether the reservation exists in the airline's database.

When you rely solely on the transcript, you're grading the agent's ability to sound convincing. You aren't grading its ability to perform work.

Scoring the reasoning path

Environmental checks confirm if a task was completed. They don't confirm if the agent is safe to deploy at scale. An agent that drops a database table to update one row achieved the outcome, but the reasoning is flawed.

New evaluation frameworks treat the logic path itself as a measurable artifact. A new "Reasoning Score" framework proposed in an IETF Internet-Draft standardizes this measurement. The system compares the agent's step-by-step logic against an expert reasoning path to see how closely they match.

The reasoning score uses mathematical similarity to measure the distance between the agent's text and the expert's text. A high score means the agent followed the approved steps. A low score means the agent invented its own process, even if the output looks correct.

Mapping the agent's trajectory against a verified path identifies where the agent deviated. If the agent skipped a validation step but still got the right answer, the reasoning score flags the behavior as unreliable.

Comparing the math evaluates the internal logic alongside the action. It ensures the agent arrived at the correct answer for the right reasons. You gain a deterministic metric for non-deterministic behavior. The reasoning score catches agents that succeed through luck or unintended shortcuts before they reach production.

Scaling the human element in intermediate validation

The limits of programmatic validation

There is a growing belief among developers that manual labeling is a dead end. Some developers now argue the system prompt itself should serve as the programmatic ground truth. Since the prompt defines the constraints, you can build automated scorers to check agent outputs directly against those rules without human intervention.

Programmatic testing works well for formatting checks. It breaks down in complex workflows.

System prompts can't anticipate every edge case in a dynamic environment. Teams deploying agents in production recommend intermediate validation loops to prevent runaway mistakes. These loops require checking agent proposals against ground truth data before allowing the agent to proceed to the next step.

Building the virtuous cycle

In production, domain experts catch the silent failures that programmatic scorers miss.

During an internal deployment of our own AI support agent, we diagnosed a silent failure in a retrieval pipeline where the agent missed critical sources. Automated checks missed the error because the output looked structurally sound. Subject matter expertise was required to spot the missing context.

We corrected the documentation syntax and added those trajectories to our benchmark datasets. Our case study on the virtuous cycle details the full process.

Human-in-the-loop evaluation scales effectively even for complex reasoning. When evaluating a healthcare AI assistant for the NIH, Mind Moves orchestrated a 6-phase workflow spanning over 20,000 annotation tasks across 100 biomedical questions. The project used HumanSignal's platform and 32 subject matter experts to evaluate the agent's logic against health literacy and policy alignment.

The experts didn't just grade the answer. They graded the intermediate steps to create a dataset of correct reasoning paths. The process yielded a 50 percent initial response acceptance rate. It created a ground truth dataset to scale automated evaluation safely.

Metacognitive poisoning is inevitable when agents hand off unverified inferences without oversight. Intermediate validation loops and outcome-based evaluation force agents to prove their logic at every step. Transitioning from static QA datasets to continuous evaluation workflows ensures your agents actually solve problems and stops them from confidently hallucinating.