State-based evaluation – Definition & Meaning

Definition of state-based evaluation

: a method for evaluating an AI agent by comparing the final state and side effects of the environment it acted on with a specified goal state, rather than relying only on the agent's response or action trace
- The support agent claimed that it had issued a refund, but state-based evaluation found no refund record in the database.
- The benchmark reset the sandbox before every run so state-based evaluation could attribute each change to the tested agent.

Origin & history

first known use2024

The 2024 tau-bench benchmark evaluated tool-using agents by comparing the database state at the end of a conversation with an annotated goal state. The approach expanded across agent benchmarks, and by 2026 researchers and systems such as Microsoft ThinkingBox were describing it as state-based or final-state evaluation.

Test yourself

Which of these is the meaning of state-based evaluation?

Related terms

eval a set of test cases used to measure how well a model or AI feature performs on the tasks you… benchmark a public, standardized test used to compare AI models, such as a set of exam questions, coding… AI agent an AI system that can pursue a goal over many steps on its own, deciding what to do next… tool use a model's ability to call external tools, such as a search engine, calculator, code runner, or… agent loop the repeating cycle at the heart of an agent: the model decides on an action, a tool carries… human-in-the-loop describing a system in which a person reviews, approves, or corrects an AI's work at key…

Cite this entry

"State-based evaluation." AI Dictionary, Dadgogo, https://dadgogo.com/dictionary/state-based-evaluation/. Accessed 5 Oct. 2026.