Evaluation awareness – Definition & Meaning

Definition of evaluation awareness

: the ability of an AI model to infer that it is being tested, benchmarked, or audited rather than used in an ordinary deployment, potentially allowing it to change its behavior in response to the evaluation setting
- Evaluation awareness made the researchers question whether the model would behave just as safely outside the audit.
- The model mentioned the benchmark in its reasoning, but that verbalized evaluation awareness did not prove that it was deliberately hiding a capability.

Origin & history

first known use2025

The capability was defined and measured directly in the 2025 study Large Language Models Often Know When They Are Being Evaluated. It became a prominent frontier-model evaluation issue through subsequent system cards, alignment research, and dedicated studies in 2025 and 2026.

Test yourself

Which of these is the meaning of evaluation awareness?

Related terms

eval a set of test cases used to measure how well a model or AI feature performs on the tasks you… benchmark a public, standardized test used to compare AI models, such as a set of exam questions, coding… alignment the work of making an AI system's goals and behavior match what its designers and users… trajectory-level monitoring the practice of evaluating an AI agent's full sequence of observations, reasoning, tool calls… chain of thought a series of intermediate reasoning steps a model writes out before its final answer, either… red teaming deliberately attacking or stress-testing an AI system, often with outside experts, to find…

Cite this entry

"Evaluation awareness." AI Dictionary, Dadgogo, https://dadgogo.com/dictionary/evaluation-awareness/. Accessed 4 Oct. 2026.