DadgogoAI Dictionary

Dictionary entry

throughput

noun \ˈthrü-ˌpu̇t\
TopicModels & Behavior Senses1
Advertisement

Definition of throughput

  1. : how much work an AI system can handle in a given time, often measured in tokens per second

    • Batching requests doubled our throughput.
    • The chip is built for throughput, not low latency.

Test yourself

Which of these is the meaning of throughput?

Related terms

latency the delay between sending a request to a model and getting a response, often measured as time… inference the act of running a trained model to produce an output, such as answering a prompt or… GPU graphics processing unit: a chip originally designed for video games that can do many…

Cite this entry

"Throughput." AI Dictionary, Dadgogo, https://dadgogo.com/dictionary/throughput/. Accessed 4 Oct. 2026.

Dictionary entries near throughput

Advertisement