DadgogoAI Dictionary

Dictionary entry

attention

noun \ə-ˈten(t)-shən\

also self-attention or attention mechanism

TopicFundamentals First known use2014 Senses1
Advertisement

Definition of attention

  1. : a technique that lets a model weigh how relevant every part of the input is to every other part, so it can connect related words even when they are far apart

    • Attention lets the model work out that 'it' in a long sentence refers to 'the server'.
    • The cost of attention grows quickly as the input gets longer.

Two meanings, one word

In everyday English

The act of noticing or concentrating on something.

In AI

A technique that lets a model weigh how relevant every part of the input is to every other part, so it can connect related words even when they are far apart.

Origin & history

first known use2014

Proposed for machine translation in 2014 by Dzmitry Bahdanau and colleagues, then made the centerpiece of the transformer in 2017.

Test yourself

Which of these is the meaning of attention?

Related terms

transformer a neural network design that processes a whole sequence at once and uses attention to work out… context window the maximum amount of text, measured in tokens, that a model can consider at one time… KV cache memory that stores intermediate attention results for text a model has already processed, so…

Cite this entry

"Attention." AI Dictionary, Dadgogo, https://dadgogo.com/dictionary/attention/. Accessed 4 Oct. 2026.

Dictionary entries near attention

Advertisement