DadgogoAI Dictionary

Dictionary entry

pre-training

noun \ˌprē-ˈtrā-niŋ\

also pretraining

TopicTraining Senses1
Advertisement

Definition of pre-training

  1. : the first and most expensive stage of building a model, in which it learns general patterns from an enormous amount of unlabeled data, usually by predicting the next token

    • Pre-training gave the model its broad knowledge; later stages taught it manners.
    • A full pre-training run can cost hundreds of millions of dollars.

Test yourself

Which of these is the meaning of pre-training?

Related terms

base model a language model that has only been pre-trained, not yet tuned to follow instructions or chat… fine-tuning further training of an existing model on a smaller, focused dataset so it gets better at a… training data the collection of examples (text, images, code, audio, and so on) that a model learns from compute computing power, especially the chips and processing time used to train and run AI models post-training everything done to a model after pre-training to make it useful and safe, such as instruction…

Cite this entry

"Pre-training." AI Dictionary, Dadgogo, https://dadgogo.com/dictionary/pre-training/. Accessed 4 Oct. 2026.

Dictionary entries near pre-training

Advertisement