DadgogoAI Dictionary

Dictionary entry

mixture of experts

noun \ˈmiks-chər əv ˈek-ˌspərts\

also MoE

TopicModels & Behavior First known use1991 Senses1
Advertisement

Definition of mixture of experts

  1. : a model design in which many smaller sub-networks ("experts") exist, but only a few are switched on for each token, making large models cheaper to run

    • The MoE model has 600 billion parameters but uses only about 40 billion per token.
    • Mixture of experts is a big reason open models got so efficient.

Origin & history

first known use1991

Described in a 1991 paper by Robert Jacobs, Michael Jordan, Steven Nowlan and Geoffrey Hinton, and revived for large language models around 2017.

Test yourself

Which of these is the meaning of mixture of experts?

Related terms

parameter one of the numbers inside a model that is adjusted during training; the total count of… inference the act of running a trained model to produce an output, such as answering a prompt or… large language model a very large neural network trained on huge amounts of text to predict the next token, which…

Cite this entry

"Mixture of experts." AI Dictionary, Dadgogo, https://dadgogo.com/dictionary/mixture-of-experts/. Accessed 4 Oct. 2026.

Dictionary entries near mixture of experts

Advertisement