Model merging – Definition & Meaning

Definition of model merging

: the creation of one model by mathematically combining the parameters, parameter differences, adapters, or checkpoints of two or more trained models, often without jointly retraining them on their original data
- The model merge combined a coding specialist and a general instruction model that shared the same base checkpoint.
- A benchmark revealed that the merged model gained both skills but lost accuracy on multilingual prompts.

Origin & history

Techniques for averaging and combining model parameters predate large language models, but model merging became a distinct and widely used LLM practice in the early 2020s through methods such as task arithmetic, TIES-Merging, adapter merging, and community tools that combine compatible checkpoints.

Test yourself

Which of these is the meaning of model merging?

Related terms

weights the learned numbers stored in a trained neural network that determine how strongly each part… fine-tuning further training of an existing model on a smaller, focused dataset so it gets better at a… LoRA low-rank adaptation: a cheap fine-tuning method that freezes the original model and trains… distillation training a smaller "student" model to imitate the outputs of a larger "teacher" model… open weights describing a model whose trained weights are published so anyone can download, run, and modify… mixture of experts a model design in which many smaller sub-networks ("experts") exist, but only a few are…

Cite this entry

"Model merging." AI Dictionary, Dadgogo, https://dadgogo.com/dictionary/model-merging/. Accessed 8 Oct. 2026.