Model Merging & Souping (task arithmetic, TIES, DARE, SLERP, Frankenmerging)

One-line summary. You can take two or more neural networks that started from the same pre-trained checkpoint, were fine-tuned separately, and then average or arithmetic-combine their weights directly — no gradient steps, no training data, no GPU cluster — and the result often inherits the abilities of all of them. This sounds like it should not work (averaging two trained networks "should" give garbage), and for unrelated networks it does. The trick is that fine-tunes of a shared base live in the same loss basin, so the straight line between them stays low-loss. Merging is the cheapest capability-composition technique in the stack — a few minutes of CPU instead of a training run — which is exactly why the open-weights community lives on it.

What it is (intuition first)

Picture a pre-trained model as a town square. Every fine-tune you do — one for code, one for math, one for Japanese, one for politeness — walks off in a slightly different direction but never goes very far, because fine-tuning barely perturbs the weights (the delta is tiny; DARE's authors measured SFT weight changes mostly within ±0.002). So all your fine-tunes end up clustered in the same valley of the loss landscape. And here is the surprising empirical fact: inside that valley, the straight line between two fine-tunes is also low-loss. This is called linear mode connectivity. It means you can stand at the midpoint — literally (W_A + W_B) / 2, the elementwise average of the two weight tensors — and land on a model that works, often better than either parent.

Contrast this with ensembling, the classic way to combine models: ensembling keeps all N models around and runs them all at inference, then averages their outputs. That triples your latency and memory for 3 models. Merging averages the weights instead of the outputs, collapsing N models into one model with the same inference cost as a single one. You pay the combination cost once, offline, and serve a single artifact forever. That is the whole economic pitch.

The family of techniques splits into two philosophies:

1. Same-architecture weight combination — soups, task arithmetic, TIES, DARE, SLERP. All parents share one architecture and lineage; you blend the tensors.
2. Frankenmerging (layer stacking / passthrough) — you don't blend; you stack layers from different models to build a deeper, larger Frankenstein model (the "120B" community models built from a 70B by duplicating and interleaving layers). No shared lineage required, but also no guarantee it works — pure alchemy.

Why it matters

• It is nearly free. A merge is a weighted sum of tensors — seconds to minutes on a CPU. Compare that to a fine-tune (GPU-hours) or a pre-train (nine figures; see [[scaling-laws]]). Merging is the single highest leverage-per-FLOP move available to anyone who is not a frontier lab.
• It composes capabilities without catastrophic forgetting. Fine-tune model A on code and B on math independently and in parallel, then merge. You sidestep the sequential-fine-tuning problem where training on math erases the code skill — the two skills were learned in isolation and combined at the end.
• It powers the entire open-weights ecosystem. The top of every open LLM leaderboard for a stretch in 2024 was dominated by merges (often merges of merges), built with MergeKit by hobbyists with no training budget. This is real, measurable, and slightly absurd.
• It is a deployment-friendly cousin of MoE and routing. Where [[mixture-of-experts]] keeps experts separate and routes per-token, and [[model-routing]] picks a whole model per request, merging fuses the experts into one dense set of weights — no router, no extra params, one forward pass.

How it works (the real mechanics)

Model Soups — just average the fine-tunes (Wortsman et al. 2022)

The foundational result. Take a base model, fine-tune it many times with different hyperparameters (learning rates, seeds, augmentations), then average all the resulting weights into one "soup." Wortsman et al. showed this often beats the best individual fine-tune and the held-out-validation-selected one, while costing the same at inference as a single model (unlike an ensemble). Two flavours:

• Uniform soup — average everything, no selection.
• Greedy soup — sort candidates by val accuracy, add a model to the soup only if it doesn't hurt held-out accuracy. This is the one that reliably wins.

The mechanism is the loss-basin geometry above: all these fine-tunes share a base, so their average stays in the basin. Soups improved ImageNet and out-of-distribution robustness without a single extra training step over the fine-tunes you already ran.

Task Arithmetic — vectors you can add and subtract (Ilharco et al. 2023)

The conceptual leap. Define a task vector as the difference between a fine-tuned model and its base:

τ_task = W_finetuned − W_pretrained

This vector is a direction in weight space that "means" the task. The startling finding: these vectors behave like arithmetic objects.

• Addition — W_base + τ_A + τ_B gives a model good at both tasks A and B.
• Negation — W_base − τ_toxic removes a behaviour (forgets toxicity, a target task) while barely touching control tasks. Subtraction is unlearning.
• Analogy — "A is to B as C is to D": τ_D ≈ τ_B − τ_A + τ_C lets you build a model for task D using no data from D at all. (E.g. combine sentiment-on-reviews, language vectors to get sentiment-in-another-language.)

A scaling coefficient λ controls strength: W = W_base + λ · Σ τ_i. Soups are the special case where you add task vectors with λ = 1/N (averaging is task arithmetic with uniform small weights). Task arithmetic is the algebra that makes all later methods expressible as "operations on deltas."

The interference problem → TIES (Yadav et al. 2023)

Naive addition of task vectors degrades as you add more tasks, because the vectors fight each other. TIES-Merging diagnoses two failure modes and fixes each:

1. Redundant parameters. Most entries in a task vector are tiny noise that contributes nothing but dilutes the signal. → Trim: keep only the top-k% by magnitude per vector (e.g. top 20%), zero the rest.
2. Sign disagreement. For a given weight, vector A wants +0.01, vector B wants −0.01; summed they cancel to ~0 and both tasks lose. → Elect a sign: for each parameter, sum the (signed) magnitudes across all vectors and take the majority sign as the agreed direction.
3. Disjoint merge. Average only the values whose sign matches the elected sign; ignore the dissenters.

TIES = Trim, Elect Sign, Disjoint merge. It scales to more tasks far more gracefully than plain summation — interference, not capacity, was the bottleneck.

DARE — drop most of the delta, rescale the rest (Yu et al. 2023)

"Language Models are Super Mario." The observation: SFT delta parameters are extremely redundant — you can randomly drop 90% (or even 99%) of them and recover the rest by rescaling, with almost no quality loss. DARE = Drop And REscale:

mask each delta to 0 with probability p          # drop (e.g. p = 0.9)
multiply the survivors by 1 / (1 − p)            # rescale to preserve the expected sum

The rescale keeps the expected embedding-shift unchanged, so a single DARE'd model behaves like the original. The payoff comes when merging: pre-sparsifying every task vector with DARE before adding them dramatically reduces interference (most deltas are now zero, so collisions are rare). DARE is a plug-in — you compose it with TIES or task arithmetic (dare_ties, dare_linear in MergeKit). The framing — absorbing abilities "as a free lunch" — captures why the community went wild for it.

SLERP — interpolate on the sphere, not the line (two models)

Linear averaging walks the straight chord between two weight vectors, which can pass through a low-norm "dead zone" in the middle. Spherical linear interpolation walks the arc on the hypersphere of constant norm between them instead, preserving the geometric character of both parents:

SLERP(W_A, W_B; t) = sin((1−t)Ω)/sin(Ω) · W_A  +  sin(tΩ)/sin(Ω) · W_B
   where  Ω = angle between W_A and W_B,   t ∈ [0,1]

SLERP is the default two-model merge in MergeKit and is widely regarded as the "just works" baseline for a clean blend of two fine-tunes. Limitation: it is strictly pairwise — you can't SLERP five models at once (you'd chain them, which is order-dependent). For >2 models you fall back to soups/TIES/DARE.

Frankenmerging — stacking layers (passthrough)

The wild west. Instead of blending tensors, you concatenate layer ranges from different models into a new, deeper network:

new_model = layers 0–23 of model_X  ++  layers 8–31 of model_Y  ++ ...

This grows parameter count (the "self-merge" that turns a 7B into an 11B, or a 70B into a 120B) with no training. It frequently produces a coherent model — and frequently produces nonsense. There is little theory; it's empirical alchemy that sometimes wins benchmarks. MergeKit calls this the passthrough method.

MergeKit & evolutionary merging — the tooling and the search

[[mergekit]] (Arcee, Goddard et al. 2024) is the de-facto toolkit: it implements linear/soup, task-arithmetic, TIES, DARE, SLERP, and passthrough behind one YAML config, runs out-of-core on CPU or 8 GB of VRAM, and made merging accessible to anyone. The natural next question — which recipe and coefficients? — is itself an optimization. Sakana's Evolutionary Optimization of Model Merging Recipes (Akiba et al. 2024) used evolutionary search over both the parameter space (merge weights) and the data-flow space (which layers route where), automatically discovering merges like a Japanese LLM with math-reasoning ability, without training data for the combined skill.

Why any of this works — the permutation/basin caveat (Git Re-Basin)

The honest theoretical backbone: merging only works when the parents share a loss basin, which for fine-tunes of one base they do by construction. For models trained from scratch independently, naive averaging gives garbage — but Ainsworth et al.'s Git Re-Basin (2023) showed that neural nets have vast permutation symmetries (you can shuffle hidden units without changing the function), and if you first permute one model to align its units with the other, you can sometimes reach near zero-barrier linear mode connectivity even between independent runs. This is why "merge any two models" is not generally safe, and why the practical methods all quietly assume shared lineage.

Key ideas & tradeoffs

• Shared base is the load-bearing assumption. Every reliable merge method assumes the parents are fine-tunes of the same checkpoint. Merge two unrelated bases and you need Git-Re-Basin-style permutation alignment first — and even then it's fragile.
• Merging vs ensembling. Same combination intent, opposite cost profile: ensemble = N× inference, merge = 1× inference but lossy fusion. Merging trades a little quality for a lot of serving cost.
• Interference is the real enemy, not capacity. TIES and DARE both exist because summing deltas makes them cancel. Sparsify (DARE) and sign-align (TIES) and the same weights suddenly cooperate. The information was there; the collisions hid it.
• It is composable. dare_ties = DARE sparsification + TIES sign-election. The methods stack into a pipeline rather than competing.
• Pairwise vs many-way. SLERP is the cleanest two-model blend; soups/TIES/DARE scale to many. Pick by how many parents you have.
• No data, no labels, no gradients. The defining property — and the reason it feels like cheating — is that a merge needs zero training data. Contrast with [[distillation]] and [[lora-and-peft]], which still require a training loop.

Honest caveats & open questions

• Quality is not guaranteed and is hard to predict. Merging often helps and sometimes badly degrades; there's no reliable a-priori test for whether a given pair will merge well. The community workflow is empirical: merge, benchmark, repeat. Greedy-soup-style held-out validation is the only principled guard.
• Leaderboard merges and contamination. Because merges are cheap to mass-produce and tuned against public benchmarks, a merge that tops a leaderboard may be overfit to that benchmark rather than genuinely more capable — a [[benchmark-contamination]] hazard. Treat "best merge on the Open LLM Leaderboard" claims with the same skepticism as any benchmark-only result.
• Frankenmerging has essentially no theory. Layer-stacking that "works" is observed, not understood; results are anecdotal and the failure rate is high. The parameter-count increase does not reliably buy capability.
• Theory lags practice. Linear mode connectivity and permutation symmetry give a partial explanation for averaging, but task arithmetic's analogy behaviour and the exact conditions under which sign-election helps are still active research, not settled science.
• It cannot create genuinely new knowledge. Merging recombines what the parents already learned. It is composition, not learning — you cannot merge your way to a capability that none of the ingredients possessed (the Sakana cross-domain result is recombination of latent skills, not de-novo knowledge).
• Vendor/community numbers are often informal. Many merging "results" circulate as HuggingFace model cards and blog posts rather than peer-reviewed evals; the methods in the Sources are primary and solid, but specific merge-X-beats-Y claims in the wild are frequently unaudited. Flagged as contested.

How it connects to OpenAlice

OpenAlice is an orchestration project over frontier models (Codex/GPT-class via routing), not a weights lab — so we do not currently merge production weights. The relevance is conceptual and forward-looking:

• The composition philosophy mirrors our stack. Merging fuses specialist weights into one artifact; OpenAlice fuses specialist agents at the system layer — [[fusion-and-llm-councils]] and [[mixture-of-agents]] are the inference-time analogue of task arithmetic ("add the code expert + the math expert"), and [[model-routing]] is its discrete cousin (pick a whole model instead of blending). Reading merging clarifies which layer you're composing at: weights vs outputs vs whole-model selection.
• If Alice ever owns small open-weight specialists, merging is the cheapest way to give one local model several skills without a training run — directly relevant to the [[small-language-models]] / on-device direction and to LoRA-fusion (a merge of adapters; see [[lora-and-peft]]).
• It's a foundation node for the Academy. This article sits beside [[distillation]] and [[mixture-of-experts]] as one of the three main "combine/compress models without full retraining" techniques, and downstream of [[scaling-laws]] (merging is a way to get capability off the pretraining FLOP curve).

See also

• [[mergekit]] — the toolkit that implements every method in this article behind one YAML.
• [[mixture-of-experts]] — keeps experts separate + routes per-token; merging fuses them into one dense set.
• [[model-routing]] · [[fusion-and-llm-councils]] · [[mixture-of-agents]] — composing at the routing/output/agent layer instead of the weight layer.
• [[distillation]] — the other "make a capable smaller model" technique, but it does need a training loop.
• [[lora-and-peft]] — adapters you can also merge (LoRA-fusion); the cheapest deltas to combine.
• [[scaling-laws]] — why getting capability without a training run matters economically.
• [[benchmark-contamination]] — why "top-of-leaderboard merge" claims deserve skepticism.
• [[small-language-models]] — the on-device setting where merging specialists into one model pays off most.