Self-consistency & sampling-based reasoning

For NAO + anyone who has wondered why "just sample the model ten times and vote" makes it noticeably smarter — and where that trick stops working. The one-sentence version: a single greedy answer commits to one reasoning path, but a hard problem has one right answer reachable by many routes and many wrong answers scattered across the rest, so drawing several samples and aggregating them recovers signal that any one draw would miss. This article is about the aggregation layer: majority vote, best-of-N, the coverage / pass@k vs cons@k distinction, when sampling beats greedy, and how accuracy scales (and saturates) with the number of samples. Sourced from the original self-consistency paper, Large Language Monkeys (repeated-sampling scaling), Universal Self-Consistency, Snell et al. on compute-optimal test-time scaling, and PRM800K on verifier-based selection.

This is the aggregation sibling of two existing articles. [[sampling-and-decoding-strategies]] covers the layer below this one — how a single token is drawn (temperature, top-p, min-p); this article assumes you already have a sampler and asks what to do with N whole answers. [[test-time-compute-reasoning]] covers the layer around it — the o1/R1 program of training a model to reason; self-consistency is its simplest, training-free member. Related: [[verifiers-and-reward-hacking]], [[process-outcome-rewards]], [[mixture-of-agents]], [[generative-verifiers]].

What it is (intuition first)

Decoding gives you one knob most people never turn past "greedy": at temperature 0 the model takes the single most-likely token at every step and produces one deterministic answer. That answer is a single sample from the model's belief — and on a hard reasoning problem, the single most-probable path is not reliably the correct path.

The fix is embarrassingly simple. Raise the temperature so the model produces diverse answers, draw k of them, and aggregate:

Majority vote (self-consistency): extract the final answer from each of the k chains, return the one that appears most often.
Best-of-N: score each candidate with a verifier / reward model and return the highest-scoring one.
Coverage / pass@k: the looser question of whether any of the k samples is correct — what you'd get with a perfect oracle selector.

Why aggregation helps is a signal-vs-noise argument. Picture the answers each chain lands on as darts. On a problem the model partly understands, the correct answer is a single attractor that many independent reasoning paths converge to, while the wrong answers are diffuse — each buggy path fails differently. Vote, and the concentrated correct answer wins; the scattered errors cancel. This is why self-consistency works without any extra training, verifier, or labels — it is pure sampling plus a Counter.

The crisp framing from Wang et al. (2022): treat the reasoning chain r as a latent variable and the answer as what you get after marginalising it out.

answer* = argmax_a  Σ_i  1[ answer(chain_i) == a ]        # majority vote over k samples
        ≈ argmax_a  P(a | question)                       # marginalise over reasoning paths r

Greedy decoding picks the most-likely path; self-consistency approximates the most-likely answer. Those are not the same thing, and the gap between them is the free accuracy on the table.

Why it matters

It is the cheapest test-time-compute method that exists. No verifier, no RL, no reward model — just sample more and count. Wang et al. report +17.9% on GSM8K, +11.0% on SVAMP, and +12.2% on AQuA over greedy chain-of-thought, purely from majority-voting sampled chains (arXiv:2203.11171). It is the first thing to try before anything fancier.
It converts an inference budget into accuracy with a clean curve. Large Language Monkeys (Brown et al., 2024) shows that coverage — the fraction of problems solved by at least one of k samples — scales with the number of samples across four orders of magnitude, and the relationship is "often log-linear and can be modelled with an exponentiated power law." Most strikingly, on SWE-bench Lite with DeepSeek-Coder-V2-Instruct, the fraction of issues solved rises from 15.9% with one sample to 56% with 250 samples, beating the single-sample SOTA of 43% (arXiv:2407.21787). Repeated sampling is a second scaling axis alongside model size.
It substitutes inference compute for parameters. Snell et al. show that on problems a small model can sometimes solve, spending the budget on samples + selection can beat a 14× larger model under matched FLOPs (arXiv:2408.03314).
It is the conceptual bridge to councils. Self-consistency is "a council of samples from one model"; [[mixture-of-agents]] / fusion is "a council of different models." Same aggregation idea, different source of diversity.

How it works (the real mechanics)

1. The aggregators, from weakest to strongest

There is a ladder of selection methods, trading off how much extra machinery they need against how well they scale with k:

Method	What it needs	Strength	Where it breaks
Majority vote (self-consistency)	parseable final answer	free, no labels	saturates; ties; needs exact-match answers
Weighted majority vote	per-chain confidence / log-prob	slightly better calibration	confidence is poorly calibrated
Best-of-N + ORM	outcome reward model	scores the answer	rewards right answers from lucky-wrong paths
Best-of-N + PRM	process reward model	scores each step, enables pruning	expensive labels; hackable
Coverage / pass@k (oracle)	a ground-truth checker	upper bound — what any selector could reach	only exists for verifiable tasks

The gap between majority vote and coverage is the headline finding of the repeated-sampling literature: a perfect selector would solve far more than the voter does, so selection, not generation, is the bottleneck once you have many samples (more below).

2. Coverage / pass@k vs cons@k vs maj@k — don't conflate the metrics

These three report wildly different numbers from the same k samples, and papers exploit the ambiguity:

pass@k / coverage — correct if any of the k samples is correct. This is an oracle upper bound: it assumes a perfect verifier picks the winner. It is the right metric only when you genuinely have a checker (unit tests, a math grader) at inference time.
maj@k / cons@k (self-consistency) — correct if the majority-voted answer is correct. This is the realistic, verifier-free number. It is always ≤ pass@k, often far below it.
pass@1 — single greedy/sampled answer.

A model can have pass@256 = 56% but maj@256 = 30%: the right answer is in there, but majority vote can't find it because the wrong answers, while individually diffuse, collectively outvote a rare-but-correct path. Always check which metric a "scaling" plot uses — a beautiful pass@k curve says nothing about deployable accuracy if you have no oracle.

3. The verifier gap — why selection is the real ceiling

Large Language Monkeys makes the sharpest point: in domains with automatic verifiers (math with a grader, code with tests), coverage scales smoothly to hundreds of samples. In domains without them, "majority voting and reward models ... plateau beyond several hundred samples and fail to fully scale with the sample budget" (arXiv:2407.21787). So:

Verifiable task → coverage is harvestable; pile on samples + run the checker.
Unverifiable task → you are capped at what majority vote / a learned reward model can pick out, which saturates early.

This is why so much of the reasoning frontier (o1/R1, see [[test-time-compute-reasoning]]) lives on math and code: the verifier exists, so both the training reward and the test-time selector are clean. See [[verifiers-and-reward-hacking]] and [[process-outcome-rewards]] for the reward-model side of this story.

4. Free-form answers — Universal Self-Consistency (USC)

Classic majority vote needs answers that match exactly — fine for "the answer is 42", useless for a summary or an essay where every sample is phrased differently. Universal Self-Consistency (Chen et al., 2023) replaces the Counter with the LLM itself: paste all k candidates into one prompt and ask the model to pick the most consistent one. It "matches the standard self-consistency performance without requiring the answer formats to be similar" on math, matches execution-based voting on code without running the code, and extends the method to summarisation and open-ended QA where exact-match voting is impossible (arXiv:2311.17311). The catch: it is itself an LLM judge, so it inherits LLM-judge biases (length, position, self-preference) — see [[generative-verifiers]].

5. Temperature: the diversity dial that feeds aggregation

Self-consistency requires diversity — at temperature 0 all k samples are identical and the vote is a no-op. So you sample at temperature > 0. But there is a sweet spot, not a "hotter is better" rule:

Too cold → samples collapse to the same chain; the vote adds nothing.
Too hot → chains derail into incoherence (you draw from the unreliable tail, see [[sampling-and-decoding-strategies]]); individual answers get worse faster than diversity helps.

The useful mental model: temperature controls the diversity that aggregation feeds on, and you want the highest temperature at which individual chains are still mostly competent. Common practice for self-consistency is a moderate temperature (often ~0.5–0.8 with top-p) — but this is a tuned hyperparameter per model and task, not a universal constant, so flag any specific number as empirical and re-tune it.

Key ideas & tradeoffs

Sampling amplifies latent ability; it doesn't create it. If the correct answer is never in the support of the model's distribution — pass@k stays 0 no matter how large k — voting cannot conjure it. Snell et al.: test-time sampling beats a bigger model only where the base model already has non-trivial success. Sampling is an amplifier, not an oracle.
Selection is the bottleneck, not generation. The coverage-vs-maj@k gap means the bigger lever, once you can afford many samples, is a better selector (verifier, PRM, generative verifier), not more samples. This is exactly Snell et al.'s "compute-optimal" point.
Parallel (sample + vote) vs sequential (one chain that backtracks). Self-consistency is parallel test-time compute — trivially scalable but votes saturate. A trained reasoner ([[test-time-compute-reasoning]]) does sequential compute inside one chain, which is more sample-efficient but needs the RL training. They compose: sample several long reasoning chains and vote/verify.
Cost is linear in `k`, accuracy is log-ish. You pay k× the tokens for a log-linear (eventually saturating) accuracy gain — fine for a hard one-shot problem, ruinous for high-QPS chat. This is why [[model-routing]] matters: only spend a big sample budget where the task is hard and verifiable.
Ties and even `k`. Majority vote needs a tie-break; weighting by per-chain confidence or log-prob is the usual fix, but model confidence is poorly calibrated, so the gain is modest.

Honest caveats & open questions

Metric laundering is rampant. A lot of "test-time scaling" hype reports pass@k (oracle) while quietly implying it's deployable accuracy. Without a verifier you live on the maj@k curve, which saturates far earlier. Read every scaling plot for which metric and whether an oracle was assumed.
Saturation is real and early on unverifiable tasks. Large Language Monkeys itself reports vote/reward-model selectors plateauing "beyond several hundred samples." More samples eventually buy nothing; the curve is not the clean log-line it looks like at small k.
Diversity-vs-correctness is a genuine tension. The temperature that maximises coverage (you want spread to hit the right answer at least once) can be higher than the one that maximises majority-vote accuracy (you want each chain competent). The optimum differs by metric — there is no single best temperature.
USC inherits LLM-judge pathologies. Using the model to pick the "most consistent" answer reintroduces position bias, length bias, and self-preference; a generative verifier can be confidently wrong in a way exact majority vote cannot. See [[generative-verifiers]] and [[verifiers-and-reward-hacking]].
Self-consistency assumes errors are independent and diffuse. When a model has a systematic bias — every chain makes the same wrong assumption — they all vote for the same wrong answer, and self-consistency confidently confirms it. Voting cancels random error, not shared error.
Faithfulness. As with all chain-of-thought, the written reasoning need not be the true cause of the answer; a chain can reach the right answer by a wrong route and still cast a "correct" vote, inflating apparent reliability.

How it connects to OpenAlice

Cheapest reasoning upgrade for verifiable agent tasks. OpenAlice's stated AGI direction includes autonomous repo→PR coding measured on SWE-bench — and SWE-bench is the exact setting where Large Language Monkeys showed repeated sampling jumping 15.9% → 56% with a verifier (the test suite). A passing test suite is a ready-made oracle selector: sample many patches, keep the ones that pass. This is a high-leverage pattern for the agentic-coding loop ([[agentic-loops]], [[agentic-rl-long-horizon-coding]]) before any expensive RL.
Council + verifier hybrids. Self-consistency (samples of one model) and [[mixture-of-agents]] (different models) are the same aggregation idea; pairing either with an outcome/process check ([[process-outcome-rewards]]) is the compute-optimal recipe Snell et al. describe. Worth prototyping in the lab/bench rig before prod wiring.
Route the budget, don't spend it everywhere. k× cost is fine for a hard Worker-mode turn (a diploma, a proof, a tricky patch) and ruinous for casual Companion chat. OpenAlice's per-chat mode axes (Companion vs Worker) are a coarse version of "decide how many samples this turn deserves" — the [[model-routing]] problem.
Free-form aggregation for Alice's open-ended replies. Most of Alice's outputs are not exact-match answers, so plain majority vote doesn't apply — USC-style LLM-as-selector ([[generative-verifiers]]) is the relevant tool, with its judge biases noted.

Self-consistency & sampling-based reasoning — majority vote, best-of-N, coverage vs pass@k, and when many samples beat one greedy answer