When a language model samples several candidate answers to the same question, the usual test-time strategies pick one of them: majority voting keeps the most common answer, and best-of-N keeps whichever a verifier or reward model scores highest. Both are selection strategies, so they can only ever return an answer that is already in the pool, and they fail on hard questions where every sample is wrong in a different place. Synthesis takes the opposite stance: a separate model reads all the candidates together, treating each chain of thought as evidence rather than as a vote, reconciles where they agree, diagnoses where each one goes wrong, and writes a new answer that need not match any candidate. Because complementary partial information is spread across mistaken samples, a small synthesizer model trained for this job can raise the accuracy of outputs from a much larger policy model, and its gains grow with the number of candidates instead of saturating the way voting does.
Current inference scaling methods, such as Self-consistency and Best-of-N, have proven effective in improving the accuracy of LLMs on complex reasoning tasks. However, these methods rely heavily on t…