AND recent paper by scientists from Google Research AND University of California, Berkeley, It shows that a surprisingly simple approach to scaling of test time can increase the ability to reason large language models (LLM). Key? Scaling of sample -based search, a technique that involves generating many answers and using the model itself to confirm them.
The basic discovery is that even a minimalist implementation of a search based on sampling, using random sampling and self -film, can increase the efficiency of the reasoning of models similar to Gemini 1.5 Pro, going beyond the review of O1 on popular reference points. Discoveries may have essential implications for the company’s applications and query the assumption that highly specialized training or complex architecture are all the time needed to attain the highest level performance.
Limits of the current calculation scaling of test time
The current popular approach to scaling the test time in LLM is the training of the model by learning to strengthen to generate longer answers with the traces of the chain (COT). This approach is used in models similar to OpenAI O1 and Deepseek-R1. Although they are helpful, these methods often require significant investments in the training phase.
Another approach to scaling the test time is “self -sufficiency”, in which the model generates many answers to the query and chooses the answer that appears more often. Self -thinning achieves its limits when solving complex problems, as in these cases, the most repetitive answer is not necessarily correct.
Sampling -based search offers a simpler and highly scalable alternative to scaling of the test time: Let the model generate many answers and select the best using the verification mechanism. Sampling -based search can complement other strategies for calculated test time and, as scientists write in their article: “It also has an exceptional advantage that it is embarrassing in parallel and enables arbitrary scaling: just try more answers.”
More importantly, searching based on sampling may be used to any LLM, including those who have not been clearly trained to justify.
How the search search for samples works
Scientists focus on the minimalist implementation of the sample -based search, using the language model to generate the response of candidates and their verification. It is a “self -drawing” process, in which the model assesses its own results, without relying on external soil answers or symbolic verification systems.
The algorithm works in a few simple steps:
1 – the algorithm begins with the generation of a set of candidate solutions of a given problem using a language model. This is done by repeatedly administration of a model of the same poem and using a non -zero temperature setting to create a varied set of answers.
2 – The candidate’s response undergoes the verification process in which LLM is monitored many times to find out if the answer is correct. The verification results are then averaged to create the end result of response verification.
3— The algorithm chooses the answer best assessed as the final answer. If many candidates are in close range, LLM is asked to check them in pairs and select the best. The answer, which wins the most comparisons in pairs, is chosen as the final answer.
Scientists considered two key axes for scaling of the test time:
Trying: The variety of answers generates a model for each input problem.
Verification: Number of verification results calculated for each generated solution
How sampling -based search is in comparison with other techniques
The study showed that the efficiency of reasoning is still improving with searching based on sampling, even when the calculation of the test time is scaled far beyond the point at which self -sufficiency is saturated.
On a sufficient scale, this minimalist implementation significantly increases the accuracy of reasoning for comparative tests similar to Aime and mathematics. For example, Gemini 1.5 Pro performance exceeded O1-P-Review performance, which was clearly trained in terms of reasoning problems, and Flash Gemini 1.5 surpassed Gemini 1.5 Pro.
“This not only emphasizes the importance of searching on sampling of scaling ability, but also suggests the usefulness of a sampling -based search as a straight base line, on which you can compare other strategies of calculated test time and measure the true improvement of models searching,” scientists write.
It is noteworthy that although the results of sampling based on search are impressive, the costs also can change into excessive. For example, at 200 attempts and 50 verification steps to sample, AME’s query will generate about 130 million tokens, which costs USD 650 from Gemini 1.5 Pro. However, this is a very minimalist approach to searching for sampling and is in line with the optimization techniques proposed in other studies. Thanks to smarter sampling methods and verification, the costs of propellers may be significantly reduced by using smaller models and generating fewer tokens. For example, using Gemini 1.5 flash to perform verification, the costs drop to $ 12 for the query.
Effective self -development strategies
There is a debate on whether LLM can confirm their very own answers. Scientists have identified two key strategies for improving self -zerity using the test time calculating:
Directly comparing candidates to the answer: Misunderstandings between candidates definitely indicate potential errors. By providing the verifier with many answers to check, the model can higher discover errors and hallucinations, dealing with the basic weakness of LLM. Scientists describe this as an example of “implicit scaling”.
Rewriting tasks: Scientists suggest that the optimal LLM starting style depends on the task. You think reflection is effective in solving the tasks of reasoning, but the answers are easier to confirm when they are written in a more formal, mathematical conventional style. Verifiers may prescribe candidates’ answers to a more structured format (e.g. Lemma proof against theorem) before evaluation.
“We expect that Self -development models in a short period are quickly improved, because the models learn to use the principles of closed scaling and the usefulness of the starting style and increase scaling indicators for samples -based search,” scientists write.
Implications for applications in the real world
The study shows that a relatively simple technique can achieve impressive results, potentially reducing the need for complex and expensive models of architecture or training regimes.
It is also a scalable technique that allows enterprises to extend efficiency by assigning a larger variety of computing resources for samples and verification. It also enables programmers to push the border language models beyond their restrictions on complex tasks.
“Considering that it complements other strategies of calculated test time, it is parallel and allows arbitrary scaling and admits simple implementation, which are clearly effective, we expect that samples -based search play a key role, because language models are designed to solve more and more complex problems with increasingly complex calculation budgets,” scientists write.
