AutoTTS: Automated Discovery of Test-Time Scaling Strategies for Large Language Models
Large language models achieve better performance when you allocate more computation during inference—a technique called test-time scaling (TTS). But current TTS strategies rely on manual design: researchers hand-craft rules for when to branch reasoning paths, prune unpromising directions, or stop computation entirely.
AutoTTS changes this approach. Instead of designing individual TTS heuristics, you design environments where TTS strategies discover themselves automatically.
The Manual Design Problem
Existing TTS methods follow predictable patterns within a width-depth computation space:
- Width: How many reasoning branches to explore
- Depth: How far to develop each branch
Methods like Self-Consistency sample many branches at fixed depth. Answer Consistency extends single chains deeper. Parallel-Probe starts wide and progressively prunes while deepening.
Each represents a hand-crafted trajectory through the same underlying space. Researchers manually hypothesize branching rules, implement them, and tune thresholds by intuition—leaving most of the computation-allocation space unexplored.
Environment-Driven Discovery
AutoTTS shifts human effort from strategy design to environment construction. You define:
- States: Active branches, depths, probe signals, remaining budget
- Actions: Branch, continue, probe, prune, or answer
- Feedback: Accuracy-cost tradeoffs and execution traces
- Objectives: Maximize accuracy while controlling computation cost
An explorer LLM then searches this space automatically, proposing and refining controllers through iterative feedback.
Key Technical Components
Offline Replay Environment
Evaluating candidate controllers online requires expensive LLM calls for each test. AutoTTS pre-collects reasoning trajectories and probe signals offline, then replays controller decisions against this stored data.
For each question, the system samples 128 independent reasoning trajectories segmented into 500-token intervals. Controllers execute against this pre-collected data rather than invoking the LLM, making evaluation deterministic and cheap.
Beta Parameterization
Early experiments revealed that agents propose controllers with excessive hyperparameters—up to 10 parameters that create an intractable search space. With limited discovery rounds, agents collapse onto extreme solutions that minimize cost on the search set but fail to generalize.
Beta parameterization constrains each controller to expose only one scalar trade-off parameter β. The controller implements a monotone mapping from β to all internal hyperparameters, where larger β corresponds to larger token budgets. This reduces overfitting and makes the search space one-dimensional.
Execution Trace Feedback
Scalar accuracy-cost metrics reveal whether a controller works but not why it fails. AutoTTS augments feedback with complete decision-making trajectories showing how controllers allocate computation over time.
This fine-grained behavioral evidence enables the explorer to diagnose failure modes—such as overly aggressive pruning—and propose targeted improvements in subsequent rounds.
Implementation Example
The discovery loop runs for five rounds using Claude as the explorer agent:
- Proposal: Agent reads accumulated history and proposes improved controller code
- Evaluation: Controller runs on search questions via offline replay
- Feedback: System records accuracy-cost curves and execution traces
- Selection: Choose controller achieving highest accuracy on search set
The final controller generalizes to held-out benchmarks and model scales.
Results
Experiments on mathematical reasoning benchmarks show discovered strategies improve the accuracy-cost Pareto frontier over strong manual baselines including Self-Consistency, ASC, ESC, and Parallel-Probe.
The discovered controllers generalize from AIME24 (search set) to held-out AIME25 and HMMT25 benchmarks across four Qwen model scales (0.6B to 8B parameters). The entire discovery process costs only $39.9 and completes in 160 minutes.
Next Steps
AutoTTS demonstrates that environment-driven discovery offers a scalable alternative to manual TTS strategy design. The framework generalizes beyond width-depth spaces to richer structures like tree search or verifier-guided refinement.
Start by identifying your computation-allocation space, then design environments where effective strategies can emerge automatically. Focus human effort on environment construction, not strategy implementation.