We're releasing Open-o3, a new open-source (MIT license) inference framework that provides mathematically guaranteed improvements to the accuracy of any Large Language Model (LLM).
The core idea of the framework came from news reports about o3, and we learned that o3 would use a surprising number of tokens, so we came up with the idea.
It leverages probabilistic resampling, ranking, and verification, drawing inspiration from the law of large numbers and combinatorial optimization. Code, detailed derivations, and benchmarks are on GitHub.

The Problem: LLM inference is inherently probabilistic. Single-shot results are often unreliable, especially for tasks requiring high precision.

Our Solution: Resampling, Ranking, and the Exponential Decay of Error

Open-o3 treats an LLM as a statistical oracle. Even if the model has a low probability, p, of producing the correct answer on a single attempt, repeated independent sampling, combined with a verification procedure, dramatically increases the overall probability of success.

Mathematical Foundation:

Let:

p = Probability of a single sample being correct.
q = 1 - p = Probability of a single sample being incorrect.
n = Number of independent samples.
The probability of obtaining at least one correct answer after n trials is:

P(success after n trials) = 1 - P(all n trials are incorrect)
= 1 - q^n
= 1 - (1 - p)^n
This is a direct application of the law of large numbers. The crucial point is the exponential decay of the error probability:

P(error after n trials) = q^n = (1 - p)^n = e^(n * ln(1 - p))
Since ln(1 - p) is negative for 0 < p < 1, the error probability decreases exponentially with n. This means we can achieve arbitrarily high accuracy, given sufficient samples.

Rate of Convergence:

The error probability, q^n, can be expressed as e^(n * ln(q)). Because ln(q) is negative (since 0 < q < 1), this is of the form e^(-Θ(n)). Therefore, the performance gain, Δ = 1 - q^n, converges to 1 at an exponential rate determined by n.

Key Innovations:

Statistical Independence: We employ temperature annealing and entropy maximization techniques to ensure that each sample is as statistically independent from previous samples as possible. This is critical for the mathematical guarantees to hold.
Efficient Verification: Open-o3 uses lightweight verifiers (e.g., self-consistency scoring, or simple logical checks) to rank candidate answers. This is a crucial area for optimization and we welcome contributions! The verifier provides a function V: Answer -> Score.

The Tradeoff (and Why It Matters):

Yes, this increases compute costs linearly. But for mission-critical applications where accuracy is paramount (think medical diagnosis, legal reasoning, code generation), the tradeoff is often worthwhile. It allows you to leverage less powerful (and cheaper) models to achieve the same accuracy as much larger models.

Ethical Considerations:

We're upfront about the potential downsides:

Compute Inequity: More resources lead to better results. This is a general issue with LLMs, but it's amplified by resampling.
Energy Consumption: More samples mean more energy. We encourage exploring carbon-aware scheduling and other mitigation strategies.

We're releasing Open-o3 under the MIT license because we believe in open, collaborative development. We're particularly interested in contributions in these areas:

Extending to Multimodal Models: The current implementation focuses on text, but the core principles apply to other modalities.
Developing Fairness-Aware Early Stopping: How can we balance accuracy gains with resource constraints in a fair way?
Benchmarking on your use cases: We want to see how it performs on a wide range of tasks.
We'd love your feedback, contributions, and bug reports! Github:https://github.com/ChihayaYuka/Open-o3