OpenAI announces o3. What a benchmark jump does and does not tell you
Announced on 20 December 2024 with striking scores on a reasoning benchmark. Announced, not released — and the gap between those words is the whole point.
On 20 December 2024, closing its run of daily announcements, OpenAI presented o3 and o3-mini: new reasoning models, not yet generally available, with results on the ARC-AGI benchmark far above anything previously reported for a language model.
ARC-AGI is a test of abstract pattern reasoning designed to be easy for people and hard for machines. Models had struggled with it for years. The reported jump is genuinely notable, and the reaction in December has run well ahead of what is known.
Three things worth holding onto
It was announced, not shipped. o3-mini was indicated for the following weeks and the larger model later. A demonstration is not a product you can buy, price, or put in a process.
The high scores came at high compute. The most striking results were produced in a configuration that consumes an enormous amount of computation per task. That is a research result, not an economic one. What matters for you is what the affordable configuration does.
A benchmark is a benchmark. Abstract puzzle reasoning is not the same skill as reading your supplier contract or understanding why a delivery was short. Progress on one does not automatically transfer to the other.
What it does tell you
That the plateau reported in November was about one specific lever — scaling pre-training — and not about progress in general. Spending computation at answer time is working, and it is working faster than most people expected.
It also tells you that capability will keep arriving in steps rather than smoothly, which is an argument for short contracts and against building anything rigid around today’s limits.
The unglamorous conclusion
Nothing in your January plan should change because of this announcement. The bottleneck in a Swiss SME is not model capability, and it has not been for at least a year.
The bottleneck is knowing which of your processes is worth automating, having data tidy enough to feed it, and having decided who checks the result. Those three things are exactly as hard today as they were on 19 December, and no benchmark will move them.