Newsletter
Join the Community
Subscribe to our newsletter for the latest news and updates
Source: ARC Prize’s original Astra evaluation. Each blue point is a completed level; points below the solid diagonal used fewer actions than the median successful human. This chart measures action efficiency inside ARC-AGI-3, not real-world task reliability.
GPT-6 Astra ARC-AGI-3 results look like a breakthrough: 62.7% in the standard test and 99.9% with OpenAI’s provider adapter. The gap is the story. Astra became dramatically better when the test preserved private reasoning state and compressed long conversations, so the headline score measures a model-plus-harness system rather than the model alone.
GPT-6 Astra on ARC-AGI-3 is a benchmark result, not a declaration of AGI. In ARC Prize’s provider-neutral Standard harness, Astra scored 62.7% on the Semi-Private set at a reported cost of $26,098. In a separate Provider Adapter harness that retained opaque reasoning state and compacted long conversations, it reached 99.9% at about $19,000. ARC Prize says both are state of the art, but only the Standard setup supports cleaner comparisons across vendors. The benchmark places an agent in unfamiliar, turn-based worlds and measures whether it can explore, infer goals, model rules, and plan without explicit instructions. The same evaluation found Astra used fewer actions than the median successful human on 96% of completed levels. That is strong evidence of efficient adaptation inside these games. It is not evidence that Astra can handle the open-ended uncertainty, stakes, or messy feedback of real software and browser work.
The model did not simply receive more “thinking time” under the same test. The surrounding system changed. In the Standard harness, Astra decides which visible notes survive between interactions. In the Provider Adapter, OpenAI’s integration preserves hidden reasoning state and summarizes long histories so the model can reuse prior work.
That difference can remove a common agent failure: losing the discoveries, constraints, and partial plans needed to finish a long task. ARC Prize reports that, across 167 game-reasoning pairs solved by both setups, Provider Adapter runs were about 3.66 times faster by aggregate elapsed time and used 49% fewer total tokens.
Source: ARC Prize’s original leaderboard image. The yellow Standard-harness line and the separate Provider Adapter point are different evaluation conditions. The chart is vendor-reported model access inside ARC Prize’s testing framework, not an independently reproduced production workload.
The comparison is useful precisely because the labels stay separate. The Standard number says more about cross-provider model performance. The adapter number says more about the best observed model-plus-context system. Developers need both questions, but should not collapse them into one leaderboard claim.
ARC-AGI-3 puts agents into unfamiliar visual worlds without explaining the goal or controls. An agent must experiment, infer causal rules, set a target, and execute a plan. ARC Prize established its human action baseline with roughly 500 members of the general public and used the median action count among people who completed each level.
In the Provider Adapter run, Astra beat that median on 96% of completed levels. It also averaged 51.7% fewer actions per level. This does not mean it used less compute than a brain or understood the world like a person. It means that once it found a workable model of a game, it often needed fewer environment interactions to complete it.
The replays suggest one mechanism. Astra compressed a scene into a small, code-like record: object positions, control coordinates, mechanism lengths, current state, and an ordered plan. ARC Prize calls it an on-the-fly algebraic shorthand rather than a full programming language. In the PRO-LONG setup, which added a code sandbox, Astra also wrote parsers, search routines, planners, and state-checking scripts. Those runs measure the model and its tools together, and ARC Prize says they are not comparable with the human test conditions.
The practical lesson is larger than one benchmark. Context management is part of agent performance. A model can look mediocre when it repeatedly loses state and excellent when the surrounding system preserves the right state in the right form.
If you are evaluating Astra or any long-running agent, log at least four things separately: task completion, environment actions, model calls, and tokens. Then test the same tasks with visible notes only, automatic compaction, and any provider-native state mechanism. A higher completion rate may come from the model, the harness, or their interaction.
Do not assume the highest reasoning setting always costs more. ARC Prize’s Standard-harness runs showed Astra’s reported cost falling as reasoning effort rose because the model completed games in fewer actions, reducing subsequent calls. That pattern is workload-specific, but it is worth testing on your own long-horizon tasks.
The exciting part is not the near-perfect number. It is Astra’s reported ability to turn an unfamiliar interactive system into a compact working model and then act efficiently. That could matter for computer-use agents, research workflows, and coding tasks where the next step depends on what happened many actions ago.
The reason to wait is equally concrete. A deterministic puzzle has stable rules, immediate feedback, and a closed goal. A real browser can change underneath an agent, hide permissions, introduce ambiguous text, or trigger irreversible actions. The benchmark does not tell us how Astra handles those conditions.
Use a small evaluation that exposes where the gain comes from:
My read is that Astra deserves a serious test if your agent currently fails because it forgets state or takes too many exploratory actions. Teams that need a portable model ranking should wait for additional provider-neutral evaluations and real-workload reproductions.
No. ARC Prize explicitly says that saturating ARC-AGI-3 is not proof of AGI. The test captures valuable ingredients—exploration, modeling, goal setting, planning, and action efficiency—but compresses them into deterministic environments with closed-ended mechanics and goals.
The 99.9% result is also tied to a provider-specific context system. That does not make it invalid; many useful products are systems, not isolated models. It does mean the honest claim is narrower: Astra plus its adapter nearly saturated this benchmark under that setup.
TechCrunch adds another reason for caution. OpenAI says Astra uses opaque recurrence, which can reduce the language-token trace available for monitoring. Better internal state may improve performance while making the path to an action harder to inspect. Product teams should measure both capability and observability.
My take: the benchmark is meaningful evidence of better interactive reasoning, and the 36-point harness gap is meaningful evidence that model rankings alone are incomplete.
What is GPT-6 Astra’s ARC-AGI-3 score?
ARC Prize reports 62.7% on the Semi-Private set using its Standard harness and 99.9% using an OpenAI Provider Adapter. Both are state-of-the-art results, but they use different context-management conditions and should be reported separately.
Why are there two GPT-6 Astra scores?
The Standard harness preserves visible notes selected by the model. The Provider Adapter retains opaque reasoning state between requests and compacts long conversations. The second setup helps Astra reuse prior work, so it measures a different system.
Did GPT-6 Astra beat humans on ARC-AGI-3?
On action efficiency for completed levels, yes: ARC Prize says Astra used fewer actions than the median successful human on 96% of levels. That does not mean it surpassed humans across intelligence, reliability, compute efficiency, or open-ended real work.
Does a 99.9% ARC-AGI-3 score prove AGI?
No. ARC Prize says the benchmark has deterministic, closed-ended worlds and does not represent the complexity of the real world. The result demonstrates strong adaptation in that bounded setting, not general competence across human tasks.
Should developers switch agent workloads to Astra now?
Test it if state loss or excessive exploration is a major failure mode. Keep a provider-neutral baseline, measure completion and cost together, and add changing conditions plus recovery tests before trusting the result with consequential actions.
Discover practical AI products and emerging developer tools at AIToolHunt.
