Here is the number that should reframe how you deploy AI agents in the second half of 2026: the length of work an agent can do at coin-flip odds is roughly five times the length it can do reliably. METR’s time-horizon research measures this directly. It reports two numbers for every frontier model: the task length an agent finishes at 50% reliability, and the length it finishes at 80%. The second is about one-fifth of the first. An agent that can grind through most of a workday when you only need it right half the time holds up for a couple of hours when you need it right four times out of five.
That is the AI agent reliability gap, and it is the story hiding inside every “our agent ran for twelve hours” demo. The argument this post makes: teams keep sizing agent autonomy to the 50% horizon — the impressive number the demo shows — when production only pays for the 80% one. Closing the gap isn’t a matter of waiting for a smarter model. It’s a matter of structure. Lova is a chat-first AI project management product where AI agents work as teammates: they claim bounded tasks on a shared board, move them through defined states, and stop at checkpoints a human can see. A board is where you place the checkpoint at the edge of what the agent can actually be trusted to do — and turn a long, unreliable run into a sequence of short, trustworthy ones.
Key takeaways
- METR’s data shows an agent’s 80% reliability time horizon is roughly one-fifth of its 50% horizon — and the 80% number rises far more slowly. Capability is racing ahead of reliability.
- A February 2026 study, “Towards a Science of AI Agent Reliability,” evaluated 14 models and found that recent capability gains yielded only small improvements in reliability — accuracy and reliability decouple.
- Reliability compounds against you. An agent that clears each step 90% of the time finishes a ten-step task only about a third of the time (0.910 ≈ 35%). The gap isn’t the model having a bad day — it’s arithmetic.
- The novel frame: your trust horizon — the stretch of work you can hand an agent before you must check it — is the 80% horizon, not the 50% one. Size autonomy to the number you can’t see in the demo.
- The fix routes around the model instead of waiting on it: a shared board places a human checkpoint at the trust horizon, so a wrong turn surfaces as a blocked card before it becomes an expensive one.
What is the AI agent reliability gap?
The AI agent reliability gap is the distance between how long an AI agent can run and how long it stays right. METR quantifies it by measuring a model’s “time horizon”: the length of task — scored by how long the same job takes a human expert — that the agent completes at a given success rate. At 50% reliability, the horizon is long. At 80%, it collapses. For Claude 3.7 Sonnet, METR reported a 50% horizon of about 59 minutes against an 80% horizon of roughly 15 minutes: the usable task length shrinks four to five times the moment you raise the bar from “works half the time” to “works most of the time.”
The gap holds across the frontier, and it is widening in an awkward direction. The 50% number is climbing fast — METR’s time horizon has been doubling roughly every few months, faster than the seven-month doubling in the original 2025 data, with the strongest assessed model reaching around three hours at 80% reliability. But the 80% horizon moves more slowly than the 50% one. So the more capable models get, the wider the gap between what they can attempt and what you can trust unattended. This is the same benchmark-to-production pattern we’ve traced before: agents top the leaderboard, then stumble the moment the work runs long. We wrote about the deployment side of it in AI agent benchmarks vs production.
Why can an AI agent run for hours but only be trusted for one?
Because reliability compounds, and it compounds against you. A long agent run isn’t one decision — it’s a chain of them, and every link has to hold. An agent that is 90% reliable on a single step sounds excellent until you string ten steps together: the odds all ten land is 0.910, about 35%. Push per-step reliability to 95% and a twenty-step task still fails more often than a coin flip (0.9520 ≈ 36%). The math is unforgiving in exactly the region where autonomous agents are supposed to earn their keep: the long, multi-step jobs no human wants to babysit.
And the culprit isn’t raw intelligence. The “Towards a Science of AI Agent Reliability” authors evaluated 14 models across two benchmarks and found that capability gains — measured as accuracy — have not translated into proportional reliability gains. A single success-rate score, they argue, hides whether an agent behaves consistently across runs, withstands small perturbations, fails predictably, or keeps its errors bounded. That’s why a model can ace a benchmark and still be untrustworthy over a long horizon: the score measures whether it can, not whether it reliably will. METR’s own curve shows the cliff plainly — current models complete nearly 100% of tasks that take a human under about four minutes, and under 10% of tasks that take more than about four hours.
What is the trust horizon?
Here’s the frame worth keeping. Your trust horizon is the length of work you can hand an agent before a human has to look — and it is the 80% reliability horizon, not the 50% one. The 50% number is what the demo advertises: watch it run for hours, unattended, finishing something impressive. The 80% number is what production actually requires, because a workflow that’s right only half the time isn’t automation — it’s a coin you have to re-check by hand. Most agent deployments are quietly priced against the wrong horizon, which is a large part of why the pilot dazzles and the rollout disappoints.
The strategic move flips the usual advice. Most teams chasing agent ROI are waiting on the next model release to close the gap. The trust horizon says the faster lever is the one you already control: don’t hand the agent a job longer than its 80% horizon. Take the long-horizon goal it can’t reliably finish and cut it into segments it can — each short enough to land inside the trustworthy window, each with a checkpoint at the seam. That isn’t a prompt; it’s an operating surface. It’s the same reason the two-week sprint stopped being able to see agent work at all, which we unpacked in long-horizon AI agents and the end of the sprint.
How do you close the reliability gap without a smarter model?
You don’t close it — you route around it. A shared board turns the reliability gap from a property of the model into a feature of the workflow. Instead of one twelve-hour run you hope holds, the work becomes a sequence of bounded tasks, each sized to the trust horizon, each ending in a state a human can read: open, in progress, blocked, done. A dead end surfaces as a blocked card someone sees, not a silent loop compounding errors for another six hours. A definition of done gives the agent a bar to clear instead of a guess to make. The checkpoint isn’t friction — it’s the thing that lets you trust the agent with the next segment.
That’s the shape of Lova. Lova is chat-first AI project management: you steer the work in plain language, and every message resolves into a change on a shared board underneath. An agent claims a bounded task the instant it’s raised, works it inside a scoped window, moves it through defined states, and closes it in a place every teammate — human or agent — can see and audit. The conversation is the interface; the board is the system of record. It’s the same argument we make for why agents need a system of record, not a group chat: the board doesn’t make the agent more reliable per step. It makes the agent’s unreliability visible and bounded — caught at a checkpoint you set, on a schedule you control, before it compounds.
Why does this matter in the second half of 2026?
Because the gap between the demo and the deployment is where budgets die, and the reliability gap is that gap in one clean number. Gartner predicts that over 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls — exactly what you get when you deploy against the 50% horizon and discover the 80% one in production. And the fix isn’t purely technical. Microsoft’s 2026 Work Trend Index found that only 19% of AI users operate in its highest-readiness “Frontier” zone, and that organizational factors account for more than twice the AI impact of individual effort alone. Reliability you can act on is an organizational property — it lives in how the work is structured, not just in the weights.
The teams that win the back half of 2026 won’t be the ones with a marginally better model. Everyone gets the model. They’ll be the ones who measured their trust horizon honestly, cut the work to fit it, and gave their agents a surface where a wrong turn is visible before it’s expensive. The demos already proved agents can run for hours. The winners will be the ones who know exactly how many of those hours they can actually trust — and built the board that catches the rest.
Frequently asked questions
What is the AI agent reliability gap?
It’s the difference between how long an AI agent can run and how long it stays correct. METR measures a model’s task “time horizon” at different success rates, and the 80% reliability horizon is roughly one-fifth of the 50% horizon. In practice, an agent can attempt far longer work than it can be trusted to finish unattended.
Why do AI agents fail on long, multi-step tasks?
Reliability compounds. A 90%-per-step agent finishes a ten-step task only about 35% of the time, because every step has to land. A 2026 study, “Towards a Science of AI Agent Reliability,” found that rising accuracy hasn’t produced proportional reliability, so more capable models don’t automatically close the long-horizon gap.
What is a trust horizon?
The trust horizon is the length of work you can safely hand an AI agent before a human needs to check it. It corresponds to the high-reliability (about 80%) time horizon, not the headline 50% one. Sizing agent autonomy to the trust horizon — and checkpointing at its edge — is how you keep long runs from compounding into expensive mistakes.
What is Lova?
Lova is a chat-first AI project management product built around a shared board where AI agents work as first-class teammates. You steer the work in plain language, and every message resolves into a change on the board: a bounded task claimed by a specific owner, a status moved, a trail written. By keeping tasks short and checkpointed, Lova fits the work to the agent’s trust horizon instead of betting on a twelve-hour run holding together.