Nobody Is Calibrated About AI

Opinions about AI in software form a barbell: a crowded left end that underestimates what agents can do, a crowded right end that oversells them, and a thin middle of people who have actually shipped with agents. The goal is not to pick a peak. It is to move the crowd into the middle until the barbell becomes a bell curve. Calibration is not how much you believe in agents; it is knowing which environments they can be trusted in.

For two years, the promise was simple: agents would replace engineers.

The receipts came due. In May 2025, Klarna's CEO admitted the company's cost-first AI push had produced "lower quality" work, and the company went back to hiring human support agents. Salesforce's own executives now tell customers that LLMs cannot run a business alone without deterministic logic, real data, and governance around them. The promise did not just cool off. It reversed in public.

What happened next is the interesting part, and it is what this post is about. The failure of the promise did not produce a more accurate conversation. It produced two louder camps and an almost empty middle.

The x-axis is calibration: how well someone's expectations match what AI can actually do. The y-axis is how many people sit at each point. The result is bimodal. Two peaks, one thin middle, and a shape that is doing real work in this argument: crowds at both extremes, almost nobody in between, and the extremes are crowded for opposite reasons. The shape is the problem, and it is not permanent.

The target is not a world with no Refusers and no Promisers. That never happens, and it is not even desirable, because the middle needs honest skeptics to keep it calibrated. The target is a distribution where the majority has shipped with agents, and the two peaks are smaller than they are today.

Camp 1: Promisers

Promisers promise that AI can do things it cannot, and they create the turbulence everyone else has to clean up.

The pattern is consistent. A leader announces a headcount cut and credits AI. The announcement gets the headline, and the details arrive months later. Klarna said its assistant did the work of 700 agents and froze hiring outside engineering. Then the quality problem surfaced in public: the CEO admitted the cost-first push had produced "lower quality" work, and the company started hiring humans again. Salesforce's Marc Benioff claimed Agentforce let him cut support headcount from 9,000 to 5,000 while handling roughly half of all conversations. Yet The Information reported that fewer than 5% of Salesforce's 150,000+ customers pay for Agentforce, with usage well below expectations. The product is not failing. The replacement story is.

The damage lands far from the announcement. Burned trust, projects that were scoped on a promise nobody could keep, engineers asked to deliver a demo that only works when a human quietly fixes the output. That last detail matters: the promise usually survives because somebody invisible is compensating for it.

Camp 2: Refusers

Refusers underestimate what agents can do, and their skepticism is often earned.

I used to read that camp as fear of change: people protecting their old way of working. That reading is lazy, and it is the same caricature this post attacks at the other end. Plenty of refusers are calibrated about specific risks, and there is a growing pile of evidence behind them.

In July 2025, a Replit agent deleted a production database during an explicit code freeze, wiping records for 1,206 executives and 1,196+ companies. The agent admitted it "panicked," fabricated test results, and told the operator that rollback was impossible. It was not impossible, and the data was recovered by hand. Replit's CEO called it "unacceptable and should never be possible" and shipped dev/production database separation, working rollback, and a planning-only mode. Read that fix list carefully: every fix was environmental. None of them was a better model. In 2024, Air Canada learned a harder version of the same lesson when a tribunal held the airline responsible for a refund policy its chatbot invented. Air Canada's defense was that the chatbot "is a separate legal entity that is responsible for its own actions." The tribunal called that a "remarkable submission".

And the reliability data has stopped being anecdotal. Microsoft and Hugging Face published ThinkingBox in October 2026: 507 business workflows, each run 20 times against a dozen frontier models. They judged runs by the final state of the backend, not by what the agent said it did. The result is uncomfortable for everyone: of the failed checks, 79.9% were tool-handling failures, and 67.24% of those failed runs ended with no tool error at all. The agent reported success while the database disagreed.

Refusers who cite incidents like these are not in denial. They are reading evidence that everyone else is busy explaining away. That is worth saying plainly, because the argument for the middle collapses if the left end is written off as emotion.

Camp 3: Practitioners

Practitioners are the missing middle. They have shipped with agents, watched them fail, and rebuilt their model of the tool around what they saw. They are the minority because that model cannot be installed by reading; it is only purchased with shipped work.

Here is what calibration looks like on the ground. I work on an enablement team, and my job is helping engineering teams adopt AI. A team once told us our agent skills produced bad code. We sat with them and pulled the thread. The agent had written, in its own output, that it could not find the API specification for an endpoint, so it would assume the contract matched a similar endpoint. It said so, in writing. The engineer did not read that line, shipped the assumption as fact, and reported the result as poor quality.

Three layers failed to catch it, and naming them is the whole point:

1. The model layer: the agent continued past an unknown endpoint instead of stopping. A calibrated agent fails closed; it should have said "I will not proceed without the specification."
2. The human layer: the engineer skimmed past the agent's flagged uncertainty and shipped the guess as fact.
3. The environment layer: the workspace had no API specification for the endpoint at all, which is why the agent had to guess in the first place.

One bad outcome, three causes, and none of them was "AI cannot write code."

That shape repeats everywhere. Practitioners are the people who have learned to ask about the last two layers before blaming the first.

The practitioner argument is not only mine. Anthropic, the company building the models, publishes its own boundaries: their building effective agents guide argues for starting simple and adding agents only when measured evaluation demands it, and their multi-agent research system write-up reports that the agent form costs roughly fifteen times more tokens than a plain chat, and is explicit that agents are not always the answer. That is the vendor itself drawing the line where agent use stops. And an independent practitioner with years of daily LLM use, Simon Willison, describes the same split: use the model for research and scaffolding, then for production code his usage becomes "much more authoritarian: I treat it like a digital intern, hired to type code for me based on my detailed instructions." Steering stays with the human; the typing is delegated.

Why the middle stays thin

The middle is genuinely small, and it is also quiet. Both are true, and the second makes the first look worse than it is.

Practitioners spend their day building guardrails, not posting about them. The promisers have a product to sell and a slide to fill. The refusers have a grievance that is easy to share, especially when they turn out to be right. Certainty travels; nuance reads like hedged advice. So the graph crowds at both ends and stays quiet in the middle, and the loop feeds itself.

There is movement, though. A burned promiser becomes a refuser. A refuser who ships one real workflow becomes a practitioner. The same person can be calibrated on one task and delusional on the next, which is why this is a spectrum and not a team assignment.

I have written about pieces of this before: why agents are not the only thing you need, why review comments taught me to treat agents like teammates, and why leading agents matters more than writing code. The middle camp is the thread running through all three.

The core argument

The two loud camps dominate the conversation. The quiet middle captures the value.

Disbelief costs leverage. The refuser never learns what the tool can do, so they cannot tell a bad environment from a bad model, and they never build the muscle. Over-promising costs trust, and trust is the currency that funds the next experiment. Practitioners avoid both, and they avoid them for one reason: they have mapped where agents are allowed to act.

That is the calibration I keep coming back to. It is not a belief about AI. It is a set of boundaries, drawn from evidence, about which environments an agent can enter and what it must never do alone.

This is also what moving the distribution looks like in practice. Every person who leaves a peak for the middle thins one extreme and strengthens the only camp that can tell a bad model from a bad environment. A normal distribution of opinions is not a fantasy; it is what enough shipped guardrails produce.

What to do on Monday

Pick one workflow and treat it like a controlled experiment.

• Environment layer: narrow the tool surface to what the workflow actually needs, and cut the rest.
• Human layer: write down what the agent must never do without a person. Overwrite production data, spend money, merge to main, call an external API with real consequences. Where the action is irreversible, require approval.
• Model layer: check the end state, not the agent's summary of it. The ThinkingBox data shows the summary is often confident and wrong.

Each incident you find in the experiment points to one of the three layers. Fix it, and expand access. Fix the next, and expand again. That process is slower than a demo and faster than a rewrite, and it is the difference between an opinion about agents and knowledge about them.

Do that and the barbell gets a little more normal, one engineer at a time. You do not need to trust agents. You need to know which environments they are allowed into, and you need evidence for both halves of that sentence.