There's no doubt that AI is making us all faster at our jobs. But what is it doing to our judgement?

Last month I performed at the Edinburgh Fringe with my improv troupe, Who Killed The Plot. Running our murder mystery improv set every night, there was no script, no slides and no way to undo anything. Someone in the audience shouts out a suggestion and the next hour comes out of the minds of the five of us on stage in real time in front of an audience that has paid to be there.

Each night, I came off stage buzzing. I'd spent an hour making decisions with nothing to lean on and finding out straight away whether they were any good. There was nothing there to suggest the best path before I took it.

I'm not complaining about my day job here. If anything it's the opposite. I've spent the last year putting AI into everything it sensibly belongs in: the tooling, the incident process, the bits of the workflow that were slowing our engineers down. It worked, and I'd make all of those calls again. But some of the thinking used to live in the friction I've removed, and I think that's worth paying attention to.

Requirements vs Intent

Back in February I wrote that AI is only as good as your understanding. AI works to requirements, not to intent. It can see what you asked for, but it can't see why you asked for it, or the years of customer conversations and failed experiments sat behind the ask.

I still think that's right. But there's an obvious follow-up question I didn't cover, which is what happens to the person who is supposed to be holding the intent.

Intent isn't a document. It lives in someone's head, and it stays there because that person keeps doing the work. They sit in the customer calls. They read the diff. They get an architecture decision wrong and find out what it costs. It's the experience gained from doing the job and it's a by-product of wrestling with the problem.

Outsource the wrestling and you'll save a lot of time, but you'll also stop building the understanding that made you worth asking in the first place.

We Have a History of Cognitive Offloading

We've been handing skills over to machines for a long time. Victorian factories took work that had once depended on skilled hands and judgement and turned much of it into machinery and process. Weaving, spinning, typesetting: machines made people dramatically faster, and some of the skills they replaced simply stopped being useful and the faded away.

London taxi drivers used to spend years learning The Knowledge, and their brains physically changed for it. Most of us now turn left because a phone told us to, and couldn't draw a map of our own city if you paid us. That's a fair trade and I'm not about to give up sat nav. The cost is a skill I don't really need any more.

It's also usually obvious when the machine gets it wrong. Most people aren't going to drive their car through a river or down a closed road just because the map told them to. Reality gives you something to argue with.

The problem gets harder when the mistake isn't obvious. If you can't reason about the system yourself, you've got no real way of telling whether the model's answer is right or just convincing.

Auto Mode

I've been watching a version of this happen in the software industry over the last few months.

Claude Code started out asking permission for everything. Every file edit, every command, a prompt and a decision. It was slow and fairly irritating, but in theory it meant a human looked at every action before it ran.

Then came auto mode, where a second model reviews the pending action and decides whether it's safe. Is this beyond what was asked for, is it touching infrastructure it doesn't recognise, does it look like it came from something hostile the model read on a web page. Read-only work and edits inside your own directory don't even reach it. It's a genuinely clever piece of design and it works. I use it, my team uses it, and I'm not turning it off. Cursor shipped much the same idea a few weeks earlier and reckon it cut approval prompts by around 84%, so this isn't one vendor's opinion, it's where the whole industry has landed.

The approval step hasn't been removed, it's been delegated. The last point in the loop where a human had to form a view now belongs to a model.

When Anthropic made auto mode the default in August, they mentioned that people had been approving 97% of permission prompts before the change.

At 97% that's no longer review, it's clicking through a dialog to get back to work. The judgement had already gone. Whatever we tell ourselves about the value of a human in the loop, the honest version is that auto mode replaced the rubber stamping people were actually doing, not the careful review nobody was doing.

That's what I find interesting about it. I thought I was watching people gradually hand something over. Really the handover had already happened, quietly, somewhere around the thirtieth prompt of the afternoon, and the tooling just caught up with where our attention had gone. Nobody decided to stop reviewing. Nobody had that meeting.

Anthropic are straight about the limits, for what it's worth. Auto mode relies on classifiers, it doesn't eliminate risk, and they still recommend reviewing high stakes production changes yourself. The question is whether you've kept enough of the habit to do that when it matters.

When The Stakes Aren't Code

In engineering we at least have machinery for when we're wrong: tests, rollbacks, incident response and postmortems. So what happens when the same curve runs through decisions that don't have a rollback?

We're already pointing these systems at policy: who gets triaged first, who gets flagged for fraud, who qualifies for support and who gets a letter saying they don't. The pitch is the same one that won in engineering, and it isn't a dishonest pitch. The model is consistent, it doesn't have a bad morning, and on average it will beat an overloaded caseworker with forty minutes per decision.

There will be a human in the loop, because there always is. That human starts out reading every case properly. Then the volume goes up, the model keeps being right, and reading every case properly starts to feel like a slow way of agreeing. Eventually they're approving decisions they couldn't have reached themselves and couldn't defend if challenged.

We've been here before in the UK. The Post Office spent years prosecuting sub-postmasters because the system said money was missing, and nobody with any authority was willing or able to believe the humans over the computer. That wasn't really a story about bad technology, it was about losing the standing to argue with it. A bad deploy gets rolled back. Those people lost their businesses, their savings, and in some cases a decade of their lives.

These systems still belong in public services. The status quo is already failing people, and a well built model will catch things that a queue of exhausted humans never will. Build them. Just be aware that "there is a human reviewing it" isn't worth much as a safeguard unless that human has the capacity to disagree, the time to use it, and an organisation that will back them when they do. That's a design problem, and it belongs to whoever builds the thing.

Dario Amodei published We Must Pace the Frontier a couple of days ago, and it's been in my head since. His concern is much larger than mine, it's about the rate at which the models themselves are getting more capable and whether alignment work can keep up. But underneath it is the same worry. He isn't arguing we stop, he's arguing we slow down enough that our understanding has a chance of tracking the thing we're building. When the CEO of a frontier lab is saying that in public, the version of it that applies to me sat in front of a code review feels worth taking seriously.

What Improv Is Actually Training

When I wrote about what Improv taught me about leadership I framed it around Yes And, confidence, failure and active listening. All still true. What I didn't appreciate at the time is that Improv trains exactly the muscles this stuff lets you skip.

You commit to a choice before you feel ready, because you can't go and gather more context or generate three options and pick the best one. You listen properly, because the scene is built out of what your partner just handed you, and a room of a hundred people can tell instantly when you're not really there. And you fail in public, with no revert, and carry on anyway.

None of that can be optimised, which is rather the point. I'm not suggesting you all take up Improv (but you really should). Anything with those properties will do. Play in a band, referee a match, give the talk you haven't rehearsed, write the first draft yourself. Just keep hold of one thing where the answer has to come from you.

Staying In The Loop

Everything here assumes you're using AI heavily, because I am and so is my team. These are the best tools we've ever had and I want more of them. The question was never whether to use them, it's whether you're still in the loop while you do.

At my level, part of the job is making sure those habits survive beyond me, but I've found it starts with being deliberate about my own.

Form an opinion first. Before I ask, I'll write down what I think the answer is, even if it's only a couple of lines. Otherwise I'm not really evaluating the output, I'm just receiving it.

Don't approve what you can't explain. If I couldn't defend the change in an incident review at 2am, I haven't reviewed it. I'd rather have an engineer who can explain forty lines than one who can generate four hundred they can't defend.

Argue with it. These models are agreeable by design. If it folds the moment you push back, that tells you something about how solid the first answer was. If it holds its ground for a good reason, that tells you something too.

Use it to learn, not just to do. You have patient, on demand access to something that will explain almost anything at whatever level you ask for. I think that's the most valuable thing about these tools and it's the bit most people skip. Ask why it chose that approach and what it ruled out. Ask it to teach you the part of the system you've been avoiding. You can ask all the questions you'd never risk asking in a meeting.

Keep some thinking unassisted. Technical strategy, org design, anything where I'm forming a position rather than executing one. I might use AI afterwards to challenge it, but the thinking is the deliverable there, not the document.

Notice the drift. The failure mode isn't dramatic. It's looking up in three months and struggling to name a decision you'd call your own.

Don't Lose Your Head

Each time you take the better answer without doing the thinking you've made a perfectly rational trade. Do it enough times and while you are still in the chair, still technically in charge, you're no longer able to tell a good decision from a bad one. At which point you start asking AI whether it is a good decision.

That's manageable in a codebase. It's a much bigger problem when the thing being decided is somebody's welfare, and the only person who could have said no has been out of practice for years.

So use the tools and push their capabilities but stay curious with them. Keep doing some of the thinking yourself, especially when it would be easy not to.