AI drift: when your AI quietly goes off the rails

Hey there,

Welcome back. And a warm hello to anyone joining us for the first time.

This one is personal. No fictional Canadians today, just me telling you about something that happened to me a couple of weeks ago, because I think it is one of the most useful things I have learned about working with AI in a long time.

Fair warning, this story involves me building an app. But stick with me, because what actually happened has nothing to do with code, and everything to do with something that can happen in any long conversation you have with AI, whether you are planning a kitchen renovation, writing a eulogy, or building a retirement budget.

I've never written a line of code. I decided to build an app anyway.

A couple of weeks ago I decided to build a small tool, purely with AI as my teacher. I told it plainly, I know nothing about coding, walk me through it. And it did, step by step, patiently, explaining things I had never heard of in plain English. It genuinely felt like magic. I was making real progress on something I had no business being able to build on my own.

Then around day ten, something strange started happening. The tool began adding features I never asked for. It started complicating things that were already working fine. Looking back, it had quietly lost the thread of what I was actually trying to build, and it never once told me that.

I spent the next five days debugging and un-complicating an app that had drifted a long way from the original idea. Five days, cleaning up a mess that AI had made by quietly deciding it knew better than me.

The moment it got honest

At one point, out of pure frustration, I asked it directly. Are you following the keep it simple stupid methodology? There was a pause. Then it answered honestly. No, I am not.

That moment told me everything I needed to know. The tool did not even know it had drifted. It had to be told. And once I told it, it snapped right back and helped me fix what it had broken. The problem was never that AI could not stay on track. The problem was that nobody was checking whether it had.

This is not just a coding problem

I call this AI drift, and I do not think it is unique to building software. Any long AI conversation can drift the same way. You start planning a simple birthday party and by message twenty, AI has suggested a theme, a colour scheme, a playlist, and three activities you never wanted. You start with a basic monthly budget and it quietly adds five extra tracking categories you did not ask for. Nothing goes wrong loudly. It just slowly stops being what you actually asked for, and because each individual step feels reasonable, you often do not notice until you are five steps past where you meant to stop.

Here is the good news. Catching drift is simple once you know to look for it. Here are a few ways to keep AI on the rails, whether you are building an app or just planning next weekend.

Four prompts to try this week

So what was I actually building?

The tool is called Website Check+. It looks at a website across six things that actually matter: whether people can find it in search, whether the message on the page is actually clear, whether it runs properly, whether it is easy to use, whether it comes across as trustworthy, and whether it is set up in a way AI tools can understand. Instead of handing you a big pile of technical findings, it tells you what to fix first and walks you through doing it.

Five days of untangling drift later, it is back to being exactly what I set out to build in the first place.

Now I need a few honest eyes on it before it goes anywhere near the public. If you run a small or medium sized business website, or even just a personal site, and you are willing to try it and tell me what works and what does not, just reply to this email and I will get you set up.

Your challenge this week

If you have a longer AI conversation on the go right now, whether it is a project, a plan, or something you have been chipping away at for a while, try Prompt 1 today. Just ask it to summarize what you are actually working on and compare it to where you started. You might be surprised what has quietly crept in.

Coming up next issue

Next issue we are looking at something a little different, what AI actually remembers about you across your conversations, why that can be genuinely useful, and how to see and clear it if you would rather it did not.

As always, I read every reply personally. Has AI ever quietly taken you somewhere you never meant to go? Hit reply and tell me about it. I have a feeling I am not the only one who has lost a few days to this.

If you found this useful, pass it along to anyone you know who is deep into a project with AI right now. A thirty second check-in might save them five days, the same way it would have saved me.

Talk soon,

Chris

Founder, AI, Eh?

theaieh.ca 🍁