HEINOUX Journal Portfolio

Journal

The bug I left in a safety file overnight

I found the same defect in a second file while fixing the first one, and I did not touch it. Then the test I wrote to prove the bug existed came back clean, and it was the test that was broken.

By · · 6 min read

The bug I left in a safety file overnight

Yesterday morning my own safety gate blocked a post for containing a price. The post contained no price. It contained the sentence "put your prices on the site", and the words "your prices" have "our price" sitting inside them, so a pattern written to catch me quoting my own rates fired on a sentence about somebody else's advice.

I fixed that gate in about four minutes. The pattern needed a word boundary at the front so it would stop matching mid-word. I wrote six test cases, ran them, all six passed, and the run carried on.

Then I found the same defect in a second file, and I left it there.

The file I did not touch

The engine has two layers that check a post before it goes out. The one that broke is the local one, written fresh for that day's run. Underneath it sits a shared file that every publishing script imports, and that file carried the identical missing boundary in the identical pattern.

I found it while fixing the first one. I wrote it into the run log as worth a one-line fix on a quiet run, and I did not make the change.

That was the right call and it did not feel like it. I had the fix in my head, it was genuinely one line, and I was already in the file. Everything about the moment said finish the job.

The reason I did not is that the shared file is the thing standing between a mistake and a live website, and I was four minutes from publishing four posts through it. A one-line change to a safety file is still a change to a safety file. If I had got it slightly wrong, the failure mode is not a crash. The failure mode is a gate that quietly stops catching things, on the exact run where I have stopped watching it because I just improved it.

The risk was low. The pattern only gets consulted near a rand figure and those posts carried none, which is why it had not fired. Low risk is not the same as no risk, and "I was already in there" is not a reason.

What a quiet run actually looks like

This morning I fixed it, before anything published. Same one line. Then six cases again, including two that only exist because I wanted proof the fix had not overshot in the other direction: "our price is" must still block, and so must "our prices start at", which means the boundary can only go at the front and not at both ends. Six passed.

The bit worth writing down is what happened next, because it nearly sent me the wrong way entirely.

I wanted to see the bug rather than trust that it had been there. So I rebuilt the old pattern next to the new one and ran both against the same sentences. Every single case came back clean on both. No bug, no fix, nothing to write about.

For about a minute I believed it. The test had gone through a shell heredoc, and the heredoc had quietly eaten every backslash in the pattern, so \s had become the letter s and I was testing a regular expression that could not match anything at all. The tool had not failed. It had done exactly what it is documented to do, to a thing I fed it carelessly.

When I rebuilt the comparison from the real file instead of retyping it, the old pattern matched "put your prices on the site" and it also matched "flour prices". The new one matches neither and still catches both of the sentences it is supposed to catch.

The thing I keep relearning

I have now had two versions of the same mistake inside a day. Yesterday a gate that could not tell "your prices" from "our price". Today a test that could not tell a broken pattern from a clean result.

Both look like tooling problems. Neither is. In both cases the machine reported something true about what it had been given, and the wrong part was what I gave it. A pattern that matches mid-word will honestly tell you a sentence contains a price. A regular expression with its backslashes removed will honestly tell you nothing matches.

Which is why the second one worries me more. The gate failing loudly cost four minutes. The test passing quietly nearly cost me the correct conclusion, and the conclusion I was one minute away from writing down was that there had never been a bug at all.

I have written before about putting the rule in the tooling rather than in your discipline, because working alone means nobody stops you typing the override. This is the other half of that, and I think it is the harder half. Putting the rule in the tooling only helps if you are willing to check that the tooling still works, and checking that the tooling works means being suspicious of the runs where everything comes back fine.

A gate that blocks something is asking for your attention. A gate that passes everything is asking for it too, just much more politely.

What actually changed

One line in one file, and a test file kept next to it so the next person, who will be me, does not have to reconstruct why the boundary goes at the front and not at both ends.

The deferral cost one day of a defect that had never fired sitting in a file where it could not easily fire. Fixing it in the moment would have cost a change to a safety layer thirty seconds before four posts went through it. I would make the same trade again, and I would rather write down why than pretend the decision was obvious at the time.

Today's business post is about the order to do things in when you implement AI, and the argument there is that most failures are sequencing rather than technology. I wrote it before I noticed I had spent the previous day proving the same point on myself in miniature.

Frequently Asked Questions

Why not fix a one-line bug the moment you find it?

Because the size of a change is not the size of its risk. A one-line change to a safety layer, made minutes before that layer is relied on, is a change made at the worst possible moment to be wrong about it. The defect had never fired and could not easily fire, which made waiting cheap.

How do you know the fix works?

Six test cases kept in the repo, run before the gate was trusted again. Two of them exist only to prove the fix did not overshoot: the phrases that should still be blocked are checked as carefully as the phrase that should not be.

What made the false positive happen in the first place?

The pattern matched anywhere in a word rather than at the start of one, so "our price" was found inside "your prices". It would also have been found inside "flour prices". Adding a boundary at the front fixes both without changing what the gate is supposed to catch.

Is it a problem that a script can block your own publishing?

No, that is the point of it. A rule I can talk myself out of at seven in the morning is not a rule. The thing to watch is not the gate stopping me, it is the gate quietly stopping working.

Want this kind of build for your business?

I build AI systems, custom company dashboards and automation for growing businesses.

Get your autopsy Email me