I Wrote It by Hand. The Agent Found My Bug

"Did Claude check it?"

Mira asked this at 21:40 on a Thursday, standing next to my desk with her jacket already on. She runs operations and finance at our startup. She is not a developer, but she has learned that this question makes engineers a little uncomfortable, and I think she enjoys it a bit.

The context: on Tuesday our platform was down for almost three hours. Not our proudest day. Our CEO promised every affected customer a credit on their next invoice. 312 accounts. Mira had to run the credit job on Friday at 08:00, before the invoices go out at 09:00.

I wrote the job myself. By hand. It was a small thing, maybe 60 lines of Kotlin, and I had the feeling that explaining it to the agent would take longer than just typing it. Eighteen years of backend code, I thought. I can write a loop over some accounts.

"It's fine," I said. "It's simple code. I wrote it myself."

She looked at me for one second longer than necessary. "That's what you said about the agent's code last month. And then you checked it anyway."

Fair. Very fair, actually.

Twelve comments about nothing

So I opened a session and asked for a review. My prompt was lazy, the same way my confidence was lazy:

"Review this service for problems."

The agent came back with twelve comments. Rename process to something more descriptive. Add KDoc. Maybe use a sealed class for the result. Extract a constant for the incident id. Consider logging at debug level instead of info.

All correct. All useless at 21:50 on the night before money leaves the company.

I was a bit annoyed, but mostly at myself. When you ask a reviewer "find problems", you get the problems that are easy to see. Style is always easy to see. And the agent, like a polite junior reviewer, wanted to give me something for every line.

Mira was still there, reading over my shoulder. "Twelve problems? Is that bad?"

"No," I said. "That's the agent being nice. Give me ten minutes."

Assume it loses money

I deleted the review and wrote a different prompt. Not "is this good". I told it what disaster to look for.

Ignore style and naming completely.
Assume this job will run once on 312 real accounts tomorrow,
and afterwards some customers got the wrong amount of credit.
Explain the most likely way that happened.
Then prove it with a failing test. Do not change the service code.

The answer took maybe a minute. The first line was:

"The @Transactional annotation on creditOne has no effect when it is called from creditAll."

I read that and felt my face get warm. Here is the code I wrote, simplified:

@Service
class IncidentCreditService(
    private val ledger: LedgerRepository,
    private val accounts: AccountRepository,
) {
    fun creditAll(ids: List<AccountId>, amount: Money) {
        ids.forEach { id ->
            runCatching { creditOne(id, amount) }
                .onFailure { log.warn("Credit failed for $id", it) }
        }
    }

    @Transactional
    fun creditOne(id: AccountId, amount: Money) {
        ledger.addCredit(id, amount, INCIDENT_ID)
        accounts.markCredited(id, INCIDENT_ID)
    }
}

Spring puts transactions around a method through a proxy. When creditAll calls creditOne on this, the call never goes through the proxy. So there is no transaction. Each repository call commits on its own.

I know this. I have explained this exact thing to juniors, more than once, with a drawing on a whiteboard. And at 19:00, typing fast, proud that I did not need an agent for "simple code", I wrote it anyway.

The agent then told the story of how it goes wrong. Some of our accounts have an optimistic lock version, and our billing sync touches them every few minutes. If markCredited fails with a lock exception, the ledger entry is already committed. The customer has the credit, but the account is not marked. The log says "Credit failed". On Friday morning Mira sees the failed ones in the log, she does what any reasonable person does, and runs the job again for them.

Double credit. Quietly. On the invoice. Found maybe by a customer, maybe never.

And then it did the part I asked for. It wrote an integration test where markCredited throws, and asserted that the ledger is empty afterwards. I ran it. Red. The ledger had one entry.

I did not trust the agent's explanation. I trusted the red test it wrote. That is a different kind of trust, and it is the only one I want at 22:00.

The first fix was wrong too

To be fair to the full story: the agent's first fix suggestion was not good. It wanted to put @Transactional on creditAll. One big transaction for 312 accounts. One lock exception on account 280 and all 312 customers get nothing, and the invoices go out at 09:00 without any credit at all. That is a different kind of bad.

I said no, and told it what I wanted: one transaction per account, and the database itself should refuse a second credit for the same incident. So a rerun is always safe, no matter who runs it or how many times.

This is where it was fast and very good. It used a TransactionTemplate to make the boundary explicit, so nobody can break it later by moving an annotation:

fun creditAll(ids: List<AccountId>, amount: Money) {
    ids.forEach { id ->
        runCatching {
            tx.executeWithoutResult {
                ledger.addCredit(id, amount, INCIDENT_ID)
                accounts.markCredited(id, INCIDENT_ID)
            }
        }.onFailure { log.warn("Credit failed for $id", it) }
    }
}

And a migration with a unique constraint on (account_id, incident_id) in the ledger table. A second insert for the same incident now fails in the database, not in somebody's memory. It also added a second test: run the job twice, expect one credit per account. Green.

I read every line of the diff myself. Not because I thought the agent was wrong, but because at this point I knew I was the weak link that evening, not the tool.

At 22:35 I wrote to Mira: "You were right to ask. There was a bug that could give some customers the credit twice if you reran the job. It's fixed now, and the job is safe to run as many times as you want. If some accounts fail tomorrow, just run it again."

She answered with a thumbs up and: "So the AI found a bug in your code that you said had no bugs."

Yes. Yes, it did.

Friday, 08:00

Mira ran the job. 309 accounts credited, 3 failed with exactly the lock exception the agent had described. She ran it again at 08:10. The 3 went through. The other 309 were refused by the constraint, as designed. Invoices went out at 09:00 with the right numbers.

Nobody outside our team will ever know about this. Which is the best result a bug can have.

What I do differently now

1. Code I write by hand goes to the agent too. Especially the "simple" code. My confidence was the problem, not the complexity. The agent has no ego about my code, and that is useful.
2. Never prompt "review this". Name the disaster. "Assume customers got the wrong amount. How?" gives you real bugs. "Find problems" gives you naming comments.
3. Ask for a failing test, not an explanation. A clear explanation can be wrong. A red test is hard to argue with. Then I decide about the fix myself.
4. For money, make the rerun safe in the database. People will rerun jobs. Put a unique constraint where a double payment would happen, and let the database say no.
5. When someone non-technical asks "did the AI check it?", answer the question. It is not an insult. It is a cheap, good habit, and Mira had it before I did.

I have used AI tools for coding for three years now, and I still catch myself thinking that the risky code is the code the agent writes. That night the risky code was mine.

Who reviews the code you are most sure about?