The Config Property That Did Not Exist

Thursday, 18:40. Most of the office was already gone. On my second screen there was a Slack message from Mark, a colleague from the business side, with three exclamation marks.

"The AI said the timeout thing is fixed!!! Why is the demo still failing?"

He was not wrong to ask. Two hours before, I had asked the agent to make our calls to an external document service more robust. The service was slow sometimes, and on Monday we had a demo with our biggest pilot customer. If an upload failed in front of them, nobody would care how nice the rest of the product looked.

The agent had answered with confidence. Clean diff, nice explanation, green build. And Mark, who reads the agent's summaries in the PR like other people read the news, had already told the customer success team that the problem was solved.

It was not solved.

A very clean diff that did nothing

The change was small. That was the first thing I liked about it, and later the first thing I should have been suspicious about. The agent added a few lines to application.yml:

spring:
  webflux:
    client:
      retry:
        max-attempts: 3
        backoff: 500ms

And it wrote in the PR description: "Configured automatic retries with backoff for all WebClient calls using Spring Boot's built-in retry properties."

It looked right. It sounded right. It had the shape of a real Spring Boot property, with the relaxed names and a duration like 500ms. I have worked with Spring Boot for many years, and for maybe ten seconds my brain said "ah yes, of course".

Then I opened the logs from the staging run. One request. One timeout. No second attempt. No third.

I asked the agent a simple question.

"Where is this property defined? Show me the class that binds it."

It apologized. It said I was right to check, and that the property is actually under a slightly different prefix. Then it proposed a new version, spring.http.client.retry.max-attempts, again with full confidence.

I felt that small heat in the neck. Not anger exactly. More like when you explain the way to someone and they nod and walk in the wrong direction.

Spring Boot will not tell you

Here is the thing that makes this mistake so expensive. Spring Boot does not fail when you give it a property that does not exist. It just ignores it. No error, no warning in the startup log. Your app starts, your tests pass, and your YAML is a nice piece of fiction.

So the agent's invention did not break anything. It was worse. It looked like a fix, and it went through review, CI and a very happy Slack channel.

I stopped the agent. Instead of asking it again "are you sure", which never works, I changed the question. I asked it to prove the property against the code we actually run.

Do not change any code yet.
Open the spring-boot-autoconfigure jar that is on our classpath
(check the exact version in the build file first).
Search the configuration metadata for any property containing "retry"
under the webflux or http client prefixes.
List what you find with the class that binds it. If nothing exists, say so.

This time the answer was short. There is no such property. Retries for WebClient are not something Spring Boot configures for you. You do it in code, with Reactor.

Two minutes, and the truth was on the table. The agent knew how to find it. It just did not do it until I asked in a way that left no room for guessing.

When the agent sounds sure, I don't ask if it is sure. I ask it to show me where it found it.

Then it surprised me

Now the agent was useful again, and actually more than useful. It wrote the retry in the client code:

webClient.post()
    .uri("/documents")
    .header("Idempotency-Key", uploadId.toString())
    .bodyValue(request)
    .retrieve()
    .bodyToMono<DocumentResponse>()
    .timeout(Duration.ofSeconds(10))
    .retryWhen(
        Retry.backoff(3, Duration.ofMillis(500))
            .filter { it is TimeoutException || it is WebClientRequestException }
    )

Look at the header. I did not ask for it. Before writing the retry, the agent stopped and said something like: "This is a POST that creates a document. If the first call reaches the server but the response times out, a retry can create a duplicate. The vendor's API documentation supports an idempotency key header. Do you want me to use it?"

That was a good catch. To be fair to myself, I was so focused on the fake property that I had not thought about duplicates yet. A double upload in the customer demo would be almost as bad as a failed one. Maybe worse, because it looks like a bug in our data, not a slow network.

So in the same evening, the agent invented a config property out of thin air, and then saved me from a duplicate-data bug I had missed. Both are true. I have stopped being surprised by this mix.

The last step was a test. Not a test that the code compiles, a test that the behavior exists. We used a mock web server that fails twice and answers on the third call, and we assert three requests arrived and only one document was created. If someone ever "simplifies" the retry into a YAML property again, this test will tell them.

The message I sent back

At 20:15 I answered his message. I did not write "the AI was wrong". That is not useful for him, and it is also not the full story. I wrote:

"The first fix was a setting that our framework silently ignores, so it looked done but did nothing. The real fix is in, with a test that simulates the slow service. I also added protection so a retry cannot upload the same file twice. Staging is green, and I checked the logs myself."

He answered with a thumbs up and then: "So how do I know next time if the AI fix is real?"

That was the best question of the week. My answer: "If nobody can show you it working, it is not fixed yet. Green build only means it compiles and the old tests still pass."

The demo on Monday went fine. The document service was slow two times. Nobody in the room noticed.

What I do now

1. Ask for the source, not for confidence. "Show me the class that binds this property" or "show me the method signature in the version we use" beats "are you sure?" every time. Confidence is free for the agent. A file path is not.
2. Be extra careful with config. Code that is wrong usually fails. Config that is wrong often just gets ignored. Any new property in a PR gets the question: where is it documented for our exact version? Your IDE marking a key as unknown is a small hint worth looking at too.
3. Verify behavior, not the diff. For anything about retries, timeouts, caching or limits, I want a log line or a test that shows the behavior happening. A clean diff proves nothing.
4. Let the agent think about side effects, out loud. I now add one line to these prompts: "Before you implement, tell me what can go wrong if this runs twice." It is very good at this when you ask.
5. Translate "done" for non-technical people. Give them one simple rule they can use without you: fixed means someone saw it work, not that the agent said so.

I still like working with the agent. It knows more API docs than I will ever read. But it does not know the difference between remembering something and making it up, and it will not tell you which one it is doing.

How do you catch these confident little inventions in your team, before they reach the demo?