murphitup @murphitup · Jun 10

I want to trust AI with code enough that I’m no longer doing line-by-line reviews. But when I look at the code, even from frontier models, I’m still finding small things that feel smelly.

In a simple PR Sonnet 4.6 made, I just found it changing the default mock data constructor for all existing test cases away from the common case to a less-common scenario. All the existing tests passed because they asserted on things that weren’t changed. The update was technically sound, but it’s the kind of change that would feel weird to a human if they came across it later.