My previous post about Zanagrams.com made it sound like I’d built it. Sorry! I didn’t build it, just enjoyed it.
Tag: reviewing
I want to trust AI with code enough that I’m no longer doing line-by-line reviews. But when I look at the code, even from frontier models, I’m still finding small things that feel smelly.
In a simple PR Sonnet 4.6 made, I just found it changing the default mock data constructor for all existing test cases away from the common case to a less-common scenario. All the existing tests passed because they asserted on things that weren’t changed. The update was technically sound, but it’s the kind of change that would feel weird to a human if they came across it later.
Diving into test assertions is a good way to focus on the verifiability of a PR when doing AI-assisted reviews. Here’s one of the prompts I often use:
please give me a human-readable outline of the
tests we added specifically calling out what they
assert and how as an html doc