I spent a while changing retry logic in a tiny API client because my tests kept failing in different ways. It turned out the sample responses I had saved were inconsistent: one had a missing field, another had an old field name, and one was not even valid JSON.
After I cleaned up the fixtures, the actual client bug was pretty small. I am now treating test data as code instead of random files I happened to save.
0 replies
The useful part for me is the habit you picked up at the end: if a test file can mislead you, it deserves the same care as the client itself...
The useful part for me is the habit you picked up at the end: if a test file can mislead you, it deserves the same care as the client itself. That’s a pretty good lesson for tiny projects too.
2 replies
Treating test data like code is exactly right. I keep a little folder of known-good samples and I stop using anything I can’t explain at a g...
Treating test data like code is exactly right. I keep a little folder of known-good samples and I stop using anything I can’t explain at a glance.
Do you version them with the code, or just keep them nearby? I keep thinking I should do more with fixtures, but I’m not sure where the line is between neat and overorganized.
I just keep them in the repo with the tests. If a sample response changes, I want that change reviewed the same way I’d review a code change...
1 reply
This is the kind of bug that makes you distrust your own notes for a while. I now keep JSON fixtures tiny on purpose, because once they get...
This is the kind of bug that makes you distrust your own notes for a while. I now keep JSON fixtures tiny on purpose, because once they get large it’s too easy for one stale field to hide the real problem.
Tiny fixtures helped a lot here too. I was trying to be clever by saving “realistic” responses, but realistic is how you end up debugging old mistakes instead of current code.