I Code Like A Rock Star!

Originally Posted On: DATE: 10/07/2005 11:27:57

Today I was working with one of my office mates on reproducing a problem reported at a field site. He had already gotten the source from the tip, built, and tested it, and then added a test case to demonstate the problem we were seeing in the field. The conversation went something like this:

Him: "I added the test, and it all works."
Me: "Really? That change should have caused a failure. Let's take a look."
We looked at the code, and sure enough, the test that should fail was in the test case.
Me: "This is really strange..."
I added a fail(...) to the test case, and it still didn't cause any failures.
Me: "This is really bothering me..."
I added a compile time error to the test case, and the compiler did complain this time.

Finally, we looked in the test suite (this is a fairly old class that's still using NUnit 1.x) and found that the suite wasn't including this test class. We added the class to the suite, and the failures happened in the way we were expecting. We then backed all the cruft we had just added to the test class back out, and worked on the problem from the beginning. It was very disconcerting to be able to add errors to the test class but not have them show up.

Two lessons I learned from this episode:

1. Don't Take Short Cuts. There's a great article on The Three Rules Of TDD by Bob Martin. We should not have added anything to a class that wasn't really being called. There are ways to validate that the classes are actually included. This problem would be alleviated by using NUnit 2 somewhat, since the tests there are tagged instead of named, and the Suite class is unneeded.
2. Don't Trust Anything. Whenever something smells wrong, there's probably a good reason. Start with the simplest thing (like - is the editor, the compiler, and test running all pointing at the same place?) and work up in complexity to find where things start to stink. Then iterate around there until you find the problem.