Unit tests are best

Last year I was reading Eric Raymond’s The Art of Unix Programming and was struck by the statement that “debugging often occupies three-quarters or more of development time”. Yes, I thought, I remember when that was normal. In the old days I used to spend a few hours coding something, then the rest of the day working out how I’d done it wrong.

These days any time spent regularly debugging is a sign of something wrong. That’s a significant, and short-term, productivity gain of Test Driven Development that’s often forgotten. Done properly TDD means when you’re done, you’re pretty confident that you’re done. You can go on and do something else.

TDD is mostly about unit-testing, and the trouble with unit-testing is that it’s only the units that get tested. The interactions between the units can be left out, and that’s where the errors creep in.

Let’s say you have a class that represents a spreadsheet; another class is responsible for populating the spreadsheet. They are both thoroughly unit tested, but one assumes the row index is zero based and the other that the row index is one-based. You may have a bit of debugging ahead.

Functional tests are commonly used to plug this gap.

Functional tests are great. They are worth having to boost confidence in the general correctness of the code; but they are a safety net. They are not a good place to be catching problems: they run on a slower feedback loop than the unit-tests and when they highlight problems, and because they cover bigger chunks of code it can be tricky to find the source of a problem: they bring you back into debugging land. They also tend to take longer to run, and complicate the build. And they can be a pain to set up, especially if you need lots of supporting data.

You can reduce your reliance on functional tests by minimising the assumptions made about the rest of the code in your unit-tests. One way to do this is to steer clear of the Mock Object fetishism that is sweeping the nation(s).

Use the real objects when you can. Sometimes, though you do need to fake things.

Let’s say you’ve some Java code that produces Json to be consumed by an Ajax request from a browser and processed by Javascript. You can unit test the Java with Junit; you can even unit test the Javascript with Jsunit. But are you confident that it all works together? Selenium or Watir tests would do the trick, but bring us back into functional testing land.

How about for each of the Json states you test in the Javascript, you make sure you have a Java test that covers producing that Json. Save off the Json and use it in both tests. Now you have pretty much all of that functionality covered in Unit Tests.

Keeping the bulk of your tests is best for rapid feedback cycles and increased productivity. You may need some functional tests, but maybe not as much as you think.

This post is a rehashing of the talk I gave on the January 19 Naked Agilist podcast. I fluffed it, I’m afraid but I would still recommend listening because the other speakers were great.

Labels: functional tests, Jsunit, Junit, TDD, unit tests

Wednesday, January 23, 2008 | Permalink | 1 Comments