Test Integrity November 5th, 2007

CruiseControl.rb is a wonderful tool. Once setup, every time you commit a change your app's entire test suite is run. If anything fails, it notifies everyone via email, with a followup when it's fixed.

In addition to providing continuous integration, it creates a social stigma when someone makes a careless commit.

I've put two apps on Cruise Control without problem. When I went to add a third project, which I'm not developing, I ran its test suite for the first time and noticed two failures. I tracked them back over 1,500 revisions, months of work. That's as far a I could go before our repository restructure, so I gave up.

I asked one of the team members about it. They said, "Oh. Those are the two tests that always fail."

Sometimes words speak louder than actions.

rake -T November 1st, 2007

Someone violated our test/ directory with odd scripts such as "testallunit.rb":

require File.dirname(__FILE__) + '/unit/business_listing_test.rb'
require File.dirname(__FILE__) + '/unit/search_ranks_test.rb'
require File.dirname(__FILE__) + '/unit/category_test.rb'
# ...

That's right, a ruby script to run all unit tests, another to run functionals, and so on.

In the end, it was because the author didn't understand "rake test:units" and its brethren. For details, run:

rake -T test

Naming Your Tests October 3rd, 2007

A common mistake is creating giant tests that test unrelated things. You want lots of small tests that target specific functionality. There are reasons beyond aesthetics and philosophy. In Test::Unit, when one assertion fails, the test haults. If it's the first assertion, you have no idea how broken things are.

Your tests should be short enough to fit its purpose in its name. The finer point in that sentence is "purpose."

What does this test?

def test_tile?
  listing = BusinessListing.new
  listing.tier = 10
  assert listing.tile?
end

How about this?

def test_should_detect_tile_tier_as_tile
  listing = BusinessListing.new
  listing.tier = 10
  assert listing.tile?
end

Starting every test with "test_should," you're encouraged to test for behavior. If this style excites you, check out rspec.