FileUtils#mkdir_p causes test suites to not register if the mkdir_p argument already exists. Easily worked around but seems like a bug. Here's a minimal example: require 'test/unit' require 'fileutils' FileUtils.mkdir_p "some_dir" class TrueTest < Test::Unit::TestCase def test_true; assert true; end end It works as long as "some_dir" doesn't already exist, mkdir_p is not called, or mkdir_p is called from within a TrueTest instance method rather than at load time.