We weren't installing the dependencies for the integration tests so the default Rake task was failing upon first run on a new machine.