Use same assertion class as Rails, if loaded

Given this scenario:

* Using Rails 4.1
* Gemfile has `gem 'shoulda-matchers', require: false`
* spec_helper has `require 'shoulda/matchers'` following
  `require 'rspec/rails'`
* Using Spring to run tests

matchers that delegate to assertions in Rails (e.g. `render_template`
and `route`) will fail in the wrong way if used. They fail because in
order to use these assertions, we expect that the assertions will
raise a specific exception, an exception that corresponds to whichever
test framework that Rails is using. For Rails versions that used
Test::Unit, this is Test::Unit::AssertionFailedError. For current Rails
versions, which now use Minitest, this exception is Minitest::Assertion.

The problem is that instead of asking Rails which exception class it's
using, we are trying to detect this exception class ourselves (for
cases in which Rails is not being used). This leads to the wrong class
being detected: when using a Rails version that uses Minitest, we choose
Test::Unit::AssertionFailedError as the class. This happens using the
exact scenario above because even though shoulda-matchers is loaded
after rspec-rails, rspec-rails itself defines
Test::Unit::AssertionFailedError.

Also add Cucumber tests that confirms this exact scenario works.
This commit is contained in:
Elliot Winkler 2014-06-21 21:24:22 -06:00
parent a1e2f19782
commit 12543ede2e
19 changed files with 393 additions and 20 deletions

View File

@ -17,7 +17,9 @@ matrix:
- rvm: jruby-19mode
include:
- rvm: 1.9.2
gemfile: gemfiles/3.1_1.9.2.gemfile
gemfile: gemfiles/3.1-1.9.2.gemfile
- rvm: 1.9.2
gemfile: gemfiles/3.2-1.9.2.gemfile
- rvm: 1.9.3
gemfile: gemfiles/4.0.0.gemfile
- rvm: 1.9.3
@ -39,3 +41,5 @@ matrix:
exclude:
- rvm: 1.9.2
gemfile: gemfiles/3.1.gemfile
- rvm: 1.9.2
gemfile: gemfiles/3.2.gemfile

View File

@ -1,5 +1,10 @@
ruby_version = Gem::Version.new(RUBY_VERSION + '')
spring = proc do
gem 'spring'
gem 'spring-commands-rspec'
end
rails_3 = proc do
gem 'strong_parameters'
end
@ -14,7 +19,18 @@ rails_3_1 = proc do
gem 'uglifier', '>= 1.0.3'
end
rails_3_2 = proc do
instance_eval(&rails_3)
gem 'rails', '~> 3.2.13'
gem 'bcrypt-ruby', '~> 3.0.0'
gem 'jquery-rails'
gem 'sass-rails', '~> 3.2.3'
gem 'coffee-rails', '~> 3.2.1'
gem 'uglifier', '>= 1.0.3'
end
rails_4 = proc do
instance_eval(&spring)
gem 'uglifier', '>= 1.3.0'
gem 'coffee-rails', '~> 4.0.0'
gem 'jquery-rails'
@ -47,14 +63,15 @@ if Gem::Requirement.new('< 2').satisfied_by?(ruby_version)
end
end
appraise '3.2' do
instance_eval(&rails_3)
gem 'rails', '~> 3.2.13'
gem 'bcrypt-ruby', '~> 3.0.0'
gem 'jquery-rails'
gem 'sass-rails', '~> 3.2.3'
gem 'coffee-rails', '~> 3.2.1'
gem 'uglifier', '>= 1.0.3'
if Gem::Requirement.new('= 1.9.2').satisfied_by?(ruby_version)
appraise '3.2-1.9.2' do
instance_eval(&rails_3_2)
end
else
appraise '3.2' do
instance_eval(&rails_3_2)
instance_eval(&spring)
end
end
if Gem::Requirement.new('> 1.9.2').satisfied_by?(ruby_version)

View File

@ -2,6 +2,10 @@
### Bug fixes
* If you have a Rails >= 4.1 project and you are running tests using Spring,
matchers that depend on assertions within Rails' testing layer (e.g.
`render_template` and `route`) will no longer fail.
* Fix `permit` so that it can be used more than once in the same test.
* Revert change to `validate_uniqueness_of` made in 2.6.0 so that it no longer

View File

@ -15,8 +15,16 @@ RSpec::Core::RakeTask.new do |t|
end
Cucumber::Rake::Task.new do |t|
options = []
options << '--format' << (ENV['CUCUMBER_FORMAT'] || 'progress')
if Bundler.definition.dependencies.none? { |dependency| dependency.name == 'spring' }
options << '--profile' << 'without_spring'
end
t.fork = false
t.cucumber_opts = ['--format', (ENV['CUCUMBER_FORMAT'] || 'progress')]
t.cucumber_opts = options
end
task :default do

1
cucumber.yml Normal file
View File

@ -0,0 +1 @@
without_spring: --tags ~@spring

View File

@ -85,6 +85,38 @@ Feature: integrate with Rails
And the output should contain "should require name to be set"
And the output should contain "should respond with 200"
@spring
Scenario: A Rails application that uses RSpec, requires shoulda-matchers manually, and uses Spring to run tests
When I configure the application to use Spring
When I configure the application to use "spring-commands-rspec"
When I configure the application to use rspec-rails
And I configure the application to use "shoulda-matchers" from this project, disabling auto-require
And I run the rspec generator
And I require shoulda-matchers following rspec-rails
And I write to "spec/models/user_spec.rb" with:
"""
require 'spec_helper'
describe User do
it { should validate_presence_of(:name) }
end
"""
When I write to "spec/controllers/examples_controller_spec.rb" with:
"""
require 'spec_helper'
describe ExamplesController, "show" do
before { get :show }
it { should respond_with(:success) }
it { should_not render_template('foo') }
end
"""
When I run `bundle exec spring stop`
When I successfully run `bundle exec spring rspec spec -fs`
Then the output should contain "3 examples, 0 failures"
And the output should contain "should require name to be set"
And the output should contain "should respond with 200"
Scenario: generate a Rails application that mixes Rspec and Test::Unit
When I configure the application to use rspec-rails in test and development
And I configure the application to use "shoulda-matchers" from this project in test and development

View File

@ -32,11 +32,28 @@ When 'I generate a new rails application' do
end
end
When /^I configure the application to use "([^\"]+)"$/ do |name|
append_to_gemfile "gem '#{name}'"
steps %{And I install gems}
end
When 'I configure the application to use Spring' do
if rails_lt_4?
append_to_gemfile "gem 'spring'"
steps %{And I install gems}
end
end
When /^I configure the application to use "([^\"]+)" from this project$/ do |name|
append_to_gemfile "gem '#{name}', path: '#{PROJECT_ROOT}'"
steps %{And I install gems}
end
When /^I configure the application to use "([^\"]+)" from this project, disabling auto-require$/ do |name|
append_to_gemfile "gem '#{name}', path: '#{PROJECT_ROOT}', require: false"
steps %{And I install gems}
end
When /^I configure the application to use "([^\"]+)" from this project in test and development$/ do |name|
append_to_gemfile <<-GEMFILE
group :test, :development do
@ -72,6 +89,12 @@ When 'I configure the application to use shoulda-context' do
steps %{And I install gems}
end
When 'I require shoulda-matchers following rspec-rails' do
insert_line_after 'spec/spec_helper.rb',
"require 'rspec/rails'",
"require 'shoulda/matchers'"
end
When /^I set the "([^"]*)" environment variable to "([^"]*)"$/ do |key, value|
ENV[key] = value
end
@ -112,10 +135,10 @@ end
Then /^the output should indicate that (\d+) tests? (?:was|were) run/ do |number|
# Rails 4 has slightly different output than Rails 3 due to
# Test::Unit::TestCase -> MiniTest
if rails_4?
steps %{Then the output should match /#{number} (tests|runs), #{number} assertions, 0 failures, 0 errors, 0 skips/}
else
if rails_lt_4?
steps %{Then the output should contain "#{number} tests, #{number} assertions, 0 failures, 0 errors"}
else
steps %{Then the output should match /#{number} (tests|runs), #{number} assertions, 0 failures, 0 errors, 0 skips/}
end
end
@ -125,14 +148,16 @@ Then /^the output should indicate that (\d+) unit and (\d+) functional tests? we
total = n1.to_i + n2.to_i
# Rails 3 runs separate test suites in separate processes, but Rails 4 does
# not, so that's why we have to check for different things here
if rails_4?
steps %{Then the output should match /#{total} (tests|runs), #{total} assertions, 0 failures, 0 errors, 0 skips/}
else
if rails_lt_4?
steps %{Then the output should match /#{n1} tests, #{n1} assertions, 0 failures, 0 errors.+#{n2} tests, #{n2} assertions, 0 failures, 0 errors/}
else
steps %{Then the output should match /#{total} (tests|runs), #{total} assertions, 0 failures, 0 errors, 0 skips/}
end
end
module FileHelpers
RAILS_VERSION_IN_GEMFILE_PATH_REGEX = %r{/([^/]+?)(?:_.+)?\.gemfile$}
def append_to(path, contents)
in_current_dir do
File.open(path, 'a') do |file|
@ -154,9 +179,30 @@ module FileHelpers
end
end
def rails_4?
match = ORIGINAL_BUNDLE_VARS['BUNDLE_GEMFILE'].match(/(\d)\.\d\.(\d\.)?(\w+\.)?gemfile$/)
match.captures[0] == '4'
def insert_line_after(file_path, line, line_to_insert)
line += "\n"
line_to_insert += "\n"
in_current_dir do
contents = File.read(file_path)
index = contents.index(line) + line.length
contents.insert(index, line_to_insert)
File.open(file_path, 'w') { |f| f.write(contents) }
end
end
def rails_version_string
ORIGINAL_BUNDLE_VARS['BUNDLE_GEMFILE'].
match(RAILS_VERSION_IN_GEMFILE_PATH_REGEX).
captures[0]
end
def rails_version
@_rails_version ||= Gem::Version.new(rails_version_string)
end
def rails_lt_4?
Gem::Requirement.new('< 4').satisfied_by?(rails_version)
end
end

View File

@ -1,4 +1,5 @@
require 'aruba/cucumber'
require 'pry'
Before do
@aruba_timeout_seconds = 60 * 2

View File

@ -28,5 +28,7 @@ gem "jquery-rails"
gem "sass-rails", "~> 3.2.3"
gem "coffee-rails", "~> 3.2.1"
gem "uglifier", ">= 1.0.3"
gem "spring"
gem "spring-commands-rspec"
gemspec :path=>".././"

View File

@ -138,6 +138,9 @@ GEM
tilt (~> 1.3)
shoulda-context (1.2.0)
slop (3.4.7)
spring (1.1.3)
spring-commands-rspec (1.0.2)
spring (>= 0.9.1)
sprockets (2.2.2)
hike (~> 1.2)
multi_json (~> 1.0)
@ -186,6 +189,8 @@ DEPENDENCIES
sass-rails (~> 3.2.3)
shoulda-context (~> 1.2.0)
shoulda-matchers!
spring
spring-commands-rspec
sqlite3
strong_parameters
therubyrhino

View File

@ -0,0 +1,32 @@
# This file was generated by Appraisal
source "https://rubygems.org"
gem "appraisal", "~> 1.0.0.beta2"
gem "aruba"
gem "bourne", "~> 1.3"
gem "bundler", "~> 1.1"
gem "cucumber", "~> 1.1"
gem "pry"
gem "rake", ">= 0.9.2"
gem "rspec-rails", ">= 2.13.1", "< 3"
gem "yard"
gem "redcarpet"
gem "pygments.rb"
gem "watchr"
gem "shoulda-context", "~> 1.2.0"
gem "sqlite3", :platform=>:ruby
gem "activerecord-jdbc-adapter", :platform=>:jruby
gem "activerecord-jdbcsqlite3-adapter", :platform=>:jruby
gem "jdbc-sqlite3", :platform=>:jruby
gem "jruby-openssl", :platform=>:jruby
gem "therubyrhino", :platform=>:jruby
gem "strong_parameters"
gem "rails", "~> 3.2.13"
gem "bcrypt-ruby", "~> 3.0.0"
gem "jquery-rails"
gem "sass-rails", "~> 3.2.3"
gem "coffee-rails", "~> 3.2.1"
gem "uglifier", ">= 1.0.3"
gemspec :path=>".././"

View File

@ -0,0 +1,200 @@
PATH
remote: .././
specs:
shoulda-matchers (2.6.1)
activesupport (>= 3.0.0)
GEM
remote: https://rubygems.org/
specs:
actionmailer (3.2.18)
actionpack (= 3.2.18)
mail (~> 2.5.4)
actionpack (3.2.18)
activemodel (= 3.2.18)
activesupport (= 3.2.18)
builder (~> 3.0.0)
erubis (~> 2.7.0)
journey (~> 1.0.4)
rack (~> 1.4.5)
rack-cache (~> 1.2)
rack-test (~> 0.6.1)
sprockets (~> 2.2.1)
activemodel (3.2.18)
activesupport (= 3.2.18)
builder (~> 3.0.0)
activerecord (3.2.18)
activemodel (= 3.2.18)
activesupport (= 3.2.18)
arel (~> 3.0.2)
tzinfo (~> 0.3.29)
activeresource (3.2.18)
activemodel (= 3.2.18)
activesupport (= 3.2.18)
activesupport (3.2.18)
i18n (~> 0.6, >= 0.6.4)
multi_json (~> 1.0)
appraisal (1.0.0)
bundler
rake
thor (>= 0.14.0)
arel (3.0.3)
aruba (0.5.4)
childprocess (>= 0.3.6)
cucumber (>= 1.1.1)
rspec-expectations (>= 2.7.0)
bcrypt-ruby (3.0.1)
bourne (1.5.0)
mocha (>= 0.13.2, < 0.15)
builder (3.0.4)
childprocess (0.5.3)
ffi (~> 1.0, >= 1.0.11)
coderay (1.1.0)
coffee-rails (3.2.2)
coffee-script (>= 2.2.0)
railties (~> 3.2.0)
coffee-script (2.2.0)
coffee-script-source
execjs
coffee-script-source (1.7.0)
cucumber (1.3.15)
builder (>= 2.1.2)
diff-lcs (>= 1.1.3)
gherkin (~> 2.12)
multi_json (>= 1.7.5, < 2.0)
multi_test (>= 0.1.1)
diff-lcs (1.2.5)
erubis (2.7.0)
execjs (2.2.0)
ffi (1.9.3)
gherkin (2.12.2)
multi_json (~> 1.3)
hike (1.2.3)
i18n (0.6.9)
journey (1.0.4)
jquery-rails (3.1.0)
railties (>= 3.0, < 5.0)
thor (>= 0.14, < 2.0)
json (1.8.1)
mail (2.5.4)
mime-types (~> 1.16)
treetop (~> 1.4.8)
metaclass (0.0.4)
method_source (0.8.2)
mime-types (1.25.1)
mocha (0.14.0)
metaclass (~> 0.0.1)
multi_json (1.10.1)
multi_test (0.1.1)
polyglot (0.3.5)
posix-spawn (0.3.8)
pry (0.10.0)
coderay (~> 1.1.0)
method_source (~> 0.8.1)
slop (~> 3.4)
pygments.rb (0.6.0)
posix-spawn (~> 0.3.6)
yajl-ruby (~> 1.1.0)
rack (1.4.5)
rack-cache (1.2)
rack (>= 0.4)
rack-ssl (1.3.4)
rack
rack-test (0.6.2)
rack (>= 1.0)
rails (3.2.18)
actionmailer (= 3.2.18)
actionpack (= 3.2.18)
activerecord (= 3.2.18)
activeresource (= 3.2.18)
activesupport (= 3.2.18)
bundler (~> 1.0)
railties (= 3.2.18)
railties (3.2.18)
actionpack (= 3.2.18)
activesupport (= 3.2.18)
rack-ssl (~> 1.3.2)
rake (>= 0.8.7)
rdoc (~> 3.4)
thor (>= 0.14.6, < 2.0)
rake (10.3.2)
rdoc (3.12.2)
json (~> 1.4)
redcarpet (3.1.2)
rspec-collection_matchers (1.0.0)
rspec-expectations (>= 2.99.0.beta1)
rspec-core (2.99.0)
rspec-expectations (2.99.0)
diff-lcs (>= 1.1.3, < 2.0)
rspec-mocks (2.99.1)
rspec-rails (2.99.0)
actionpack (>= 3.0)
activemodel (>= 3.0)
activesupport (>= 3.0)
railties (>= 3.0)
rspec-collection_matchers
rspec-core (~> 2.99.0)
rspec-expectations (~> 2.99.0)
rspec-mocks (~> 2.99.0)
sass (3.3.8)
sass-rails (3.2.6)
railties (~> 3.2.0)
sass (>= 3.1.10)
tilt (~> 1.3)
shoulda-context (1.2.1)
slop (3.5.0)
sprockets (2.2.2)
hike (~> 1.2)
multi_json (~> 1.0)
rack (~> 1.0)
tilt (~> 1.1, != 1.3.0)
sqlite3 (1.3.9)
strong_parameters (0.2.3)
actionpack (~> 3.0)
activemodel (~> 3.0)
activesupport (~> 3.0)
railties (~> 3.0)
thor (0.19.1)
tilt (1.4.1)
treetop (1.4.15)
polyglot
polyglot (>= 0.3.1)
tzinfo (0.3.39)
uglifier (2.5.1)
execjs (>= 0.3.0)
json (>= 1.8.0)
watchr (0.7)
yajl-ruby (1.1.0)
yard (0.8.7.4)
PLATFORMS
ruby
DEPENDENCIES
activerecord-jdbc-adapter
activerecord-jdbcsqlite3-adapter
appraisal (~> 1.0.0.beta2)
aruba
bcrypt-ruby (~> 3.0.0)
bourne (~> 1.3)
bundler (~> 1.1)
coffee-rails (~> 3.2.1)
cucumber (~> 1.1)
jdbc-sqlite3
jquery-rails
jruby-openssl
pry
pygments.rb
rails (~> 3.2.13)
rake (>= 0.9.2)
redcarpet
rspec-rails (>= 2.13.1, < 3)
sass-rails (~> 3.2.3)
shoulda-context (~> 1.2.0)
shoulda-matchers!
sqlite3
strong_parameters
therubyrhino
uglifier (>= 1.0.3)
watchr
yard

View File

@ -21,6 +21,8 @@ gem "activerecord-jdbcsqlite3-adapter", :platform=>:jruby
gem "jdbc-sqlite3", :platform=>:jruby
gem "jruby-openssl", :platform=>:jruby
gem "therubyrhino", :platform=>:jruby
gem "spring"
gem "spring-commands-rspec"
gem "uglifier", ">= 1.3.0"
gem "coffee-rails", "~> 4.0.0"
gem "jquery-rails"

View File

@ -145,6 +145,9 @@ GEM
rdoc (~> 4.0, < 5.0)
shoulda-context (1.2.0)
slop (3.4.7)
spring (1.1.3)
spring-commands-rspec (1.0.2)
spring (>= 0.9.1)
sprockets (2.10.1)
hike (~> 1.2)
multi_json (~> 1.0)
@ -201,6 +204,8 @@ DEPENDENCIES
sdoc
shoulda-context (~> 1.2.0)
shoulda-matchers!
spring
spring-commands-rspec
sqlite3
therubyrhino
turbolinks

View File

@ -21,6 +21,8 @@ gem "activerecord-jdbcsqlite3-adapter", :platform=>:jruby
gem "jdbc-sqlite3", :platform=>:jruby
gem "jruby-openssl", :platform=>:jruby
gem "therubyrhino", :platform=>:jruby
gem "spring"
gem "spring-commands-rspec"
gem "uglifier", ">= 1.3.0"
gem "coffee-rails", "~> 4.0.0"
gem "jquery-rails"

View File

@ -145,6 +145,9 @@ GEM
rdoc (~> 4.0, < 5.0)
shoulda-context (1.2.0)
slop (3.4.7)
spring (1.1.3)
spring-commands-rspec (1.0.2)
spring (>= 0.9.1)
sprockets (2.10.1)
hike (~> 1.2)
multi_json (~> 1.0)
@ -201,6 +204,8 @@ DEPENDENCIES
sdoc
shoulda-context (~> 1.2.0)
shoulda-matchers!
spring
spring-commands-rspec
sqlite3
therubyrhino
turbolinks

View File

@ -21,6 +21,7 @@ gem "activerecord-jdbcsqlite3-adapter", :platform=>:jruby
gem "jdbc-sqlite3", :platform=>:jruby
gem "jruby-openssl", :platform=>:jruby
gem "therubyrhino", :platform=>:jruby
gem "spring-commands-rspec"
gem "uglifier", ">= 1.3.0"
gem "coffee-rails", "~> 4.0.0"
gem "jquery-rails"

View File

@ -151,6 +151,8 @@ GEM
shoulda-context (1.2.0)
slop (3.4.7)
spring (1.1.2)
spring-commands-rspec (1.0.2)
spring (>= 0.9.1)
sprockets (2.11.0)
hike (~> 1.2)
multi_json (~> 1.0)
@ -208,6 +210,7 @@ DEPENDENCIES
shoulda-context (~> 1.2.0)
shoulda-matchers!
spring
spring-commands-rspec
sqlite3
therubyrhino
turbolinks

View File

@ -1,6 +1,9 @@
module Shoulda
module Matchers
if Gem.ruby_version >= Gem::Version.new('1.8') && Gem.ruby_version < Gem::Version.new('1.9')
if defined?(ActiveSupport::TestCase)
# @private
AssertionError = ActiveSupport::TestCase::Assertion
elsif Gem.ruby_version >= Gem::Version.new('1.8') && Gem.ruby_version < Gem::Version.new('1.9')
require 'test/unit'
# @private
AssertionError = Test::Unit::AssertionFailedError