May 13, 2011
#51 Integration Tests With Cucumber
In this post, we'll work off our last post and introduce Cucumber for writing integration tests in user story format. In the process, we'll see how to employ internationalization/localization and custom Cucumber helper methods.
Setting Up
Cucumber apparently needs its own database in MongoDB, so we set that in our YAML configuration file. (Note the syntax for using the previously defined "defaults".) After that, we install and initialize Guard::Cucumber so that it will run our Cucumber tests automatically.
# mongoid.yml cucumber: <<: *defaults database: arailsdemo2_cucumber _____________________________________________________ # Gemfile group :development do ... gem 'guard-cucumber', '0.3.0', :require => false end _____________________________________________________ # Terminal $ bundle $ guard init cucumber $ guard _____________________________________________________ # Guardfile guard 'cucumber' do watch(%r{features/.+\.feature}) watch(%r{features/support/.+}) { 'features' } watch(%r{features/step_definitions/(.+)_steps\.rb}) { |m| Dir[File.join("**/#{m[1]}.feature")][0] || 'features' } end
Declarative Vs. Imperative
(Please see these Railscasts and the Cucumber wiki for a primer on Cucumber.) When using Cucumber, we start off writing the scenarios. There are two ways of writing features: declarative and imperative. The declarative way is more abstract and the imperative way is more granular.
The cucumber-rails gem generates some step definitions in web_steps.rb. These are "thin wrappers around the Capybara ... API." "If you use these step definitions as basis for your features you will quickly end up with features that are: hard to maintain and verbose to read." It's worth reading the links in that file. ([Ben Mabey], [Dan North] and [Elabs]) The point seems to be that we should write stories that are more readable and valuable to the stakeholders. This probably means writing scenarios in a more declarative ("get to the point") way, however, some scenarios will need to be more explicit when our story readers require it.
Feature: Creating View Templates As the site owner I want to create view templates via the web So that I can personalize my site # declarative Scenario: Creating a new valid view template Given I am authenticated When I create a view template for the home page Then the new template is used for the home page _____________________________________________________ # imperative Scenario: Creating a new valid view template Given I am authenticated When I visit the new view template path And I fill in "Name" with "home" And I fill in "Prefix" with "pages" And I fill in "Source" with "Mongoid view" And I click "Save" And I visit the home page Then I should see "Mongoid view"
Writing The Step Definitions
Keep in mind that what Cucumber provides is a higher level organization/abstraction to your Ruby code. As we saw in Post #41, we can write integration tests with just RSpec and Capybara. With the Cucumber step definitions, we are just wrapping those lines of RSpec and Capybara code into groupings. So in order to be comfortable writing step definitions, we should be comfortable with RSpec and Capybara.
# common_steps.rb Given /^I am authenticated$/ do true end _____________________________________________________ # view_template_steps.rb When /^I create a view template for the home page$/ do visit new_view_template_path fill_in "Name", :with => "home" fill_in "Prefix", :with => "pages" fill_in "Source", :with => "Mongoid view" click_button "Save" ViewTemplate.count.should == 1 end Then /^the new template is used for the home page$/ do visit root_path # Edit 5/22/11. Use root_path instead of root_url to keep Selenium happy page.should have_content "Mongoid view" end _____________________________________________________ # routes.rb Arailsdemo2::Application.routes.draw do ... root :to =>"pages#home"
Using I18n To Try To Avoid Brittle Tests
One of the problems with integration tests is that they are necessarily coupled to the details of our views. This makes our tests fail sometimes when we make minor changes to our view elements. One strategy to deal with this is to references view elements in more abstract terms. Let's try to do this with our locale file.
To see what I18n translations are currently present, we can drop down to the console, load the translations and view them. Now we add our own translations. To have Rails automatically translate some of our form elements, we will follow convention for defining attribute translations for our ViewTemplate model. Then for convenience, we create another translation for view_template directly. (This way we can use "view_template" instead of "mongoid.attributes.view_template".) Lastly we define some links that we're going to use.
# Terminal $ rails c > I18n.backend.load_translations > pp I18n.backend #<I18n::Backend::Simple:0x102e5f940 @initialized=false, @skip_syntax_deprecation=false, @translations= {:en=> {:datetime=> {:prompts=> {:day=>"Day", :month=>"Month", ... _____________________________________________________ # en.yml en: mongoid: attributes: view_template: &view_template name: Name prefix: Prefix partial: Partial source: Template Source Code locale: Locale formats: Format handlers: Handler view_template: <<: *view_template form: save: Save Template links: index: Show All Templates show: View This Template edit: Edit This Template new: Create A New Template destroy: Delete
Refactoring To Use I18n
In order to allow Cucumber to use I18n, we extend the ActionView::Helpers::TranslationHelper module. This is done by passing the module to Cucumber's "World" method. Now we can use our I18n translations in our tests and views.
If we decide to change the ViewTemplate's :source attribute name from "Template Source Code" to "Source Code", then we only need to do it in one place. Any views or tests that relied on that will not need to be modified.
# hooks.rb ... World(ActionView::Helpers::TranslationHelper) _____________________________________________________ # view_template_steps.rb When /^I create a view template for the home page$/ do visit new_view_template_path fill_in t("view_template.name"), :with => "home" fill_in t("view_template.prefix"), :with => "pages" fill_in t("view_template.source"), :with => "Mongoid view" click_button t("view_template.form.save") ViewTemplate.count.should == 1 end _____________________________________________________ # _form.html.haml = form_for @view_template do |f| .actions -# ... = f.submit t("view_template.form.save") _____________________________________________________ # show.html.haml %p#notice= notice %p %b= "#{t('view_template.name')}:" = @view_template.name %p %b= "#{t('view_template.prefix')}:" = @view_template.prefix -# ... = link_to t("view_template.links.edit"), edit_view_template_path(@view_template) \| = link_to t("view_template.links.index"), view_templates_path
A Translation Helper
To make it a little easier to use translations in our Cucumber steps, we'll create a helper method #t which uses the I18n gem directly. Basically, we'll take the translations and make them accessible as though they were methods (using "dots".) Since the #translations method is a protected Backend method, we use instance_eval to access it. Then we wrap it in a Hashie::Mash object. Now, instead of adding ActionView::Helpers::TranslationHelper module to the "World," we add our custom module. With this, we can forgo the parentheses and quotations marks when translating.
# Gemfile gem 'hashie', '1.0.0' _____________________________________________________ # Terminal $ bundle _____________________________________________________ # cucumber_helpers.rb module CucumberHelpers def t return @t if @t I18n.backend.load_translations @t = Hashie::Mash.new(I18n.backend.instance_eval{translations}).en end end World(CucumberHelpers) _____________________________________________________ # view_template_steps.rb When /^I create a view template for the home page$/ do visit new_view_template_path fill_in t.view_template.name, :with => "home" fill_in t.view_template.prefix, :with => "pages" fill_in t.view_template.source, :with => "Mongoid view" click_button t.view_template.form.save ViewTemplate.count.should == 1 end
A Form Helper
Let's simplify filling out a form. First we write the code that we want to have in our step definition. Then we define the #submit_form helper method. This method will visit the page that has the form of interest. Then it will loop through the hash of form elements and values and have Capybara set those values. Lastly, we submit the form.
With this helper in place, we can use our FactoryGirl :view_template factory in our step definitions. (At this point, we may have DRY'd things out too much since adding more levels of indirection can make things harder to debug. With our helper method, we're essentially adding our own DSL on top of Capybara's DSL which is just an interface to various drivers, and on top of it all is Cucumber. If you're looking for more Cucumber helper methods, though, try the Kelp gem.) Finally, we generalize our step definition to work for other models.
# view_template_steps.rb When /^I create a view template for the home page$/ do submit_form :view_template, :name => 'home', :prefix => 'pages', :source => 'Mongoid view' ViewTemplate.count.should == 1 end _____________________________________________________ # cucumber_helpers.rb module CucumberHelpers ... def submit_form(model, form_values) visit send(:"new_#{model}_path") # visit the form page model_translations = t.send(model) form_values.each_pair do |field, value| fill_in model_translations.send(field), :with => value end click_button model_translations.form.save # submit the form end end _____________________________________________________ # view_template.rb FactoryGirl.define do factory :view_template do name "home" prefix "pages" source "Mongoid view" end factory :home_page_view_template, :parent => :view_template do end end _____________________________________________________ # view_template_steps.rb =begin When /^I create a view template for the home page$/ do valid_attributes = Factory.attributes_for(:home_page_view_template) submit_form :view_template, valid_attributes ViewTemplate.count.should == 1 end =end Then /^the new template is used for the home page$/ do attributes = Factory.attributes_for(:home_page_view_template) visit root_url page.should have_content attributes[:source] end _____________________________________________________ # common_steps.rb # "When I create a view template for the home page" When /^I create an? (.+) (?:for|with(?: the)?) (.+)$/ do |model, factory| factory_name = model.gsub(" ", "_") valid_attributes = Factory.attributes_for(factory_name) submit_form factory_name, valid_attributes factory_name.classify.constantize.count.should == 1 end
Preventing Duplicate View Templates
Let's create a scenario where we try to create a duplicate template. We refactor the authentication step into the "Background." The "Given a view template exists" step is defined for us by FactoryGirl. (We just have to make sure that the "require 'factory_girl/step_definitions'" line is placed after the line where the factory files are required.) In the next step, we try to submit an invalid form. Here we extend our #submit_form helper method to deal with checkboxes and select elements. Finally, when we assert the error message that is displayed matches the error message that Mongoid has defined for us.
(Again, if we wanted, we could use a Factory in our step definition.)
# creating_view_templates.feature ... Background: Logged in Given I am authenticated Scenario: Creating a new valid view template ... Scenario: Submitting a duplicate view Given a view template exists When I attempt to create a duplicate template Then I should get a duplicate error _____________________________________________________ # env.rb ... Dir[File.expand_path(File.join(File.dirname(__FILE__),'..','..','spec','factories','*.rb'))].each {|f| require f} require 'factory_girl/step_definitions' ... _____________________________________________________ # view_template_steps.rb When /^I attempt to create a duplicate template$/ do submit_form :view_template, :name => " home \n", # :prefix => "\n pages\n\r", :prefix => " pages\n\r", # Edit 5/22/11. Remove newline to make Selenium happy :source => "stuff", :partial => :check, :handlers => {:select => "Erb"} ViewTemplate.count.should == 1 end _____________________________________________________ # cucumber_helpers.rb module CucumberHelpers ... def submit_form(model, form_values) visit send(:"new_#{model}_path") # visit the form page model_translations = t.send(model) form_values.each_pair do |field, value| case value when String fill_in model_translations.send(field), :with => value # textfield when Symbol # checkbox, radio send value, model_translations.send(field) # eg. "check('A Checkbox')" when Hash # select lists send value.keys.first, value.values.first, :from => model_translations.send(field) # eg. "select('Option', :from => 'Select Box')" else raise "Yo no comprende" end end click_button model_translations.form.save end end _____________________________________________________ # en.yml en: mongoid: ... errors: messages: duplicate: is a duplicate _____________________________________________________ # common_steps.rb Then /^I should get an? (.*) error$/ do |type| page.should have_content I18n.t("mongoid.errors.messages.#{type}") end _____________________________________________________ # view_template.rb FactoryGirl.define do ... factory :duplicate_view_template, :parent => :view_template do name " home \n" # prefix "\n pages\n\r" prefix " pages\n\r" # Edit 5/22/11 end _____________________________________________________ # view_template_steps.rb When /^I attempt to create a duplicate template$/ do whitespace_attributes = Factory.attributes_for(:duplicate_view_template) whitespace_attributes.merge!({:partial => :check, :handlers => {:select => "Erb"}}) submit_form :view_template, whitespace_attributes ViewTemplate.count.should == 1 end
Getting To Green
In order to get the scenario passing, we'll drop down to RSpec and do some BDD. By going to RSpec at this point, we're going to be duplicating the model testing that our Cucumber scenario is already testing. Some people might prefer just to rely on Cucumber and not have any RSpec unit tests here. However, others would tolerate the duplication in favor of having a complete set of unit tests. We'll have to figure out for ourselves the right balance of integration and unit testing to optimize time efficiency and test coverage. (Integration testing is much more important, however, when we're mocking and stubbing in our unit tests.)
# _form.html.haml = form_for @view_template do |f| -# ... .field = f.label :handlers = f.select :handlers, options_for_select([['Haml', 'haml'],['Erb', 'erb']]) _____________________________________________________ # view_template_spec.rb ... it { should validate_uniqueness_of(:name).scoped_to(:prefix) } ... context "validation" do before do @existing = Factory(:view_template) @template = Factory.build(:duplicate_view_template) @template.valid? end it "removes whitespace from the name and prefix" do @template.name.should == @existing.name @template.prefix.should == @existing.prefix end it "adds error message to :name if name/prefix is a duplicate" do @template.errors[:name].should == [I18n.t('mongoid.errors.messages.duplicate')] end end _____________________________________________________ # view_template.rb class ViewTemplate ... validates_uniqueness_of :name, :scope => :prefix, :message => :duplicate before_validation :strip_whitespace private def strip_whitespace self.name.strip! self.prefix.strip! end
Validating Haml
The last thing we'll do here is to make sure that any Haml that is submitted for a view template is valid. After writing the scenario, we write the steps and factory definition. Note that in the "I should get a haml error" step, we take advantage of I18n interpolation so that we can display the Haml error. Again we drop down to RSpec to get things to green. Notice here, we use RSpec's (v2.6) new .any_instance method to perform our stubs. Also, we create a spec to make sure that we allow a LocalJumpError from Haml::Engine. This is so we can put "= yield" into a layout view template.
# creating_view_template.feature ... Scenario: Submitting invalid Haml When I submit a view template form with invalid haml Then I should get a haml error _____________________________________________________ # common_steps.rb # "When I submit a view template form with invalid haml" When /^I submit a (.+) form with (.+)$/ do |model, prefix| model = model.gsub(" ", "_") factory = prefix.gsub(" ", "_") + "_" + model submit_form model, Factory.attributes_for(factory) end _____________________________________________________ # view_template.rb ... factory :invalid_haml_view_template, :parent => :view_template do source "= this_method_no_exist" end _____________________________________________________ # common_steps.rb ... Then /^I should get an? (.*) error$/ do |type| exception = "undefined local variable" if type == "haml" page.should have_content I18n.t("mongoid.errors.messages.#{type}", :exception => exception) end _____________________________________________________ # en.yml en: mongoid: ... errors: messages: ... haml: "has some stanky Haml: %{exception}" _____________________________________________________ # view_template_spec.rb ... context "Haml validation" do before { @template = Factory.build(:invalid_haml_view_template) } it "gives a :source error if invalid Haml is present" do Haml::Engine.any_instance.stub(:render) { raise "any instance!" } @template.valid? @template.errors[:source].first.should match /#{I18n.t('mongoid.errors.messages.haml', :exception => 'any instance!')}/ end it "allows 'no block given' exceptions" do Haml::Engine.any_instance.stub(:render) { raise LocalJumpError } @template.source = "=yield" @template.should be_valid end it "doesn't validate if handlers isn't 'haml' " do @template.handlers = 'erb' @template.should be_valid end end _____________________________________________________ # view_template.rb ... validate :haml_syntax_is_valid private ... def haml_syntax_is_valid begin Haml::Engine.new(source).render rescue Exception => e unless e.is_a? LocalJumpError errors.add(:source, :haml, :exception => e) end end if handlers == "haml" end
Conclusion
We took our first steps in learning how to use user stories to document and then drive the development of a feature in our application. We put in extra time to do this type of development, but hopefully it is worth the effort in the long run.
We've also employed I18n and developed Cucumber helper methods with the goal that these will aid in developing and maintaining our application. Whether these goals are realized remains to be seen, but using Cucumber and writing integration tests in general, forces us to be more deliberate in how we write code. We have to think more about our business domains, and we have to be careful about "over-specifying the implementation or specifying unnecessarily broad requirements that mix concerns."[Dan North] If we aren't more deliberate, then we're likely to end up with a bunch of brittle tests that just impede development.
(The source code is available in our Github repo.)
(Please login to submit comments.)
Comments