May 13, 2011

#51 Integration Tests With Cucumber

In this post, we'll work off our last post and introduce Cucumber for writing integration tests in user story format. In the process, we'll see how to employ internationalization/localization and custom Cucumber helper methods.

Setting Up

Cucumber apparently needs its own database in MongoDB, so we set that in our YAML configuration file. (Note the syntax for using the previously defined "defaults".) After that, we install and initialize Guard::Cucumber so that it will run our Cucumber tests automatically.

config/mongoid.yml, Gemfile, Terminal, Guardfile
# mongoid.yml
cucumber:
  <<: *defaults
  database: arailsdemo2_cucumber
_____________________________________________________

# Gemfile
group :development do 
  ...
  gem 'guard-cucumber', '0.3.0', :require => false
end
_____________________________________________________

# Terminal
$ bundle
$ guard init cucumber
$ guard
_____________________________________________________

# Guardfile
guard 'cucumber' do
  watch(%r{features/.+\.feature})
  watch(%r{features/support/.+})          { 'features' }
  watch(%r{features/step_definitions/(.+)_steps\.rb}) { |m| Dir[File.join("**/#{m[1]}.feature")][0] || 'features' }
end

Declarative Vs. Imperative

(Please see these Railscasts and the Cucumber wiki for a primer on Cucumber.) When using Cucumber, we start off writing the scenarios. There are two ways of writing features: declarative and imperative. The declarative way is more abstract and the imperative way is more granular.

The cucumber-rails gem generates some step definitions in web_steps.rb. These are "thin wrappers around the Capybara ... API." "If you use these step definitions as basis for your features you will quickly end up with features that are: hard to maintain and verbose to read." It's worth reading the links in that file. ([Ben Mabey], [Dan North] and [Elabs]) The point seems to be that we should write stories that are more readable and valuable to the stakeholders. This probably means writing scenarios in a more declarative ("get to the point") way, however, some scenarios will need to be more explicit when our story readers require it.

features/view_template/creating_view_template.feature
Feature: Creating View Templates
  As the site owner
  I want to create view templates via the web
  So that I can personalize my site

  # declarative
  Scenario: Creating a new valid view template
    Given I am authenticated
    When I create a view template for the home page
    Then the new template is used for the home page
_____________________________________________________

  # imperative
  Scenario: Creating a new valid view template
    Given I am authenticated
    When I visit the new view template path
    And I fill in "Name" with "home"
    And I fill in "Prefix" with "pages"
    And I fill in "Source" with "Mongoid view"
    And I click "Save"
    And I visit the home page
    Then I should see "Mongoid view"

Writing The Step Definitions

Keep in mind that what Cucumber provides is a higher level organization/abstraction to your Ruby code. As we saw in Post #41, we can write integration tests with just RSpec and Capybara. With the Cucumber step definitions, we are just wrapping those lines of RSpec and Capybara code into groupings. So in order to be comfortable writing step definitions, we should be comfortable with RSpec and Capybara.

features/step_definitions/common_steps.rb and view_template_steps.rb, config/routes.rb
# common_steps.rb
Given /^I am authenticated$/ do
  true
end
_____________________________________________________

# view_template_steps.rb
When /^I create a view template for the home page$/ do
  visit new_view_template_path
  fill_in "Name", :with => "home"
  fill_in "Prefix", :with => "pages"
  fill_in "Source", :with => "Mongoid view"
  click_button "Save"
  ViewTemplate.count.should == 1
end

Then /^the new template is used for the home page$/ do
  visit root_path  # Edit 5/22/11. Use root_path instead of root_url to keep Selenium happy
  page.should have_content "Mongoid view"
end
_____________________________________________________

# routes.rb
Arailsdemo2::Application.routes.draw do
  ...
  root :to =>"pages#home"

Using I18n To Try To Avoid Brittle Tests

One of the problems with integration tests is that they are necessarily coupled to the details of our views. This makes our tests fail sometimes when we make minor changes to our view elements. One strategy to deal with this is to references view elements in more abstract terms. Let's try to do this with our locale file.

To see what I18n translations are currently present, we can drop down to the console, load the translations and view them. Now we add our own translations. To have Rails automatically translate some of our form elements, we will follow convention for defining attribute translations for our ViewTemplate model. Then for convenience, we create another translation for view_template directly. (This way we can use "view_template" instead of "mongoid.attributes.view_template".) Lastly we define some links that we're going to use.

Terminal, config/locales/en.yml
# Terminal
$ rails c
> I18n.backend.load_translations
> pp I18n.backend
#<I18n::Backend::Simple:0x102e5f940
 @initialized=false,
 @skip_syntax_deprecation=false,
 @translations=
  {:en=>
    {:datetime=>
      {:prompts=>
        {:day=>"Day",
         :month=>"Month",
...
_____________________________________________________

# en.yml
en:
  mongoid:
    attributes:
      view_template: &view_template
        name: Name
        prefix: Prefix
        partial: Partial
        source: Template Source Code
        locale: Locale
        formats: Format
        handlers: Handler
  view_template:
    <<: *view_template
    form:
      save: Save Template
    links:
      index: Show All Templates
      show: View This Template
      edit: Edit This Template
      new: Create A New Template
      destroy: Delete

Refactoring To Use I18n

In order to allow Cucumber to use I18n, we extend the ActionView::Helpers::TranslationHelper module. This is done by passing the module to Cucumber's "World" method. Now we can use our I18n translations in our tests and views.

If we decide to change the ViewTemplate's :source attribute name from "Template Source Code" to "Source Code", then we only need to do it in one place. Any views or tests that relied on that will not need to be modified.

features/support/hooks.rb, features/step_definitions/view_template_steps.rb, app/views/view_layouts/_form.html.haml and show.html.haml
# hooks.rb
...
World(ActionView::Helpers::TranslationHelper)
_____________________________________________________
# view_template_steps.rb
When /^I create a view template for the home page$/ do
  visit new_view_template_path
  fill_in t("view_template.name"), :with => "home"
  fill_in t("view_template.prefix"), :with => "pages"
  fill_in t("view_template.source"), :with => "Mongoid view"
  click_button t("view_template.form.save")
  ViewTemplate.count.should == 1
end
_____________________________________________________

# _form.html.haml
= form_for @view_template do |f|
  .actions
  -# ...
    = f.submit t("view_template.form.save")
_____________________________________________________

# show.html.haml

%p#notice= notice

%p
  %b= "#{t('view_template.name')}:"
  = @view_template.name
%p
  %b= "#{t('view_template.prefix')}:"
  = @view_template.prefix
-# ...

= link_to t("view_template.links.edit"), edit_view_template_path(@view_template)
\|
= link_to t("view_template.links.index"), view_templates_path

A Translation Helper

To make it a little easier to use translations in our Cucumber steps, we'll create a helper method #t which uses the I18n gem directly. Basically, we'll take the translations and make them accessible as though they were methods (using "dots".) Since the #translations method is a protected Backend method, we use instance_eval to access it. Then we wrap it in a Hashie::Mash object. Now, instead of adding ActionView::Helpers::TranslationHelper module to the "World," we add our custom module. With this, we can forgo the parentheses and quotations marks when translating.

Gemfile, Terminal, features/support/cucumber_helperss.rb, features/step_definitions/view_template_steps.rb
# Gemfile
gem 'hashie', '1.0.0'
_____________________________________________________

# Terminal
$ bundle
_____________________________________________________

# cucumber_helpers.rb
module CucumberHelpers
  def t
    return @t if @t
    I18n.backend.load_translations
    @t = Hashie::Mash.new(I18n.backend.instance_eval{translations}).en
  end
end

World(CucumberHelpers)
_____________________________________________________

# view_template_steps.rb
When /^I create a view template for the home page$/ do
  visit new_view_template_path
  fill_in t.view_template.name, :with => "home"
  fill_in t.view_template.prefix, :with => "pages"
  fill_in t.view_template.source, :with => "Mongoid view"
  click_button t.view_template.form.save
  ViewTemplate.count.should == 1
end

A Form Helper

Let's simplify filling out a form. First we write the code that we want to have in our step definition. Then we define the #submit_form helper method. This method will visit the page that has the form of interest. Then it will loop through the hash of form elements and values and have Capybara set those values. Lastly, we submit the form.

With this helper in place, we can use our FactoryGirl :view_template factory in our step definitions. (At this point, we may have DRY'd things out too much since adding more levels of indirection can make things harder to debug. With our helper method, we're essentially adding our own DSL on top of Capybara's DSL which is just an interface to various drivers, and on top of it all is Cucumber. If you're looking for more Cucumber helper methods, though, try the Kelp gem.) Finally, we generalize our step definition to work for other models.

feature/step_definitions/view_template_steps.rb, features/support/cucumber_helpers.rb, spec/factories/view_template.rb, features/step_definitions/common_steps.rb
# view_template_steps.rb
When /^I create a view template for the home page$/ do
  submit_form :view_template, :name   => 'home',
                              :prefix => 'pages',
                              :source => 'Mongoid view'
  ViewTemplate.count.should == 1
end
_____________________________________________________

# cucumber_helpers.rb
module CucumberHelpers
  ...
  def submit_form(model, form_values)
    visit send(:"new_#{model}_path")  # visit the form page
    model_translations = t.send(model)
    form_values.each_pair do |field, value|
      fill_in model_translations.send(field), :with => value
    end
    click_button model_translations.form.save  # submit the form
  end
end
_____________________________________________________

# view_template.rb
FactoryGirl.define do
  factory :view_template do
    name "home"
    prefix "pages"
    source "Mongoid view"
  end

  factory :home_page_view_template, :parent => :view_template do
  end
end
_____________________________________________________

# view_template_steps.rb
=begin
When /^I create a view template for the home page$/ do
  valid_attributes = Factory.attributes_for(:home_page_view_template)
  submit_form :view_template, valid_attributes
  ViewTemplate.count.should == 1
end
=end

Then /^the new template is used for the home page$/ do
  attributes = Factory.attributes_for(:home_page_view_template)
  visit root_url

  page.should have_content attributes[:source]
end
_____________________________________________________

# common_steps.rb
# "When I create a view template for the home page"
When /^I create an? (.+) (?:for|with(?: the)?) (.+)$/ do |model, factory|
  factory_name = model.gsub(" ", "_")
  valid_attributes = Factory.attributes_for(factory_name)
  submit_form factory_name, valid_attributes
  factory_name.classify.constantize.count.should == 1
end

Preventing Duplicate View Templates

Let's create a scenario where we try to create a duplicate template. We refactor the authentication step into the "Background." The "Given a view template exists" step is defined for us by FactoryGirl. (We just have to make sure that the "require 'factory_girl/step_definitions'" line is placed after the line where the factory files are required.) In the next step, we try to submit an invalid form. Here we extend our #submit_form helper method to deal with checkboxes and select elements. Finally, when we assert the error message that is displayed matches the error message that Mongoid has defined for us.

(Again, if we wanted, we could use a Factory in our step definition.)

[features/]creating_view_templates.feature, []support/env.rb and cucumber_helpers.rb, []step_definitions/view_template_steps.rb and common_steps.rb, config/locales/en.yml, spec/factories/view_template.rb
# creating_view_templates.feature
...
  Background: Logged in
    Given I am authenticated

  Scenario: Creating a new valid view template
    ...

  Scenario: Submitting a duplicate view
    Given a view template exists
    When I attempt to create a duplicate template
    Then I should get a duplicate error
_____________________________________________________

# env.rb
...
Dir[File.expand_path(File.join(File.dirname(__FILE__),'..','..','spec','factories','*.rb'))].each {|f| require f}
require 'factory_girl/step_definitions'
...
_____________________________________________________

# view_template_steps.rb
When /^I attempt to create a duplicate template$/ do
  submit_form :view_template, :name => "  home  \n",
#                             :prefix => "\n  pages\n\r",
                              :prefix => "  pages\n\r",    # Edit 5/22/11. Remove newline to make Selenium happy
                              :source => "stuff",
                              :partial => :check,
                              :handlers => {:select => "Erb"}
  ViewTemplate.count.should == 1
end
_____________________________________________________

# cucumber_helpers.rb
module CucumberHelpers
  ...
  def submit_form(model, form_values)
    visit send(:"new_#{model}_path")  # visit the form page
    model_translations = t.send(model)

    form_values.each_pair do |field, value|
      case value
      when String
        fill_in model_translations.send(field), :with => value # textfield
      when Symbol  # checkbox, radio
        send value, model_translations.send(field) # eg. "check('A Checkbox')"
      when Hash  # select lists
        send value.keys.first, value.values.first, :from => model_translations.send(field) # eg. "select('Option', :from => 'Select Box')"
      else
        raise "Yo no comprende"
      end
    end

    click_button model_translations.form.save
  end
end

_____________________________________________________

# en.yml
en:
  mongoid:
    ...
    errors:
      messages:
        duplicate: is a duplicate
_____________________________________________________

# common_steps.rb
Then /^I should get an? (.*) error$/ do |type|
  page.should have_content I18n.t("mongoid.errors.messages.#{type}")
end
_____________________________________________________

# view_template.rb
FactoryGirl.define do
  ...
  factory :duplicate_view_template, :parent => :view_template do
    name "  home  \n"
#   prefix "\n  pages\n\r"
    prefix "  pages\n\r"    # Edit 5/22/11
  end
_____________________________________________________

# view_template_steps.rb
When /^I attempt to create a duplicate template$/ do
  whitespace_attributes = Factory.attributes_for(:duplicate_view_template)
  whitespace_attributes.merge!({:partial => :check, :handlers => {:select => "Erb"}})
  submit_form :view_template, whitespace_attributes

  ViewTemplate.count.should == 1
end

Getting To Green

In order to get the scenario passing, we'll drop down to RSpec and do some BDD. By going to RSpec at this point, we're going to be duplicating the model testing that our Cucumber scenario is already testing. Some people might prefer just to rely on Cucumber and not have any RSpec unit tests here. However, others would tolerate the duplication in favor of having a complete set of unit tests. We'll have to figure out for ourselves the right balance of integration and unit testing to optimize time efficiency and test coverage. (Integration testing is much more important, however, when we're mocking and stubbing in our unit tests.)

app/views/view_template/_form.html.haml, spec/models/view_template_spec.rb, app/models/view_template.rb
# _form.html.haml
= form_for @view_template do |f|
  -# ...
  .field
    = f.label :handlers
    = f.select :handlers, options_for_select([['Haml', 'haml'],['Erb', 'erb']])
_____________________________________________________
# view_template_spec.rb
...
  it { should validate_uniqueness_of(:name).scoped_to(:prefix) }
...
  context "validation" do
    before do
      @existing = Factory(:view_template)
      @template = Factory.build(:duplicate_view_template)
      @template.valid?
    end

    it "removes whitespace from the name and prefix" do
      @template.name.should == @existing.name
      @template.prefix.should == @existing.prefix
    end

    it "adds error message to :name if name/prefix is a duplicate" do
      @template.errors[:name].should == [I18n.t('mongoid.errors.messages.duplicate')]
    end
  end
_____________________________________________________

# view_template.rb
class ViewTemplate
  ...
  validates_uniqueness_of :name, :scope => :prefix, :message => :duplicate

  before_validation :strip_whitespace

  private

  def strip_whitespace
    self.name.strip!
    self.prefix.strip!
  end

Validating Haml

The last thing we'll do here is to make sure that any Haml that is submitted for a view template is valid. After writing the scenario, we write the steps and factory definition. Note that in the "I should get a haml error" step, we take advantage of I18n interpolation so that we can display the Haml error. Again we drop down to RSpec to get things to green. Notice here, we use RSpec's (v2.6) new .any_instance method to perform our stubs. Also, we create a spec to make sure that we allow a LocalJumpError from Haml::Engine. This is so we can put "= yield" into a layout view template.

[features/]creating_view_template.feature, []/step_definitions/common_steps.rb, spec/factories/view_template.rb, config/locales/en.yml, spec/models/view_template_spec.rb, app/models/view_template.rb
# creating_view_template.feature
...
  Scenario: Submitting invalid Haml
    When I submit a view template form with invalid haml
    Then I should get a haml error
_____________________________________________________

# common_steps.rb
# "When I submit a view template form with invalid haml"
When /^I submit a (.+) form with (.+)$/ do |model, prefix|
  model = model.gsub(" ", "_")
  factory = prefix.gsub(" ", "_") + "_" + model
  submit_form model, Factory.attributes_for(factory)
end
_____________________________________________________

# view_template.rb
...
  factory :invalid_haml_view_template, :parent => :view_template do
    source "= this_method_no_exist"
  end
_____________________________________________________

# common_steps.rb
...
Then /^I should get an? (.*) error$/ do |type|
  exception = "undefined local variable" if type == "haml"
  page.should have_content I18n.t("mongoid.errors.messages.#{type}", :exception => exception)
end
_____________________________________________________

# en.yml
en:
  mongoid:
    ...
    errors:
      messages:
        ...
        haml: "has some stanky Haml: %{exception}"
_____________________________________________________

# view_template_spec.rb
...
  context "Haml validation" do
    before { @template = Factory.build(:invalid_haml_view_template) }

    it "gives a :source error if invalid Haml is present" do
      Haml::Engine.any_instance.stub(:render) { raise "any instance!" }
      @template.valid?
      @template.errors[:source].first.should match /#{I18n.t('mongoid.errors.messages.haml', :exception => 'any instance!')}/
    end

    it "allows 'no block given' exceptions" do
      Haml::Engine.any_instance.stub(:render) { raise LocalJumpError }
      @template.source = "=yield"
      @template.should be_valid
    end

    it "doesn't validate if handlers isn't 'haml' " do
      @template.handlers = 'erb'
      @template.should be_valid
    end
  end
_____________________________________________________

# view_template.rb
...
  validate :haml_syntax_is_valid

  private
  ...
  def haml_syntax_is_valid
    begin
      Haml::Engine.new(source).render
    rescue Exception => e
      unless e.is_a? LocalJumpError
        errors.add(:source, :haml, :exception => e)
      end
    end if handlers == "haml"
  end

Conclusion

We took our first steps in learning how to use user stories to document and then drive the development of a feature in our application. We put in extra time to do this type of development, but hopefully it is worth the effort in the long run.

We've also employed I18n and developed Cucumber helper methods with the goal that these will aid in developing and maintaining our application. Whether these goals are realized remains to be seen, but using Cucumber and writing integration tests in general, forces us to be more deliberate in how we write code. We have to think more about our business domains, and we have to be careful about "over-specifying the implementation or specifying unnecessarily broad requirements that mix concerns."[Dan North] If we aren't more deliberate, then we're likely to end up with a bunch of brittle tests that just impede development.

(The source code is available in our Github repo.)

    Comments

(Please login to submit comments.)



View All Posts