June 12, 2011

#56 Cloud Foundry - Creating An API Client

Now that we've seen how Cloud Foundry's Cloud Controller generates authentication tokens, we'll look at how we can interact with with the Cloud Controller API. Cloud Foundry's VMC gem allows a user to interact with the Cloud Controller to do things such as logging in and pushing a new application into Cloud Foundry. We use the VMC gem as a reference to guide us in how to develop an an API wrapper/interface client.

Establishing The Gem

We'll develop a gem that will allow us to "login" into Cloud Foundry. Then we wil create some interfaces to a couple of the Cloud Controller API points. We create a new gem skeleton and add the necessary gem dependencies. Since the Cloud Controller exposes a JSON-based API, we will include the mutli_json and yajl-ruby gems. multi_json "is a general-purpose swappable JSON backend library." (The other new gems will be discussed below.)

Terminal, vmc.gemspec

# Terminal
$ bundle gem vmc
__________________________________________________

# vmc.gemspec
...
Gem::Specification.new do |s|
  ...
  s.add_development_dependency 'rspec', '~> 2.6'
  s.add_development_dependency 'webmock', '~> 1.6'
  s.add_development_dependency 'guard-rspec', '~> 0.4'

  s.add_runtime_dependency 'multi_json', '~> 1.0'
  s.add_runtime_dependency 'yajl-ruby', '~> 0.8'
  s.add_runtime_dependency 'faraday_middleware', '~> 0.6'
  s.add_runtime_dependency 'rash', '~> 0.3'
end

Webmock

As we develop our client, we will avoid direct interaction with the Cloud Controller. We will use Webmock to block any HTTP requests and instead provide mock responses for our tests. To use Webmock, we require webmock/rspec is our spec_helper file.

When defining a stub request, we have the option of returning a simulated response from the blocked HTTP request. We keep these responses in files within the spec/fixtures directory. The #fixture and #fixture_path methods help retrieve these responses. (These files are from Cloud Foundry's VMC spec/assets folder.)

(Recall from the last post that in order to obtain an authorization token from the Cloud Controller, we have to login by sending a POST request to "/users/*email/tokens" that contains our password in the body of the request. From the VCAP README: "The pre-defined domain *.vcap.me maps to local host, so when you use this setup, the end result is that your development environment is available at http://api.vcap.me.")

spec/spec_helper.rb, spec/fixtures/login_succes.txt & login_fail.txt

# spec_helper.rb
require 'vmc'
require 'rspec'
require 'webmock/rspec'

def fixture_path
  File.expand_path("../fixtures", __FILE__)
end

def fixture(file)
  File.new(fixture_path + '/' + file)
end

def stub_login(status)
  fixture_file = status == :success ? "login_success.txt" : "login_fail.txt"
  stub_request(:post, "http://api.vcap.me/users/foo@example.com/tokens").
          with(:body => {:password => 'foo'}).
          to_return(fixture(fixture_file))
end
__________________________________________________

# login_success.txt
HTTP/1.1 200 OK
Server: nginx/0.7.65
Date: Thu, 03 Mar 2011 18:26:45 GMT
Content-Type: application/json
Connection: keep-alive
Keep-Alive: timeout=20
Content-Length: 28

{"token":"valid_auth_token"}
__________________________________________________

# login_fail.txt
HTTP/1.1 403 Forbidden
Server: nginx/0.7.65
Date: Thu, 03 Mar 2011 18:29:40 GMT
Content-Type: application/json
Connection: keep-alive
Keep-Alive: timeout=20
Content-Length: 52

{"code":200,"description":"Operation not permitted"}

The Login Spec

Now that our stub requests for logging in are stup, we work on the #login method. Our specifications for this method are: 1) It takes the user's email and password as arguments. 2) It returns an auth token. 3) It sets the user and auth_token attributes of the client instance if logging in was successful. 4) If not successful, an exception is raised.

spec/vmc/client_spec.rb

# client_spec.rb
require "spec_helper"

describe VMC::Client do
  describe "#login" do
    def login
      @client = subject
      @auth_token = @client.login('foo@example.com', 'foo')
    end

    context "when successful" do
      before do
        stub_login(:success)
        login
      end

      it "sets the user" do
        @client.user.should == 'foo@example.com'
      end

      it "sets and returns the auth_token" do
        @client.auth_token.should == @auth_token
        @auth_token.should == "valid_auth_token"
      end
    end

    it "raises an exception when failed" do
      stub_login(:failed)
      expect { login }.to raise_error VMC::Client::TargetError
    end
  end
end

Faraday

Cloud Foundry's VMC gem uses Rest Client to make its HTTP calls. For our client, we will use Faraday. Faraday provides a "modular HTTP client library using middleware heavily inspired by Rack.". We use a Connection object to send an HTTP request and also receive the response. In building these connection objects, we can include modules that can modify the requests as they go out and responses as they come back. Faraday supports many HTTP adapters including Typhoeus and EM-Synchrony (fiber wrapped Event Machine HTTP requests.) For our purposes, we will be using the NET::HTTP adapter.

The first thing we need do is build a connection object. When using the block syntax, we can include middleware to modify requests or responses as needed. Faraday::Request::JSON will take an outgoing request and encode the body to JSON using Yajl. The other two middlewares come from the faraday_middleware gem. ParseJson will decode the incoming JSON response body into a hash. After that, Rashify converts the hash into a Hashie::Rash object. Rash "subclasses Hashie::Mash to convert all [camel case] keys in the hash to underscore."

Once we have a connection object, all we have to do to make a POST request is call #post on the connection with the path of interest and any parameters that are needed. A Faraday::Response object is returned, and we can get the body of the response.

lib/vmc/client/connection.rb, lib/vmc.rb

# client.rb
require 'faraday_middleware'

module VMC
  class Client
    class TargetError < RuntimeError; end

    attr_accessor :auth_token
    attr_reader :user

    def login(username, password)
      connection = Faraday.new(:url => 'http://api.vcap.me') do |builder|
        builder.use Faraday::Request::JSON
        builder.use Faraday::Response::Rashify
        builder.use Faraday::Response::ParseJson

        builder.adapter :net_http
      end

      response = connection.post("/users/#{username}/tokens", :password => password).body

      raise TargetError if response.code == 200
      @user = username
      @auth_token = response.token
    end
  end
__________________________________________________

# vmc.rb
require "vmc/version"
require 'vmc/client'

module VMC
end

Refactoring

Our gem structure will differ from Cloud Foundry's VMC gem and will mimic the structure of the Twitter and Octokit gems which are other API wrappers. These gems also use Faraday for their HTTP requests. This structure will organize our Client class into component modules.

We keep some the Cloud Foundry VMC constants in our base VMC module. We make our Client configurable, so that we can change the target url and HTTP adapter. In our Request module, we change to passing a block to the connection action (i.e. :post) that will allow us to set some request information before the connection is made. In addition, we create a #post helper method. The #login method is placed into the Authentication module. Finally, we move custom error classes into the errors.rb file.

lib/vmc.rb & client.rb, lib/client/connection.rb & request.rb & authentication.rb & errors.rb

# vmc.rb
...
module VMC
  DEFAULT_LOCAL_TARGET = 'http://api.vcap.me'
  USERS_PATH           = '/users'
end
__________________________________________________

# client.rb
require 'vmc/client/authentication'
require 'vmc/client/connection'
require 'vmc/client/errors'
require 'vmc/client/request'

module VMC
  class Client
    attr_accessor :http_adapter, :target_url

    def initialize
      @target_url = VMC::DEFAULT_LOCAL_TARGET
      @http_adapter = :net_http
    end

    include Authentication
    include Connection
    include Request
  end
__________________________________________________

# connection.rb
require 'faraday_middleware'

module VMC
  class Client
    module Connection
      private

      def connection
        connection = Faraday.new(:url => target_url) do |builder|
          builder.use Faraday::Request::JSON
          builder.use Faraday::Response::Rashify
          builder.use Faraday::Response::ParseJson

          builder.adapter http_adapter
        end
      end
__________________________________________________

# request.rb
module VMC
  class Client
    module Request
      def post(path, options={})
        request(:post, path, options)
      end

      private

      def request(action, path, options)
        response = connection.send(action, path) do |request|
          request.body = options[:body] if options[:body]
        end

        response.body
      end
__________________________________________________

#authentication.rb
module VMC
  class Client
    module Authentication
      attr_accessor :auth_token
      attr_reader :user

      def login(username, password)
        response = post("#{VMC::USERS_PATH}/#{username}/tokens",
                        :body => { :password => password })
        raise TargetError if response.code == 200

        @user = username
        @auth_token = response.token
      end
__________________________________________________

# errors.rb
module VMC
  class Client
    class TargetError < RuntimeError; end

The Info Spec

If we look at the Cloud Controller's routes files, we see additional API points that we can access. A request to "/info" will provide some general information about Cloud Foundry. If the incoming request provides a valid authorization token within the headers, the Cloud Controller will also return some user specific data. Thus this API point provides a means to see if the user's auth_token is valid.

spec/vmc/client/info_spec.rb, spec/fixtures/info_authenticated.txt & info_returned.txt

# info_spec.rb
describe VMC::Client::Info do
  describe ".info" do
    it "obtains user and general info when a valid auth token is present" do
      stub_request(:get, "http://api.vcap.me/info").
              with(:headers => {'AUTHORIZATION' => 'token'}).
              to_return(fixture('info_authenticated.txt'))

      client = VMC::Client.new('token')
      info = client.info

      info.support.should == "ac-support@vmware.com"
      info.user.should == 'foo@example.com'
    end

    it "obtains just the general info when an auth token is not present" do
      stub_request(:get, "http://api.vcap.me/info").
              to_return(fixture('info_returned.txt'))

      client = VMC::Client.new
      info = client.info

      info.support.should == "ac-support@vmware.com"
      info.user.should be_nil
    end
  end
__________________________________________________

# info_authenticated.txt
HTTP/1.1 200 OK
Server: nginx/0.7.65
Date: Thu, 03 Mar 2011 19:25:34 GMT
Content-Type: application/json
Connection: keep-alive
Keep-Alive: timeout=20
Content-Length: 380

{
  "name": "vcap",
  "build": "3465a13ab528443f1afcd3c9c2861a078549b8e5",
  "support": "ac-support@vmware.com",
  "version": 0.999,
  "limits": {
    "apps": 50,
    "memory": 8192,
    "app_uris": 4,
    "services": 4
  },
  "user": "foo@example.com",
  "description": "VMware's Cloud Application Platform",
  "usage": {
    "apps": 1,
    "memory": 128,
    "services": 0
  }
}
__________________________________________________

# info_returned.txt
HTTP/1.1 200 OK
Server: nginx/0.7.65
Date: Thu, 03 Mar 2011 19:04:04 GMT
Content-Type: application/json
Connection: keep-alive
Keep-Alive: timeout=20
Content-Length: 189

{
  "name": "vcap",
  "build": "3465a13ab528443f1afcd3c9c2861a078549b8e5",
  "support": "ac-support@vmware.com",
  "version": 0.999,
  "description": "VMware's Cloud Application Platform"
}

The Info Module

Our Client class can now be initialized with an auth_token. The #info method with just make a GET request to the /info path using the new #get helper method. Since other methods may be passing in the auth_token in the request headers, we deal with that in the #request method.

[lib/]vmc/client.rb, []vmc/client/info.rb, []vmc.rb, []client/request.rb

# client.rb
require 'vmc/client/info'
...
module VMC
  class Client
    ...
    def initialize(auth_token=nil)
      @auth_token = auth_token
      ...
    end

    include Info
    ...
__________________________________________________

# info.rb
module VMC
  class Client
    module Info
      def info
        get(VMC::INFO_PATH)
      end
__________________________________________________

# vmc.rb
module VMC
  ...
  INFO_PATH            = '/info'
end
__________________________________________________

# request.rb
module VMC
  class Client
    module Request
      def get(path, options={})
        request(:get, path, options)
      end

      private

      def request(action, path, options)
        headers = auth_token ? { 'AUTHORIZATION' => auth_token } : {}
        headers.merge!(options[:headers]) if options[:headers]

        response = connection.send(action, path) do |request|
          request.body = options[:body] if options[:body]
          request.headers = headers unless headers.empty?
        end

        response.body
      end

Th App Spec

When a user attempts to create a new application (via the method VMC::Cli::Command::Apps#push), one of the first steps is to check that the app name that we are using doesn't already exists (using a method called #app_exists?.) This steps relies on the Client#app_info method.

We will reproduce the functionality of this method. There are three situations to consider. The first is when we already have a valid auth_token. In this case, we make a request to /info which lets us know if the token is valid. If it is, then we send the token in a request to /apps/[app_name] to receive information about that app. In the second situation, we don't have an existing token, so we login first. The third situation is when we aren't authenticated.

spec/vmc/client/app_spec.rb, spec/fixtures/app_info.txt

# app_spec.rb
describe VMC::Client::App do
  describe "#app_info" do
    it "obtains the given app's info when a valid auth_token is present" do
      stub_request(:get, "http://api.vcap.me/info").
          to_return(fixture('info_authenticated.txt'))
      stub_request(:get, "http://api.vcap.me/apps/my_app").
          with(:headers => {'AUTHORIZATION' => 'token'}).
          to_return(fixture('app_info.txt'))

      client = VMC::Client.new('token', 'my_app')

      client.app_info.resources.memory.should == 64
    end

    it "obtains the given app's info when logged in" do
      stub_login(:success)
      stub_request(:get, "http://api.vcap.me/apps/my_app").
          with(:headers => {'AUTHORIZATION' => 'valid_auth_token'}).
          to_return(fixture('app_info.txt'))

      client = VMC::Client.new(nil, 'my_app')
      client.login('foo@example.com', 'foo')

      client.app_info.resources.memory.should == 64
    end

    it "raises an AuthError if not logged in" do
      stub_request(:get, "http://api.vcap.me/apps/my_app")
      stub_request(:get, "http://api.vcap.me/info").
          to_return(fixture('info_returned.txt'))

      client = VMC::Client.new(nil, 'my_app')
      expect { client.app_info }.to raise_error VMC::Client::AuthError
    end
  end
__________________________________________________

# app_info.txt
HTTP/1.1 200 OK
Server: nginx/0.7.65
Date: Fri, 04 Mar 2011 02:56:21 GMT
Content-Type: application/json
Connection: keep-alive
Keep-Alive: timeout=20
Content-Length: 243

{"resources":{"memory":64},"uris":["foo.vcap.me"],"staging":{"stack":"ruby foo.rb","model":"http://b20nine.com/unknown"},"state":"STARTED","instances":1,"name":"foo","meta":{"version":1,"created":1299207348},"services":[],"runningInstances":1}

The App Module

For our Client, we will be able to instantiate a new Client object with an app name. We will then use this to send a GET request to the Cloud Controllers app path. Our #request method will handle the authorization. We indicate when this is necessary by specifying the :require_auth => true option.

When authorization is needed, we check the login status by ensuring that either we are currently logged in or that our auth_token is valid. We put the #auth_token_valid? and #logged_in? methods in our Authentication module.

There are many more API points that our Client gem needs to be able to interface, but for now this is a good start.

[lib/]vmc/client.rb, []vmc/client/app.rb, []vmc.rb, []client/request.rb & errors.rb & authentication.rb

# client.rb
require 'vmc/client/app'
...
module VMC
  class Client
    attr_accessor :app_name, ...

    def initialize(auth_token=nil, app_name=nil)
      @app_name = app_name
      ...
    end

    include App
    ...
__________________________________________________

# app.rb
module VMC
  class Client
    module App
      def app_info
        get("#{VMC::APPS_PATH}/#{app_name}", :require_auth => true)
      end
__________________________________________________

# vmc.rb
module VMC
  APPS_PATH            = '/apps'
__________________________________________________

# request.rb
module VMC
  class Client
    module Request
      ...
      def request(action, path, options)
        check_login_status if options[:require_auth]
        ...
      end

      def check_login_status
        raise AuthError unless logged_in? || auth_token_valid?
      end
__________________________________________________

# errors.rb
module VMC
  class Client
    class AuthError <  RuntimeError; end
__________________________________________________

# authentication.rb
module VMC
  class Client
    module Authentication
    ...
      def auth_token_valid?
        descr = info
        if descr
          return false unless descr[:user]
          return false unless descr[:usage]
          @user = descr[:user]
          true
        end
      end

      def logged_in?
        user
      end

Comments

I don't like the code in the fixture method. Bad practice to hard-code path separators. def fixture(file) File.new( File.join(fixture_path, file) ) end would be better

By Jeffrey Jones Jun 12, 2011 17:39
Good point there Jeff. Using File.join will avoid some potential bugs.

By aRailsDemo Jun 12, 2011 21:37

(Please login to submit comments.)

View All Posts

#56 Cloud Foundry - Creating An API Client

Establishing The Gem

Webmock

The Login Spec

Faraday

Refactoring

The Info Spec

The Info Module

Th App Spec

The App Module

Comments

Announcements

Comments

Contact