jstest

Asynchronous tests

jstest provides a simple mechanism for asynchronous testing based on continuation-passing style. Test suites are organised using contexts with each test delimited by an it block. For example:

JS.Test.describe('Ajax', function() { with(this) {
  it('adds numbers', function() { with(this) {
    assertEqual(2, 1 + 1)
  }})
}})

The test runner assumes that your tests are synchronous by default; that is when we reach the end of the it block that test has finished and we can move onto the next. But it might not have finished – we may have started an Ajax request or set a timeout that we’re waiting on before finishing the test.

To tell jstest that a test is asynchronous, we can add a parameter to the it block. The test runner will then fill this parameter with a callback function that you can use to resume the test. When you call the resume function you pass in a function containing the assertions you want to make.

For example, here’s a quick test using jQuery to make an Ajax call to a server:

JS.Test.describe('Ajax', function() { with(this) {
  it('fetches a page', function(resume) { with(this) {
    jQuery.get('/foo.html', function(response) {
      resume(function() {
        assertEqual('Hello, World', response)
      })
    })
  }})
}})

If you add the resume parameter to the it block, the test runner will not consider the test complete until the resume function is invoked. If you do not invoke the resume function the test runner will make the test time out after a few seconds and move onto other tests.

resume can be invoked in various ways:

• With no arguments, which simply resumes the test runner
• With a String or Object, in which case the argument is treated as an error
• With a function, which will be called and any errors it raises will be caught and reported

The second option means resume can be used like this to catch errors in Node:

before(function(resume) {
  fs.writeFile('/some/path', 'content', resume)
})

If the file write fails, then jstest will report the error passed to the resume function.

The third option allows async blocks to be nested; if the function you pass to resume itself takes a parameter, then jstest assumes this function is also asynchronous and passes in another resume function that works the same way as the first.

it('is asynchronous', function(resume) { with(this) {
  jQuery.get('/foo.html', function() {
    resume(function(resume) {
      anotherAsyncCall(function() {
        resume()
      })
    })
  })
}})

However, this typicaly doesn’t scale well in terms of readability. If you’re running an integration test with a lot of asynchronous parts, it’s more useful to abstract the steps in the test into functions that hide the async plumbing, as described below.

Asynchronous stories

Let’s take a somewhat contrived example: we want to build an API to manipulate blog posts over HTTP, and we’re going to call it using jQuery. Our test will involve the following steps:

• When we POST the blog post’s details to the API, the web service should create the blog post for us and returns its ID.
• When we GET a blog post by ID, the web service returns the blog post’s data.

We’re going to test all of this from the outside by making requests. A test for these steps might look like this:

JS.Test.describe('blog post API', function() { with(this) {
  it('creates a blog post on the server', function() { with(this) {
    create_blog_post({title: 'JavaScript testing'})
    assert_json_response()
    assert_response_has_field('id')
    get_blog_post_by_id()
    assert_response_has_field('title', 'JavaScript testing')
  }})
}})

This test has no nested sections and is easier to read than our previous example, and it’s more abstract: it tells a high-level story and the details are hidden inside the various functions.

jstest gives us a way to write asynchronous tests that look like the example above. The first step is to implement all the steps the test needs, but give each function an additional callback argument (which we’ll call resume) that it should invoke when its work is done. We create a set of testing steps using the asyncSteps() function:

var BlogPostSteps = JS.Test.asyncSteps({
  create_blog_post: function(attributes, resume) { with(this) {
    var testCase = this
    jQuery.post('/blog_posts', attributes, function(response) {
      testCase.response = response
      resume()
    })
  }},

  assert_json_response: function(resume) { with(this) {
    assertNothingThrown(function() { JSON.parse(response) })
    resume()
  }},

  assert_response_has_field: function(field, value, resume) { with(this) {
    var data = JSON.parse(response)
    assert(data.hasOwnProperty(field))
    assertEqual(value, data[field])
    resume()
  }},

  get_blog_post_by_id: function(resume) { with(this) {
    var testCase = this
    var id = JSON.parse(response).id
    jQuery.get('/blog_posts/' + id, function(response) {
      testCase.response = response
      resume()
    })
  }}
})

Note how all functions used as test steps must take a resume callback after all the arguments used explicitly in the test, and invoke it to indicate the work of that step is finished. This function works similarly to resume in normal test blocks, in that if you call it with an argument the argument is treated as an error. jstest uses these callbacks to glue your steps together when running a test.

If you need to retain state during a test, for example holding on to an HTTP response as shown above, you can store data on the current test case object (referred to by this) and it will be available in subsequent steps.

Once you’ve made all your test steps, you just need to add them to your test using the include() function:

JS.Test.describe('blog post API', function() { with(this) {
  // Make the step functions available in this test
  include(BlogPostSteps)

  it('creates a blog post on the server', function() { with(this) {
    create_blog_post({title: 'JavaScript testing'})
    assert_json_response()
    assert_response_has_field('id')
    get_blog_post_by_id()
    assert_response_has_field('title', 'JavaScript testing')
  }})
}})

Note that when you write a test like this, jstest does not make the steps blocking; the purpose of the code in the it block is to queue up a set of actions that define the test, then jstest deals with sequencing the actions for you. For this reason, you can’t inspect the state of the test from within the it block, you must put any debugging into the step functions themselves.

It is good practise to confine the steps themselves to at most one asynchronous action per step. The purpose of this pattern is to break a large async test up into composable parts that can be easily re-ordered. Note how the async examples above don’t have complex logic in their callbacks, they typically just store the result of an action for processing by other steps.

Placing complex logic or assertions in the async callbacks means that jstest cannot catch any exceptions they throw, which is another reason to keep them as simple as possible.