Javascript testing problems

Steven van der Vegt — Tue, 04 Mar 2008 17:02:00 +0100

A few months ago I got the assignment to set up and build javascripts-tests for the scheduling view of moves. At first I had to select a testing-framework where the test would be written in. The framework had to be able to run both unit and integrations tests. Also it would be very nice if we can run these test automatically through the command-line instead of clicking around in the browser (autotest).

After reviewing some frameworks, I chose the Crosscheck framework (http://www.thefrontside.net/crosscheck). Crosscheck is a javascript-testing-framework written in java. It is crossplatform and you can control it from the command-line. It is even able to emulate the behavior of multiple popular browsers like ie6, Firefox 1.0 and Firefox 1.5. Sounds like the ultimate testing framework!

So after my decision I started playing around and implementing tests. However, when the tests became more complex I ran into trouble. Some basic browser features like document.write() (ie6) and the Option object where missing. I was able to work around these problems, but the real trouble began with the integration tests. As our application relies heavily on ajax through the prototype framework, testing this functionality is crucial. However, I was not able to do this. Performing one hack after another, I finally gave up.

The crosscheck framework was clearly not mature enough to satisfy my needs. My conclusion is: Using crosscheck for unit-testing is doable, but the framework is not mature enough for the use of integration tests. So what are the problems with crosscheck? As mentioned earlier, it is written in Java, so it tries to emulate the browser behavior based on it's specifications. The advantages of this approach is that you can emulate more than one browser, the disadvantages are that the emulation is an approach, so you will never really get the real behavior of the browser.

If a browser changes its implementation, the framework is always outdated. You don't get the browsers quirks, so if it your tests pass in the test framework, there is no guarantee it will work in the real-browser world. Another problem with crosscheck is that is seems to be abandoned. The last changes are from end-2007 and as far as I can see none of the reported bugs have been fixed yet. Firefox 3.x and ie7 are becoming standard, but these browsers are not available as a test-browser in crosscheck (and there are no clues they will be soon).

What can we expect from the continuity of crosscheck? If you, like us, want to be able to test a long-term project, then it is important that your test-framework is long-term too. So what does my ideal javascript-test-framework look like?

It should:

Run from the command line
Represent the browser realistically
Emulate ajax responses
Mock objects
Run implementation tests

As far as I know, no such framework exists. However, in my view, it shouldn't be too hard to realize. We could embed the mozilla tree in an application. That way we always have the latest version of the mozilla engine and an exact copy of the behavior including it's quirks. Through the Mozilla API we can access the DOM and other functions. The framework should be able to easily load tests and run them (for this part it's important to take a good look at the x-unit patterns).

At this moment I don't have a clue how easy it is to emulate the ajax responses through mozilla API calls. Furthermore the framework should be give a rich toolkit for testing including abilities to mock objects. You start the framework via the command-line for easy integration with test-runners like autotest. Of course this application should be released under the GPL-licence. Now we only need someone to implement this for us!

Steven van der Vegt (s.vandervegt TA student.utwente.nl)

Ps we are offering an internship for the development of the described plugin at Nedap healthcare. Interrested? bart.tenbrinke@movesonrails.com International students welcome!

Autotest 100% CPU solution

bart.tenbrinke@movesonrails.com (Bart ten Brinke) — Wed, 06 Feb 2008 09:42:00 +0100

Autotest is great, but when it is waiting for test, my MacBook turns into a whirlwind as autotest takes 100% CPU. After looking at autotest.rb, we easily find the waiting function:

def wait_for_changes
  hook :waiting
  Kernel.sleep self.sleep until find_files_to_test
end

As self.sleep is defaulted to 1 it means that my laptop does not sleep at all. Changing this is quite easy, as you can just add this to your .autotest file in your home directory:

Autotest.add_hook :initialize do |at|
  at.sleep = 5
end

Experiment with the amount to determine what works for you. Offcourse autotest will now react slower to you changes, but hey: you can't have everything. Enjoy the silence!

Moves On Rails: Tag autotest

Javascript testing problems

Autotest 100% CPU solution