Moving away from Selenium

Recently I’ve spent a lot of time developing MVC.NET based web apps. The automated functional tests for these web apps have all been written using Selenium, with the tests written in C#  and organized as NUnit test fixtures.

Whilst I found Selenium to be a good tool, I ran into some serious problems with it:

  1. Some of my Selenium based functional tests would fail approximately 10% of the time when they were run against Firefox.   Switching the exact same tests to using Internet Explorer (still with Selenium) resulted in the tests returning to 100% stability.  This meant abandoning automated functional testing with Firefox in favour of using IE.
  2. Slow performance in Internet Explorer (mitigated to some extent by switching the XPath engine Selenium uses with it).  IE just isn’t a fast browser when used with Selenium.
  3. Many of Seleniums operations are non-blocking (i.e. asynchronous).  This means Selenium doesn’t wait for those operations to complete before allowing a test to continue.  For example opening a new webpage in Selenium is a non-blocking operation; Selenium will happily load a page and then (via the next line in the functional test) attempt to click a button on the page when the button hasn’t even loaded yet.

Looking round for an alternative to Selenium, there was one obvious replacement: WebDriver.  In many ways WebDriver is quite similar to Selenium.  However the two differ quite markedly in one regard: Selenium performs its automation using its own JavaScript running within the browser it’s automating whereas WebDriver controls the browser from the outside, using an COM automation API for Internet Explorer and custom browser extensions for Firefox and Google Chrome. The WebDriver approach has proven much more flexible and faster performing than the Selenium approach.  Simon Stewart has written an interesting post on the Google Blog that delves further into this topic.  Another important point in WebDriver’s favour is that Selenium and WebDriver have started to merge into a single project / code base.

Having gotten very interested in WebDriver, I was really surprised to find it doesn’t (yet) support .NET.  At the moment WebDriver can be used from tests written in Java (and Ruby/Python) but not .NET.

Given that I was very keen to use WebDriver with C#, I started to think about any “out of the box” ways I could get this working.  I was thinking some sort of interop bridging or remoting between Java and .NET.

First I looked at the new jni4net project.  jni4net allows you to fireup an instance of the Java Virtual Machine (JVM) from within your .NET application; you can then interact with the java classes running in the JVM using pre-generated .NET proxy classes.  jni4net looked promising but ultimately wasn’t a workable solution; the current version of jni4net isn’t able to generate proxy classes for Java nested static classes – a Java language feature that the WebDriver API relies on.

Then I came across IKVM.NET – a project I’ve read about before but never used.  I’m happy to say I was able to get WebDriver to work with C# using IKVM.NET!! The solution even works with JetBrains ReSharper test runner for Visual Studio. Read my step-by-step guide to getting WebDriver working with C# for more info.

6 comments

  1. Hi Simon,

    jni4net is still alpha quality yet, but you know ‘release early, get feedback’. Any other problem you run into with jni4net ? More feedback ?

    Anyway, if webdriver works with IKVM, it’s probably better solution for now. It will take me some time to match qualities of IKVM 😉

    Cheers
    Pavel

Leave a Reply

Your email address will not be published. Required fields are marked *