October 4, 2007 at 11:48 am (HtmlUnit, HttpUnit, Java)
There’s
a lot of misinformation out there regarding web application test tools,
so I’ve decided to post a series of short articles comparing some of
the open source options available here in Java-land, circa 2007. The
first of these articles will focus on HtmlUnit and HttpUnit.
Please take my criticism and praise with a grain of salt, as I’m a
committer to the HtmlUnit project and thus probably biased.
Nevertheless, I will do my best to be objective. I may even
overcompensate in the other direction!
Confusion
The HtmlUnit and HttpUnit projects are often confused due to the
similarity of their names. And the similarity doesn’t end there: they
are both open source projects; they are both 100% Java frameworks,
rather than drivers for native browsers like IE or Firefox; and they
are both fairly mature projects.
This confusion is compounded by the fact that many test frameworks
which once used HttpUnit under the covers have since switched to using
HtmlUnit, mainly in order to benefit from its excellent JavaScript
support. Examples include JWebUnit, whose FAQ briefly explains the switch, and Canoo WebTest, which switched in 2004 due to JavaScript support issues and an unresponsive development team [1].
HttpUnit
HttpUnit is the granddaddy web app testing framework. Started in the
summer of 2000 by Russ Gold [2], it was the first project to focus on
this niche area. The project has since stagnated somewhat, with nearly 40% of bugs remaining open, some of them nearly three years old. Its latest maintenance release is about a year and a half old.
The API is fairly low-level, modeling web interactions at something
approaching the HTTP request and response level. The following is a
slightly modified example from the HttpUnit Cookbook:
WebConversation wc = new WebConversation();
WebResponse resp = wc.getResponse("http://www.google.com/");
WebLink link = resp.getLinkWith("About Google");
link.click();
WebResponse resp2 = wc.getCurrentPage();
As you can see, things center around WebConversations, WebRequests and WebResponses.
Unfortunately, any page with a decent amount of JavaScript is likely to
break HttpUnit, and you can absolutely forget testing any pages which
use third party JavaScript libraries.
Nevertheless, HttpUnit continues to generate 3,000 to 4,000 downloads
per month. A good analogy, if I may be allowed a brief subjective
comment, is that HttpUnit is to the web app testing world what Struts
is to the web app framework world: there are many “better” options out
there, but it just won’t go away! 
HtmlUnit
HtmlUnit is itself a fairly old project, having been started by Mike Bowler
in early 2002. Mike has since ceased active development, but the
project currently boasts 3 or 4 active developers and a total of seven committers (whereas HttpUnit remains a one-man show). It averages about three releases per year, and has seen increased developer activity in the past six months or so, especially in the area of JavaScript support.
HtmlUnit’s API is a bit more high-level than HttpUnit’s, modeling
web interaction in terms of the documents and interface elements which
the user interacts with:
WebClient wc = new WebClient();
HtmlPage page = (HtmlPage) wc.getPage("http://www.google.com");
HtmlForm form = page.getFormByName("f");
HtmlSubmitInput button = (HtmlSubmitInput) form.getInputByName("btnG");
HtmlPage page2 = (HtmlPage) button.click();
As you can see, the code centers around WebClients, as well
as pages, links, forms, buttons, etc. Pages with a modicum of custom
JavaScript will probably work when tested with HtmlUnit. Unfortunately,
pages which use third party libraries might or might not work when
tested via HtmlUnit. As of the current version, Prototype, Script.aculo.us, DWR and jQuery are known to be supported fairly well, Dojo is a bit of an unknown, YUI is known to be unsupported, and GWT
is known to work with fairly simple applications. Most of this
compatibility has been achieved in the past two or three releases, so
obviously things are fairly fluid.
Conclusion
If you’re using HttpUnit for legacy reasons, it’s a fairly solid
package, but don’t expect to get much support when you need to report a
bug or submit a patch for a new feature. If you’re starting a new
project and are trying to decide between these two frameworks, HtmlUnit
wins hands down. It has the features, the community and the momentum.
Of course, if you’re considering web application testing tools, you’re probably looking at more than just these two options. Canoo WebTest, TestMaker, JWebUnit, Selenium, WebDriver and JMeter are all likely to be on your list. Depending on your project budget and requirements, Squish and Mercury QTP
may also be under consideration. If that’s the case, stay tuned,
because I intend to post a series of web app testing framework
comparisons in the coming months — all of them involving HtmlUnit, of
course!
[1] It’s interesting to note that both Marc Guillemot and I (two
HtmlUnit committers) began by using HttpUnit, submitting patches for
missing features — but settled on HtmlUnit when the patches were not
applied in a timely manner.
[2] The HttpUnit website states that
Russ currently works for Oracle,
developing the OC4J application server. Coincidentally, this is the
production application server we’re using at my day job. Thanks, Russ!
Powered by ScribeFire.