Showing posts from February, 2005

Performance vs. load vs. stress testing

Here's a good interview question for a tester: how do you define performance/load/stress testing? Many times people use these terms interchangeably, but they have in fact quite different meanings. This post is a quick review of these concepts, based on my own experience, but also using definitions from testing literature -- in particular: "Testing computer software" by Kaner et al, "Software testing techniques" by Loveland et al, and "Testing applications on the Web" by Nguyen et al.

Update July 7th, 2005

From the referrer logs I see that this post comes up fairly often in Google searches. I'm updating it with a link to a later post I wrote called 'More on performance vs. load testing'.

Performance testing

The goal of performance testing is not to find bugs, but to eliminate bottlenecks and establish a baseline for future regression testing. To conduct performance testing is to engage in a carefully controlled process of measurement and analys…

Web app testing with Python part 1: MaxQ

I intend to write a series of posts on various Web app testing tools that use Python/Jython. I'll start by covering MaxQ, then I'll talk about mechanize, Pamie, Selenium and possibly other tools.

First of all, I'll borrow Bret Pettichord's terminology by saying that there are two main classes of Web app testing tools:
Tools that simulate browsers (Bret calls them "Web protocol drivers") by implementing the HTTP request/response protocol and by parsing the resulting HTML
Tools that automate browsers ("Web browser drivers") by driving them for example via COM calls in the case of Internet Explorer Examples of browser simulators:
mechanize (Python), which is based on WWW::mechanize (Perl)webunit (Python)MaxQ (written in Java, generates Jython scripts)
HttpUnit, jWebUnit, HtmlUnit, HtmlFixture (Java)WebUnit (Ruby) Examples of browser drivers:
Pamie (Python), which is based on Samie (Perl): IE automation via COM
Watir (Ruby): IE automation via COM
JSSh (Mozill…

Articles and tutorials

Agile Documentation with doctest and epydoc

This post was inspired by an article I read in the Feb. 2005 issue of Better Software: "Double Duty" by Brian Button. The title refers to having unit tests serve the double role of testing and documentation. Brian calls this Agile Documentation. For Python developers, this is old news, since the doctest module already provides what is called "literate testing" or "executable documentation". However, Brian also introduces some concepts that I think are worth exploring: Test Lists and Tests Maps.

Test Lists

A Test List tells a story about the behavior expected from the module/class under test. It is composed of one-liners, each line describing what a specific unit test tries to achieve. For example, in the case of a Blog management application, you could have the following (incomplete) Test List:Deleting all entries results in no entries in the blog.Posting single entry results in single valid entry.Deleting a single entry by index results in n…

L.A. Piggies? Trying to organize Python Interest Group in L.A./O.C.

If anybody is interested in putting together a Python Interest Group in the Los Angeles/Orange County area, please let me know. Add a comment to this post or send me email at grig at

Python as an agile language

Here are some ideas on why I think Python is an agile language. I use the term agile as in "agile software development practices", best exemplified by Extreme Programming. I find this definition by Ron Jeffries, from his article "What is Extreme Programming", particularly illuminating:

Extreme Programming is a discipline of software development based on values of simplicity, communication, feedback, and courage. It works by bringing the whole team together in the presence of simple practices, with enough feedback to enable the team to see where they are and to tune the practices to their unique situation.

Let's see how Python fares in light of the 4 core XP values: simplicity, communication, feedback and courage.

1. Python fosters simplicity
Clean and simple syntax reads like pseudo-codeBuilt-in high level data types (strings, lists, tuples, dictionaries) make it possible to pack a lot of functionality in very few lines of code, without sacrificing readabilityAs…

New Google group: extreme-python

Troy Frever from Aviarc Corporation posted a message to the fitnesse mailing list announcing that he created a Google group for topics related to both Python and Agile methodologies (Extreme Programming and others).

Appropriately enough, the name of the group is extreme-python. Visit , take a look, post a message.

Web app testing with Jython and HttpUnit

There's been a lot of talk recently about "dynamic Java", by which people generally mean driving the JVM by means of a scripting language (see Tim Bray's post and Sean McGrath's post on this topic). One of the languages leading the pack in this area is Jython (the other one is Groovy). In fact, a Java Republic poll asking "What is your scripting language for Java for 2004?" has Jython as the winner with 59% of the votes.

Update: As a coincidence, while writing this post, I came across this blog entry: Gosling on JVM scripting

Jython is also steadily making inroads into the world of test frameworks. It is perhaps no coincidence that in a talk given at Stanford, Guido van Rossum lists "Testing (popular area for Jython)" on the slide that talks about Python Sample Use Areas. Because Jython combines the agility of Python with easy access to the Java libraries, it is the scripting language of choice for test tools such as The Grinder v3, TestMaker,