Web Site Usability Testing on a Budget

One of the biggest myths about usability testing is that it costs thousands of dollars a day to run the most basic of tests. Sure, many high-profile engineers will charge you over $10,000 a day, but if you want to conduct a test yourself, you could easily do it for around $250.

All you need is five users, an average home computer, camcorder with tripod, stopwatch, $250 and a few hours of your time.

Number of Users:Five
Type of Users:Any Internet User (Targeted users preferred)
Test Location:Your/Customer's Office
Test Supervisor:You
Observers:None Present
Test Schedule:Anytime
Test Preparation:List of Tasks and Questions
Costs:$250 ($50 Per User Per Hour)

Number of Users

According to Jacob Nielson - a usability expert - testing five users could uncover about 85% of your site's usability problems.

In your effort to minimize usability testing costs, you should try not to use co-workers, or anyone else who has a hand in creating the site, as test subjects. Even using other Web designers could be a big mistake. Their prior experience with the site, the technology and a bias towards pleasing you could easily spoil the test results.

However, it is acceptable - and economical - to use co-workers to run through, but not complete, the usability tests to make sure there are no problems with your test plan.

Types of Users

To a certain extent, a basic familiarity with a Web browser is the only experience that your test subjects need. Ideally they should have been using the Web for at least a few months, and understand all of the basic concepts.

Of course, the ideal scenario would be to recruit users that fit the demographic you are targeting. But that could be much more difficult and increase the costs.

However, there are exceptions, if you are targeting:

  • Clearly defined demographic groups: If your site is targeted at teenagers, by all means conduct testing sessions using only teenagers;
  • Clearly defined interest groups: If your site targets fitness enthusiasts, conduct at least one round of testing with fitness enthusiasts; and
  • Groups that require certain knowledge: If your site is targeted at certain professionals, such as doctors, you should conduct at least one round of testing with users in the profession.

Test Location

You can test anywhere that has a computer - in your office or even at your home.

You should also consider conducting usability tests in your customers' offices. This way, you can observe your users in the context of their work environment.

If you force your users to work in a sterile 'examination' environment, they might feel uncomfortable, and make mistakes which they might not necessarily make in a normal situation.

UPS.com's site design team often pay live visits to the offices of all types of actual customers to observe how they are using UPS.com in the real world.

You could do the same. Most users won't mind you observing their usage of their site, as long as you agree to meet all costs, and don't disrupt them more than necessary.

Test Supervisor

You only require one test supervisor to conduct a usability testing session. Virtually anyone that has patience, is observant, and is a good listener can supervise a usability test. The test supervisor doesn't have to be a usability expert, but should simply be familiar with the aim of the session.

Observers

The best policy with observers is to keep them away from the testing session. Most people get nervous when they sit a test, or know they are being watched by a large group of people.

Instead, use a camcorder to record the testing session for later review.

This has a number of advantages:

  • You'll have the option to review a particular testing session over and over again; and
  • All members of the design team can dicsuss a testing session as they are reviewing it.

Test Schedule

You should conduct as many usability tests as it takes to create a user-friendly Web site.

At the bare minimum, there should be at least three rounds of testing:

  1. Uncover your users' needs: To design a site that meets your intended users' needs, you should start by conducting some basic usability tests on competitors' sites. Find out what your users particularly like or dislike about a competitor's site;
  2. Design your site's objectives to meet your users' needs: Translate your users' needs into specific priorities for your site design. Once the design requirements have been formalized, you should test the site plan to make sure the translation of your users' needs have been met properly; and
  3. Beta test your site: A beta test - also considered as a "pre-release test" - is a phase of testing in which a sampling of the intended audience tries the product out prior to launch. This round of testing allows you to validate the most critical elements of your site before it goes live. Once your site goes live, you should start over with the first test.

Like I said, you should conduct a minimum of three rounds of usability testing. Ideally, you will conduct more tests throughout the development of your Web site.

Test Preparation

Before you conduct a round of testing, make sure you have the following items to hand:

  • A stopwatch;
  • A camcorder to record the testing session;
  • A list of prepared questions to ask the test subject; and
  • And a list of prepared tasks for the user to complete.

Costs

Most of the costs involved in basic usability testing are to pay the Web users a small fee for their time. Set a reasonable fee that would encourage the Web user to attend the testing session.

Expect to pay US$50-100 per hour for the average Web user. For professionals who have certain knowledge - such as doctors or bankers - the fee could easily be US$200-300 per hour unless you have good personal contacts.

If your test subjects are existing customers, you could reward them with complimentary products and services, instead of cash.