User Testing Made Easy with 3 Important Tips

I am not being paid or compensated in any way from any company or entity for my remarks in this article.

At my day job, usability was a main concern of ours, even before our first line of code was written.  If your software product does not have a focus on usability, then it will undoubtedly fail.  But what is usability? 

Read a Book (You know you want to…)

I thought that I knew what usability was.  I mean, it’s simple, right?  You just make sure people can use your product.  Wrong – kind of…  Even though you can simplify it like that, there is so much more to consider when usability is being addressed.  How do I know that?  Well, one of my favorite books these days is not a technical book, per se, but rather a non-technical book that focuses on design in technology.  It is called Don’t Make Me Think, written by Steve Krug.

If you are like I was, and you haven’t given any dedicated time to learning usability from a seasoned usability veteran, then stop all design and development now.  Read Krug’s book, and then apply what you have learned.  Seriously.  Don’t worry.  If you listened to me, you will only be dedicating about 2 hours of your day to reading.  If you compare the cost of man-hours to read the book, to the invaluable knowledge you will have after having read it, there will be no doubt that the book was well worth your time.

The book takes you through many seemingly simple design issues, and talks to you about the right and wrong things to do when doing things such as creating buttons, links, layouts, and work flows.  Krug does a fantastic job of writing in a style that mirrors the concepts he speaks about in his book.  The book is very usable: easy to read, quick to get through, and relays all of the information you need without any work on your part.  This book is for anyone concerned about design, and not just programmers.

Use a 3rd Party User Testing Service Provider

In Krug’s book, he mentions user testing quite a bit, and even lays out a plan for you to execute your own user testing session.  However, through the amazing breakthroughs in technology, a company has combined a few common technologies to make user testing much more cost-effective for any company.  How so? 

A typical user-testing plan will involve one or more (or all/more) of the following steps:

1. Have company meetings to discuss user testing goals
   1. What areas do we need tested?
   2. What kind of tasks need to be defined?
   3. Who is our target audience to test?
   4. What department(s) and project sponsor(s) need to be assigned/involved?
   5. What projects need to be shelved to accomplish this?
   6. Do we know what we are doing?
2. Have company meetings to discuss user testing execution plan
   1. How many people will be tested?
   2. Where will those people come from?
   3. Where will the user testing take place?
   4. How will we compensate or otherwise give incentives to the users?
   5. What physical resources will be needed to execute the user testing?
   6. What human resources will be needed to execute the user testing?
   7. How will the user testing be reviewed?
   8. How and where will the results be distributed and stored?
3. Define a budget for the user testing project
4. Execute the logistics to get the user testing site set-up
5. Begin user testing
6. Handle any unknown anomalies that could not be planned for
7. Close the user testing and clean up the user testing site
8. Compile and review the user testing results and data
9. Distribute the user testing results
10. Make appropriate changes as needed

The preceding list is not for the faint of heart.  Each bullet will obviously require substantial planning and man-hours to accomplish.  Also, I purposely left some things out to not turn you away.  (If you’ve made it this far.)  Needless to say, this is a lot for any size company to take on.  If you are doing this yourself, hopefully you have had some formal experience with this before.  Otherwise, I’d say that your user testing results will be skewed.

The first and most substantial problem I see with doing this yourself, is that you are unlikely to find user testing participant who will test your product(s) without being compensated in some way.  That compensation will normally result in the tester feeling obligated to be less than truthful about their experience and feelings.  I don’t know about you, but I would much prefer to have brutal honesty about my software product.  If something really sucks, tell me about it.  I would rather have the problem corrected right then and there, versus finding out later that the problem was much bigger than originally communicated.

Second, this is a lot of work to do.  In order to get the user testing done right, you would have to dedicate a substantial amount of time and staff to make your user testing sessions successful, and worth the time and resources that you’re paying for.  In my opinion, both of these reasons are reason enough to not move on.  It’s time to revise the user testing strategy.

This is where technology has stepped in to change our lives forever, in the user testing space.  It goes without saying that my previous concerns warrant you hiring a 3rd party to conduct your user testing project.  However, conventional user testing methods are still quite expensive, and when not conducted correctly, their results will be mixed.

There is an online service out there at UserTesting.com, where you can sign up, and pay for exactly what you want.  They have an ala carte selection work flow.  The work flow allows you to choose your demographic, number of testers, tester technical ability, user computer set-up, define tasks, and more. 

UserTesting.com uses a screen capture tool on the tester’s machine, along with a microphone, allowing you to both see what they’re doing, as well as hearing what they think.  The tester is encouraged to speak about what they are doing and thinking throughout the duration of the test.  Tests generally last about 10 minutes, depending on the tasks you have defined for the tester.  Once the tester is done, the finished video is hosted on their site, where you can download and/or view it yourself.

This is great!  I can choose the age, gender, technical ability, income level, and more for each tester.  In traditional testing, you get what you get, and sometimes that’s not very clear.  What’s more, you get a person who is not interested in saying only nice things, since they are not getting compensated by you.  They have no attachment to you since you’re not in the room with them, holding their gift certificate or something.

Now let’s talk bullets and band-aids…  For just $29.00 USD per tester, you take care of most of the list above.  If you test 10 people, that’s a total of $290.00 USD out of pocket.  If you do this yourself, that amount is going to increase exponentially, perhaps $2,900.00 USD or more – not including the cost of staff involvement.  You only had to involve management, and your staff still kept moving on their current projects.  You didn’t have to hire any consultants, rent or buy physical resources, or anything else to make this happen. 

This is definitely one of those “it’s too good to be true” situations, but as a customer, I can tell you that I would not do user testing any other way again.  We learned so much about our customers, our product, and about things we never would have thought about before.

Most Important User Testing Tip! Define Tasks, NOT Directions

Here’s a quick tip for you when planning your user testing tasks… When you’re planning your user testing tasks, DO NOT give them a step-by-step description of how to accomplish the task.  Instead, give them a one-sentence description of the task, and let them figure it out.  Believe it or not, you will get more information from that, every time.

WRONG!

1. Click on the About Us tab
2. Click on the Contact Us form
3. Click on the Submit by Form link
4. Tell us what you think

RIGHT!

1. Send us feedback using the form on our website

This scenario will show you if you got your overall design right.  You will learn if the placement of your links, pages, and other objects are in intuitive areas of the page, and easy to use or find.  Your tester will tell you if they did or did not like any of the preceding concerns or design elements.  They will show and tell you where they expected certain design elements to be and look like.

I hope this blog post got you excited about user testing.  I have become so passionate about user testing, but usability overall.  A company the looks at usability, is a company that cares about customer service.  And that puts you to the front of the line in my short list of companies that I will spend money with and/or recommend.

This article is cross-posted from my personal blog.