Tournology recently hosted our first private tournament to alpha test our software among laypersons. This test would provide us with not only motivation to complete the coding required to run a simple multi-stage tournament, but also would provide us with valuable usability testing.

Of course, the crucial piece of usability testing is reviewable evidence. And that means spying on our tournament participants! We set up two kiosks so that teams could easily access our application, and then used a screen capture program to record their every mouse movement, keyboard key press, and each vocal expletive used when they couldn’t find what they were looking for.

Of course, there are a ton of different screen capture programs out there, and your mileage will vary when using each one. Some are intended for simple demonstration videos, where a presenter might go through a variety of steps one after another in quick succession to display how to accomplish a task. These are generally intended to be shorter videos, and software developed primarily for this use case may not handle the longer recording sessions that a usability test will require.

Here, we’ll go through a few of the programs we tried, and give you a little bit of information on the strengths and weaknesses as they relate to usability testing.

Testing Criteria

1. Must record high resolution videos. We wanted to run our kiosks at the highest resolution the monitor would allow (1920×1200), and then use the page zoom feature of the browser to increase the fonts on the page to a readable size. Ultimately, this would look much better than running a lower resolution on our 24″ kiosk monitors. That means the capture program would need to either record in full 1920×1200, or be smart enough to record the portion of the viewable window needed for the displayed browser content.
2. Must output in a common video format. We ultimately wanted something that would play in a normal video player, like VLC. We certainly did not want the program to use its own proprietary format. Also, the more a codec could compress the output, the better. We don’t need a lossless codec for this task.
3. Must not affect system performance. Sure, any old program can record 1920×1200 resolution, but will the mouse jump in 20px increments?
4. Simultaneous audio recording is a plus, video is even better.
5. Visual indicator of Meta Key Presses and Mouse Clicks. Later on, how will I know that the user hit the CTRL key, or the ALT key, or the Apple key on the keyboard?

Another key factor we faced is that three of the four developers on the project had Apple MacBooks, so it was pretty much guaranteed that we’d be using the MacOS platform for our kiosks. This turned out to be a beneficial requirement, as it would appear that there are quite a few more reputable capture programs for MacOS than Windows, not to mention that the MacBook has a built in web cam (for any possible video recording). If you want to use a MacBook for a usability test with an external monitor, please check out these tips from Christian Cantrell.

Silverback

The coolest feature about Silverback is that it will overlay the video from your web cam right into the screen capture video. While the correlation between mouse movement and eye movement has been extensively studied, we find it infinitely more useful to look at the user’s emotional response. Are they getting frustrated? Angry? Were they delighted at the ease of use of a particular feature?

Unfortunately, in our tests, Silverback is not yet stable when trying to export videos longer than 30 minutes. It is a problem they are actively working on, and I hope they successfully solve it soon, because this is a very valuable feature.

CamStudio

One of the best features in CamStudio is that they let you choose which codec you want to encode the resulting video in. So, naturally, I tried the XVID codec and it worked just fine. CamStudio also has an option to export to SWF (Flash).

IShowU

In the IShowU feature set, one of the interesting pieces is that IShowU HD and HD Pro work with full screen OpenGL applications. This is an important note to consider if you’re testing the usability of something 3-dimensionally accelerated.

Big Hitters We Didn’t Review

Final Decision

Ultimately, we decided to go with ScreenFlick. It handled full 1920×1200 without a noticeable drop in system performance, and resulted in very manageable file sizes. The program displays a nice video size counter in the taskbar, which only increments when something on the screen changes. Very efficient recording. Also, for a demo, the watermark it added to the resulting video wasn’t too obtrusive to use for our purposes, especially compared to the very intrusive IShowU demo watermark. It would have been really nice to get web cam video integrated with our usability test, and it looks like a purchased version of IShowU would be a very viable alternative which we’ll have to consider when we have our next Tournology tournament.

Tags: Screen Capture, Usability

This entry was posted on Saturday, December 13th, 2008 at 12:52 pm and is filed under Usability. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.