Testing Silverback for user testing

You might have heard about the relatively new user testing application, Silverback from ClearLeft. It came out last summer. If you haven’t and you are a mac user, then hurry up and go try out the free trial version. That’s exactly what I did just before Christmas as we were conducting user tests on vmax.dk – a website for Danish magazine with the same name that I’ve recently developed with my co-workers at Benjamin Media.

Categorized in: testing, user testing

I am more excited about Silverback than a 6 year old tasting candy for the first time, why I think you shouldn’t be left out. Here are the pros and cons of the product – hopefully these will encourage you to try out the product and conduct even more user tests than you already do.

First of all: Silverback is the first serious contender for a screen capture application fit for user testing for the mac. As I first heard about the application, it puzzled me why no one had thought about combining the built-in iSight camera of almost all mac computers with a screen capture application. The two are the perfect match!

At $49.95, Silverback is far cheaper than it’s competitors in the PC world. Camtasia is $299.00 for a single user license!

The cheaper product however comes with a price: You only get the absolutely most essential features needed for recording your sessions with Silverback. Buying Camtasia will give you a full featured product. But as I have never used more features in Camtasia than what Silverback offers, the cheaper product to me does not translate to any downsides. If anything, I only perceive Silverback’s simple interface as extra value.

Setting up the session

I have never conducted a user test with so little hassle setting everything up. I had everything I needed in my MacBook: a built in microphone and camera. No cables or external equipment. Instead of fiddling with external microphones and web cameras, I could spend my time concentrating on setting up an extra user in OS X not having permission to access my personal files. For companies without a dedicated testing lab, this solution is perfect!

What am I not getting?

Silverback does not allow you to surveil a test from another computer. What you however can do is to hook up an external monitor and mirror the displays (Silverback only allows recording on the computer’s primary display anyway). With a long cable, this will allow people in another room to see what’s going on. You can even use your apple remote to start and stop the session.

Downsides

Once the session is over, the data is saved in temporary files with the screen recording, mouse actions, and iSight video in separate files. These are joined into one Quicktime movie file when you click “Export”.

The usability tests we conduct at Benjamin Media rarely lasts under 45 minutes, and the vmax.dk test was no exception. Before exporting, the raw files stole around 6 GB of space on my hard drive. On a computer with 90 GB of total space that does not leave room for many sessions.

Furthermore, an export of the 50 minute recording took at least 4 hours (I’m not sure exactly how long it took, as I stepped away from the computer to let it finish). So start the export when you’re leaving work and you’ll find it ready the next morning.

As I wanted to show the best clips in a presentation for my team, I wanted to use Final Cut Pro to cut the best clips out of the 50 minute long video file. However, this was a tedious task, as the video isn’t encoded DV (understandably) and thus took forever to render and edit. Using iMovie was even worse, as I couldn’t figure out how to set the dimensions to the 1440×900 pixel size of my MacBook Pro screen. After messing around with it for a long while, I decided just to write down the timestamps for the recorded interactions I wanted to show and manually start and stop Quicktime when presenting. I guess that’s what you get with Camtasia.

So is it worth it?

Oh yes it is. It would also be worth its price at the $299.00 that Camtasia costs. Even though editing the video recorded is a hassle, the simplistic nature of the application and its great utilization of the built-in microphone and camera more than makes up for it.

Silverback is a must have for any mac using usability professional conducting user tests.

5 Comments

Mike Rayo

8 Jan, 2009

I totally agree with the ease of use and simple interface on Silverback.

Also, I never had the issues that you had with iMovie. I was using iMovie ’08, and the movies imported right in (after I exported them from Silverback). I built a very nice user reel in about 45 minutes and showed it to the entire team (even the developers from India!).

Great product.

Anders Toxboe

8 Jan, 2009

@Mike Rayo: Hmm… I’ll give it another go with iMovie then. Thanks for the feedback!

galavanr

22 Jan, 2009

I fell in love with Silverback at first sight. Initially, i felt there was no better alternative to the likes of Morae and Camtasia.

However, that was before I conducted my first large scale usability study using Silverback (20+ tests, each ranging between 15 & 30 minutes long).

Exporting is, well, a nightmare. You export in real time, and one by one. So the exporting takes the exact same amount of time as the testing did. That is a lot of time. At least with Morae you can que up 10 recordings, hit export, and go home to bed for the night. Not with Silverback. You are required to sit by your machine, wait until video 1 has finished exporting, then start exporting video 2.

In addition to this, exporting a video takes up a lot of memory, so the functionality of your machine during an export is compromised (quite significantly at times).

My final woe with Silverback is that it often crashes during export, leaving me literally pulling my hair out with frustration.

In saying all of this, Silverback has the potential to wipe the floor clean with the fancier more expensive applications. Because lets face it, although they certainly have their place in some usability studies, the fancy bells and whistles that Morae and the like provide are seldom used (e.g., time to task, success rates, task type markers). The real value comes from qualitative analysis of the test.

So my christmass list to ClearLeft is as follows:

1) Multiple export capabilities
2) Smaller file sizes
3) No crashing

With these in place – Silverback will kick ass!

My 2c

Anders Toxboe

22 Jan, 2009

@galavanr:

I have to agree with you on that. I rarely conduct more than 3-4 tests in a row, but when doing so, I have to set aside at least a weeks time before I have all the exported videos ready.

Sometimes, the export fails, where after you try the exact same thing once more, and everything runs smoothly. So you have to count in 1-2 nights of export per 1-hour session.