Why session replay upstages traditional methods of usability testing and user experience research.
In Shakespeare’s The Merchant of Venice, the antagonist, Shylock, forces the protagonist, Antonio, to promise a pound of his own flesh as collateral if his friend defaults on a loan.
While most types of collateral are less gruesome, their effect is similar. When you have skin in the game, failure is going to cost you. Like Antonio, you have a vested interest in success.
Of course, it's not the 16th century, and usability testing isn't Shakespeare. Your test subjects aren't asked to pony up such high stakes, or, really, any stakes at all.
Therein lies a problem.
But, wait. What is usability testing, anyway?
Usability testing helps teams understand how a product or service might be used through having a representative set of users interact with the product or service to complete some specific task.
As users complete tasks, user experience researchers observe and assess how easy it is for tasks to be completed. Usability testing is important because it helps identify areas of weakness, confusion, or other friction that would drive users to leave your site. As Jakob Nielsen put it:
On the Web, usability is a necessary condition for survival. If a website is difficult to use, people leave.
Website owners and app developers turn to usability testing to make sure users stay.
Why traditional usability testing just doesn't cut it
Every product manager would agree that they would love to conduct user testing with people who truly need and want to use their product, but finding these people is much easier said than done.
You're so focused on building the product that you don't have the time or resources to identify and coordinate test users. So perhaps you hire a third-party recruiter, give them all the information you can, including demographics and customer traits, and send them off to find user testers.
But that introduces new problems to the mix. Consider how these third party companies find their testers. They turn to a database of users who signed up to be on a list, very likely because they enjoy making extra money testing products in their free time. These are semi-professional user testers who primarily want to get on the panel and will tell the recruiter what they want to hear. Like it or not, the motivation of these testers is first and foremost a paycheck.
As a result, they are almost certainly not your ideal customers. At best, these user testers only have an artificial interest in your product or service. They just don't have any skin in the game.
Testing the users without skin in the game will taint your results. So what are you to do? Is there no alternative to usability testing?
How to test users with skin in the game
Finding test subjects with a true desire to use your product is a difficult challenge. So instead of you trying to identify and find them out in the world, what if you could just test them after they find you?
With replay, you can playback sessions of actual customers as they use your product to get an accurate idea of what's working—or not. Why? Because these users are real. They have skin in the game, an actual need to fulfill, actual concerns that constrain their experience, and something real at stake if your product doesn't work.
The motives of real users are natural. They're coming to your product or service because they have a genuine need. They are intrinsically motivated to navigate through your site or app, as required, to help them accomplish what they want to do. If they hit a barrier, they become frustrated and will try another way. This type of organic user behavior by a motivated subject is impossible to capture in a controlled experiment.
Session replay also captures real customers experiencing your product in their own environment. When you play back their experiences, these users are on their own devices, unencumbered by user testing software.
Most importantly, if (and when) users fail to achieve their objectives, as Jakob Nielsen pointed pointed out, they leave—but not before the tape (e.g. Fullstory) records their moments of frustration online.
Test the right users to deliver the right results
Don't make the mistake of tweaking your site or app based on the needs of artificially incentivized paid user testers. To glean insights that will unlock your product's full potential, reveal new ways to exceed your customers' expectations, and fuel your company's growth, you have to learn from users who are actual users your product.
Consider using Fullstory as an intuitive, cost-effective, natural usability testing research method. It'll help you capture user behaviors from actual customer experiences and leave the crapshoot of traditional user testing behind once and for all.
Afterward: In addition to the structural challenges presented by user testing, there are other problems with testing in a lab setting. For that, take a read through our post on testing biases, as with the Hawthorne Effect.
