scoreberlin GmbH
Agentur für Usability-Beratung & Optimierung

Haupt-Navigation:

• Services
• Aktuell
• Profil
• Kontakt
• Presse

Case Study: Usability Tests and Eye Tracking With Morae

Sub-Navigation:

1. Neues & Projekte
2. Usability-Artikel
3. Article: Usability Tests...

Usability Tests and Eye Tracking

Extreme Usability (XU): TechSmith's Morae in Practice

Update 2010: This article on retrospective think aloud methods in combination with eyetracking is from 2006 and kind of outdated. It describes live integration of eyetracking gaze data into Morae Observer utilizing a VNC setup. Meanwhile, the eyetracking solution NYAN XT comes with Morae integration: On click Morae 3 records eyetracking data from NYAN and saves it automatically into Morae's RDG file. In Morae Observer, eyetracking gaze data are visually synchronized with the interaction. The complete interaction including eyetracking can be saved and exported as a video from Morae Observer. In Morae Manager you can analyze and export the eyetracking visuals.

Within shortest time TechSmith's usability software Morae has entered many usability labs, big and small. Morae is even compatible with modern eye tracking systems. This article describes how this works and which method is especially well suited.

TechSmith's Morae contains three components: »Recorder«, »Remote Viewer«, and »Manager«. Morae records user videos, audio, screen and system events and synchronizes them real-time, creating a complete digital record of the test participants' user experience. The Remote Viewer connects to Recorder over a network and displays the full experience. Afterwards the recordings can be edited in Morae Manager utilizing markers, automatic segment creation, powerful search and producing highlight videos quick and easy.

In a common usability test setup we have three »usabiliteers« run a test. The facilitator is testing the participant, while the other two are observing: One is taking »quotes'n'notes« (ad-hoc findings) directly into a web-based application, and one is setting markers and notes via Remote Viewer for later analysis in Morae Manager. Usually customer's developers, designers, or project managers are also observing tests, taking notes and drinking lots of coffee.

Total User Experience DVD

While the test is running, the complete user experience is recorded simultaneously into a big WMV file by the Remote Viewer – including the participant's video and audio as PiP (picture in picture), highlighted mouse movements and clicks, and the screen recordings. When the last session is completed, the developers or project managers drink a last cup of coffee while we burn DVDs with the WMV files and export the »quotes'n'notes« into PDF or MS Excel. (Certainly we offer the following formats as well: AVI, Quicktime MOV, MPEG or DV.)

The advantage is obvious: Directly after the test customers are able to evaluate our first findings and analyze the Morae videos. And only a few days later they receive a full size usability report in CIF (Common Industry Format) with entire transcriptions and highlight videos. Now think back two or three years, remember the old times and laugh about yourself having offered »Rapid Prototyping« back then. Today is »Extreme Usability« (XU).

UCEE – a Hybrid Usability Methodology

One of the great advantages of Morae is certainly the opportunity of delivering and communicating the results really fast. But Morae also allows us to share our insight into total user experience when we additionally use our eye tracking system and our UCEE methodology (User/Customer Experience Eyetrack Evaluation).

UCEE (speak: You See) is a hybrid usability methodology which combines quantitative eye tracking and qualitative interviews into one session. This was not possible with common eye tracking systems before due to their low head motion tolerance (which is one of the reasons for using eye tracking as a single method for pure quantitative testing). With modern eye trackers this key limitation belongs to the past. Our binocular 120 MHz eye tracker allows participants an enormous free head range while calibration happens automatically and lasts only seconds. So we can test eye gaze and qualitative user experience together within one session.

In brief, UCEE lets the participant focus on the stimulus and task without having to talk or think aloud. After each »Eyetrack Walkthrough« we repeat the scenario with the think-aloud method by replaying the eye gaze video with the visualized scanpaths. We have found participants have considerably fewer problems to reflect on events that just happened and express their thoughts in own words more precisely when the scenario is instantly replayed (in comparison to conducting both methods separately; Guan, Lee, Cuddihy, Ramey, »The Validity of the Stimulated Retrospective Think-Aloud Method as Measured by Eye Tracking«, 2006). This is especially true in cases with shy and analytical thinking participants (Lewis/Mack, »Evidence from thinking aloud protocols«; Nisbett/Wilson, »Verbal Reports on Mental Processes«).

Results To Go

The complete walkthrough is recorded by Morae and synchronized into one file. Morae sends this file (including video, audio, mouse movements and clicks, system events and eye tracking video) as a live video feed to the Remote Viewer in the observation room. During the »Eyetrack Walkthrough« the observers have insight into the whole user experience: On the canvas they see the actual screen recording, in the lower left side they can watch the video of the test participant and listen to him. The mouse cursor is highlighted by a yellow dot, clicks are visible by a red triangle. The eye gaze overlay is visualized by a green dot. In this way the observers are able to track the participant's mouse and eye movements simultaneously – the complete interaction at a glance. This is really an advantage because observers have a live view on what participants are really looking at or what they ignore.

How to Gain »Total User Experience«

Again, Morae helped us to build this quick'n'dirty setup. In the usability lab we have computer A with the eye tracker and a running VNC server and computer B with mouse, keyboard, Morae Recorder and a VNC client connected to computer A. Computer C with Remote Viewer is located in the observation room, connected to computer B.

All gathered data is synchronized in real-time and broadcasted live to the observation room.

Computer A (eye tracker) broadcasts the gaze positions together with the screen contents to the Morae computer B. On computer B Morae is recording the synchronized screen contents, system events, and the test participant via webcam. Whenever computer C connects to computer B over Remote Viewer, it receives a fully synchronized video together with video, audio, mouse and eyegaze highlighting (video example on right side).

 
Left: Participant's scanpath. Right: Heatmap of a group of participants. (Click images to view a full size version.)

As a relevant evaluation factor eye tracking is undoubtedly closing the big gap in the usability method's mix, especially in combination with classical methods like user tests and expert evaluations. For example Web2.0, AJAX, and web based dynamic applications lead to new hypes as to new questions concerning de facto standards, conventions, and common sense, also concerning users and their experiences. Our pragmatic usability approach gives the answers to these urging questions.

Beneath traditional methods Usability2.0 needs user centered approaches and methods which consider the requirements that new technologies and products demand from their users. More than ever aspects like user acceptance, aesthetics, fun, joy of use, and hedonistic quality come to the fore. The combination of qualitative and quantitative methods like UCEE bears these aspects in mind.

Morae is especially great for qualitative user testing. Problems occur when you have to explain why the navigation has been difficult to understand for users or how they can be better supported by better working designs. At this point the combination of Morae and eye tracking really makes sense.

Questions you can't answer without eye tracking methods are typically those referring to attention, contact time, or orientation:

• Were elements or objects perceived?
• For how long has an object been perceived?
• Which elements attract attention first?
• Does the user find important elements?
• Which positions, which designs work better?
• What do users really read, what do they ignore?

Conclusions

Morae revolutionizes the usability lab. Especially in processing and analyzing the tests and presenting the results: Usability testing has never been that efficient. Our customers are completely hooked.

»Extreme Usability«: By using Morae we save time. Almost immediately after ending the sessions the customer receives the videos on DVD and first results he can work with. Afterwards, we open the project in Morae Manager with the markers being set via Remote Viewer. Now we are able to easily analyze the scenarios and cut and edit highlight videos – all in one application.

Morae allows to develop newer, simpler and faster methods. Today, without Morae we would still work with methodologies and technologies from the last century.

• TechSmith Morae
• scoreberlin's usability lab
• Eye tracking analyses

Published: 05/2006
 Marcus Völkel 

Artikel auf deutsch lesen.

Real-time video: The green dot visualizes the eye gaze (eye tracking), the yellow dot highlights the mouse. A red triangle shows clicks. In the lower corner you see the participant's video.

Heatmap: Red areas received most attention. The green line represents the page fold on the computer screen; participants had to scroll to view the content below.
View a full size version.

© 1999-2010 scoreberlin® GmbH. scoreberlin ist eine inhabergeführte Agentur für Usability-Beratung und -Optimierung. Zu unseren Leidenschaften zählen Usability-Testing, User Experience, Mobile UX, Eyetracking, UX Research sowie projekt- und prozessbegleitendes Usability Consulting. Wir arbeiten für renommierte Agenturen, Behörden und Unternehmen. Über 85 Prozent unserer Auftraggeber binden uns wiederholt in ihre Projekte ein.