Eye Tracking in Usability Evaluation
Benefits of eye tracking: Eye tracking has many benefits. Using eye tracking methods for usability evaluation one can track how the users interact with a given interface or environment. They help in measuring the effectiveness of a visual element on the interface. The measurements recorded can give an insight into the user’s mind while making a decision regrading searching or navigating an interface. This can be done by measuring what users are interested in, what they are ignoring and what is distracting them from their current tasks. They help in figuring out the Area of Interest which I a randomly shaped box that may contain some information or object or potential interest to the user. The quantitative data collected through eye tracking gives a visuals representation usually in the form of heatmaps(hotspots) which help in better understanding the results. Eye tracking equipment’s used now-a-days like Tobii and SMI are easy to set-up, calibrate and handle. They are unobtrusive and provided accurate real-time feedback. The fast passed usability testing of the software development industry, test conducted using eye tracking can of valuable and useful input providing benefits what will help the users.
Eye Tracking in Usability Evalusation: A practitioner’s Guide [1]
This paper is a general guideline on the use of eye tracking used for usability evaluation. The author talks about the specialists who conduct the study, the metrics used with the testing, the protocol followed to conduct a test. They also talk about when eye tracking can come in handy as compared to the normal usability test which usually include heuristic evaluation, cognitive walkthrough, checklist, remoter-usability testing, click-tracking etc. The authors talk about the different types of eye movements that are measured like saccades, fixation, pupil dilation, scanpaths etc. These movements provide a log of the visual attention of a user on the interface. It is also interesting to know that the eye tracking gadgets only track the movement of one eye, usually the dominant eye of the beholder.
Saccades: rapid eye movements.
Fixation: A linger at a point for more than 300ms.
Scanpath: established by the sequence of saccades and fixations.
Most of the studies conducted analyzed the fixation pattern. Which can be realized through heatmaps. Red indicates the highest level of fixation followed by green yellow. The areas which are not colored drew no attention.
F-Shaped Pattern for reading web content[4]
The author says that a eye tracking study was conducted on 232 users on their reading behaviour on webpages. They found a governing “F” shaped pattern which was undeviating among most of the users. The participants of the experiment were fond to first adhere a long horizontal gaze across the screen, followed by a short horizontal gaze on the area below and finally giving a quick vertical scan of the content of the webpage. Occasionally there were a few cases of an “E” and an “L” shaped pattern but mostly “F” was dominant with variation in the placement of the horizontal bars. These patterns suggest hat usres don’t read the text on a website line by line. They like to give a scan to see if any of the content available if of interest or not. Such reading patterns suggest that first paragraph should contain important and catchy information. While scanning vertically users generally users generally read the first two words of the text.
Visual Hierarchy and Viewing Behaviour: An eye tracking study[2]
The authors of this paper aim to suggest effective improvements in the design of a web page to attract more users and to disperse important information in an effective manner. For this purpose, they have analyzed the gazing patterns of users. The author have tried to decipher the reason behind the “F” shape viewing pattern by most users which could be due to the arrangements of element on the interface or the task at hand. To test this they used 4 prototypes of a webpage with each having a different visual hierarchy. The prototypes differed in the form of being well structed or not and 2 had images of people and 2 did not. 48 participants were used for the study which were from different field of the industry and each one was given either a searching task or a browsing task on one of the 4 prototypes. Fixation patterns using heatmaps was used to analyze the data. It was found that users fixate more on the part of the page which is above the fold. The “F” shaped pattern was not apparent with any of the conditions as the website was more visually complex as compared to a text based simple website. The well structured and organized pages drew more attention. During searching users tend to gaze in a scattered manner with more focus on the left side of the page where the navigation links are usually located. Browsing is less scattered with few fixations time and is usually defined within the center of the interface. Faces drew more attention only when they were placed above the fold of the page and during browsing. They concluded that these insights into a users viewing pattern can help make better designs for different type of websites to attract users.
No subjective ratings were used to access users comfort while dealing with the different types of interfaces.
Efficiency Trust and visual appeal: Usability testing through eye tracking[5]
The authors conducted a research on the efficiency, trust and visual appeal of a website and made recommendations on what kind of an interface would attract more customers to an e-commerce website and what type of user experience would make a first-time visitor a customer. The author believed that an element of social presence will give a more satisfying experience while navigating an automated website which lack humane warmth. This accompanied with a aesthetically pleasing interface will make the users trust the website more. To test this the authors came up with a set of tasks for the users and two hypothesis which were related to images of people on a website and the visual appeal of a website. 522 users were recruited for an online study and each of them were provided with a different prototype of a website. Users were asked the visual appeal and the level of trust on the website using seven point Likert scales. After performing p on the online study, authors concluded that websites with images of people are more visually appealing and in turn elicit a higher level of trust. It was fond that the users conduct the search from left to right. The online study was followed by an eye tracking experiment to confirm the results of the online study. They used a count heat map and a fixation heatmap. 40 were recruited for the eye tracking experiment. The experiment confirmed that images of faces are more helpful in drawing attention.
Gender Preferences in Web Design: Usability Testing through Eye tracking[6]
The authors of this papers tried to investigate the visual predilections of male and female users. They have tried to detect he noticeability of bricklets(small box containing useful information) based on their background colour images by male and female users. Since men and woman have different notion of attractiveness, an hypothesis was made on the noticeability of the different prototypes of the bricklets. Men have a tendency to like dark colors and women have a tendency to like light color. 36 participants were chosen for the study out of which 17 were male and 19 were female. 4 type of bricklets were designed with dark and light background color and with and without images. Tasks were designed in such a way so that the users were tested on their ability to detect certain kind of bricklets. It was found that both male and female fixate less on images and more on the bricklets without images. Female users were attracted more towards the darker background bricklets and male users were not biased towards a specific colour. The study concluded that what users find appealing is not necessary noticeable this was supported by the discrepancy in self report survey and the fixation duration on the bricklets by the users.
Generation Y and Web Design: Usability through Eye tracking[7]
The aim of the authors of this paper is to come up for better design recommendation for retail web sites to attract the millennial generation also knows as generation Y. This generation poses the strongest internet skills and spend around $200 a year on online retail. An hypothesis is made that Generation Y like large images preferably of celebrities and with search features and as little text as possible. Top 50 retail pages were selected. These pages were ranked on the characteristics posed by them based on the predilections of generation Y. A self report study and an eye tracking study was conducted to compare both the results. 99 participants were selected who possessed strong internet skills for the self report study. 9 participants were selected for the eye tracking experiment. The survey and the data provided by the eye tracking device supported the hypothesis made.
Eye tracking in Web Search Tasks: Design Implications [3]
The authors of this paper conducted an eye tracking usability testing for a web portal application which was developed by Oracle portal software. Seven users were selected to complete six tasks for the study. The users came across 15 screens while completing their tasks and had to spend around 360 seconds to complete one task. The authors recorded the key presses and mouse clicks on the interface and eye movements like saccades and fixations and the dwell time on the Area of Interest were diligently recorded. Every task started with a set of scenario and required the user to perform them based on his understanding. The results in this study were not in the form heatmaps but rather in the form of screen bitmap files, fixation file and object file. These files were amalgamated to perform data analysis. The study was performed in a very systematic way. The authors studied the eye movements of the users at the task level then at the screen level and then finally at the object level with the focus on Area of Interest. In-depth study was conducted regarding screen visits and distribution. The authors analyzed the pattern for navigating within and between portlets and concluded that there is an inclination for the users to notice the portlets on the left and the upper part of the screen in view. The authors tried to discover a relationship between user action, scene sequence and the data derived from the eye movement.
Age-related Differences in Eye Tracking and Usability Performance: Website usability for Older Adults [8].
The authors of this paper conducted usability study of websites through eye tracking to make design recommendations which keep in mind the needs and ease of usability and navigation of older adults. 5 websites were used for the experiment. And a comparison was made on how you and old people interact with the environment. Around 3-7 young adults and 2-3 older adults were recruited. All of them a possessed a working knowledge about using the internet. The participants were given predefined tasks and their fixations were recorded using the unobtrusive eye tracking device. The participants also filled out a questionnaire to report their satisfaction level. It was found that older adults focus more on the center portion of the screen and ignore the information located on the peripheral areas. Cluttered periphery results in performance hindrance for the old users. The ignorance of the edge of the screen can be related to the cognitive decline encountered due to growing age. Older adults were less accurate and efficient as compared to younger adults. Older adults who are ususally unfamiliar with the environment get easily distracted by unnecessary elements.
Seven users are very less as compared to the recommendation of 10-30 participants made by [1]. There were often stements made by the authors that there is “presently too little evidence to support the notion” [3] suggesting that clearly 7 participants for the experiment was not enough and the results produced can be a threat to external validity. [1] also suggest to recruit 20% more participants in case there are problems with calibiratio of the equipment with the users. [2] used 48 users from different fields of the industry sector age not mentioned. 40 people in [5]. [6] 99 participants were recruited. [8] small number of users. Limits on generalizations made.
Users were tested for their visual acuity
The users were made to sign demographic, consent and disclosure forms.
The authors conducted initial training tasks
They used a head mounted system. Although these systems are good for tasks which may require a lot of head or body movement, they are obtrusive. They obstruct a certain portion of the of the field of view of the user and can interrupt a user in hi cognitive process or task at hand as he is constantly reminded of a apparatus being setup on his head.
Since this experiment was conducted in 2002 the eye tracking instruments used that time were not technologically that advanced. Therefore, a lot of time and effort went into analyzing the data. Tape recorders were used to capture users comments and a video recorder was also used for in depth analysis.
While conducting the experiment it was found that one of the users took very less time to complete the task as compared to others. The authors concluded that users learned about the interface while performing the tasks. But since one of the users showed less reaction to time to the given task at hand the inference suffers from a threat to internal validity because he possessed previous knowledge about similar interface.
Threats to generalizibilty as an intrusive setup was used.
Dis advantages: tedious, requires extensive data reduction, focus on micro-level behaviours, too much noise in the data. Lots of work required to narrow down on the data that is needed
Eye tracking study is usually conducted where results of traditional software usability results are compared with that of the results from the eye tracking derived results. The traditional usability tests are usually done using five point Likert scale.
In many experiments authors have concluded that users are inclined towards noticing the upper left side of the interface. It maybe possible that users in these experiments are used to languages that goes left to right and hence are natural disposed to look at the upper left side of the screen. If there experiments were conducted on a subset of population that are used to languages that go from right to left then there could have been different results.
Most of the papers have conclude that users tend to ignore important information what is located at the bottom of the page which if below the fold. It could be father investigated that is that information is accompanied with an image of a person then would it attract users or not.
While investigating the usability of an interface mostly two types of tasks were used: browsing and searching. This is because it the viewing behaviours of the users change depending on their task. In searching the users try to find a point of entry into a page and then from their he scans the near the point of entry for relevant information.
In the heatmap the areas that are red could be the point of entry for those elements of the interface and the yellow and green areas in the heatmap are the surrounding locations where the users try to locate relevant information.
Since the search process is majorly influenced by the factors such as, images, color and text. Designers can make use of these factors to guide the searching process of the users
A trend was found that larger images attracts more attention as compared to smaller images. This can be used to attract user attention towards information that need immediate attention.
Studies suggest that images of people tend to attract more attention. For [2] more prototypes of the web pages could have be formulated and compared to see if that was really true.
Heat maps are created using data from several participants and are used to analyze the pattern of fixation on an area on the interface.
The findings in [2] are in contrast with the one in [4] where the author has concluded on a dominant “F” shaped pattern. It can be absorbed what for a text based website with a simple visual hierarchy an “F” shaped pattern of viewing is more visible but for websites that have a complex visual hierarchy the viewing pattern in more scattered although focusing a lot on the left side if the interface which again is dominated by the users reading style form left to right. Different results will most certainly be produced with users who are habitual in reading from right to left.
In general websites with good visual appeal and easy navigation elicit trust worthiness and are major contributing factors towards an enhanced user experience.
[5] says that when since images attract attention, can be used to draw users attention to important information below the fold of a page. But the author of [2] contradicts this by saying that even though images of human faces were placed below they fold they attracted very little attention.
[5] says that there is no effect of the gender of the image of the face on the website but in [?] notice a lot re spots on the heat map over the images of a female. Also, further studies can be conducted to see male images are more appealing than female image and which gender of the images attract the male users and which gender of the images attract the female users.
The analysis of the heat maps performed by [5] was more structured as they used a standardized heatmap. Such a map puts a limit on the number of fixations to be considered as a red zone across different prototypes.
Although in [5] the authors conclude that users trust level can be enhanced by the increasing visual appeal it should be noted that visual appeal is not all that matters, functionality of a website also plays a major role.
the authors of [5] said that users are attracted toward the area where the images of faces were located but little fixation on the images. This is opposed by [?] where the authors have concluded that generation Y is then to fixate on the images of faces more. Then again the age difference between the participants in [5] was spread out therefore the two papers made contradictory statements. Also, since [5] is making recommendations for a e-commerce website which is mostly used by generation Y they could have narrowed down the are limit of the participants of the experiment.
Papers [5] and [2] concluded that images of faces attract more attention but it maybe possible that images of different genre may be more effective. These images when paired with a lowers aesthetic quality webpage may not attract any attention at all.
Keeping the ethical issues in mind [],[], and [] did not made the users sign Informed consent form and no information is provided if the participants were briefed or not.
[7] suggest that generation Y is more attracted towards images of celebrities. But the study may fail external validity as it has not considered the population of the people who are not familiar with those celebrities. To extend this study a search task could have been given to the users to gain a more indepth understang of likes and dislikes of generation as done ny authorss in [2] and [3].
[7] conducted a research on generation Y but we know that baby boomers are constitute major part of the population that uses the internet. Results form [2], [7] and [5] can be combined to make recommendations for designs that would attract more users and increase user satisfaction. These designs would generally contain more images of faces, important information will be located on top left corner of the screen. To draw attention towards the part of the page below the fold images of celebrities can be used in an effective manner.
Most studies concluded that users tend to focus on the left side of the interface but the studies conducted in[8] concluded that older adults ignore the left navigation area. This suggests that younger users who are more familiar with the interface know where to look at for navigation but this lack of knowledge and experience hinder the performance of an older adult.
To help out the tease out the age effect designers can do a “dual placement” where navigation information can be placed in the center and the peripherals of the screen.
All the eye tracking studies performed were under a laboratory condition. Here the users are asked to perform task in which they might be interested. These results maybe different in a real world situation when users are performing tasks of their own interest and are aware that the are not being monitored.