Alex Poole - Interaction design and research

• Home
• Services
• Commercial work
• Academic work
• FAQ

Back to contents

Masters Thesis - Issues of Saliency and Recognition in the Search for Web Page Bookmarks

6. Results

6.1) Response times

Mean response times per condition were derived for each participant and reflected the time taken between the appearance of a bookmark menu and the participant registering that the target bookmark had been detected.

Faster response times were taken as indicating superior recognition (Table 6).

To retain as much data as possible, response times were scored even if participants failed to find the target bookmark. This does not invalidate the results as the response times were not a measure of failure or success in recognising the bookmarks, but rather a measure of relative differences in recognition. If a bookmark was not found, a maximum response time of 30 seconds was scored.

_{Note. Values enclosed in parentheses represent standard deviation.}

Correlations

No significant correlations were found between response time on any condition and the years using the Web or years using computers, as reported by participants in the questionnaire. Also, no significant correlations were detected between the size of the site name logo and the response times for the three conditions of bookmarks with a top-down structure.

ANOVA

A two-way repeated measures ANOVA was used to analyse these data and revealed that there was no main effect of Bookmark structure (top-down vs. bottom-up) (F_{(1, 29)} = 0.155, p = 0.697) but there was a main effect of number of cues (one, two or three) (F_{(2, 58)} = 8.443, p = 0.001).

Employing the Bonferroni post-hoc test, significant differences were found between the one cue and two cue conditions (p = 0.004) and the one and three cue conditions (p = 0.011). However, no significant differences were found between the two and three cue conditions.

6.2) Adjusting eye movement data for phrase length

As the eye movement data was analysed per area of interest, a raw count of fixations would show misleading results as they do not take into account the length of the text phrases contained within the areas. To adjust for this, the mean number of fixations on each element was divided by the mean number of words in the phrase. (2.79 for site name, 6.83 for article title and 1.79 for the section name). In this way, we are able to separate higher fixation frequency due to the simple fact that there were more words to read, and higher fixation frequency because an item is actually harder to recognise.

Mean fixation duration is not contingent on the number words in the phrase, so this measure was not adjusted. It remains a measure of mean fixation duration on the whole area of interest. Finally, eye movements were analysed for 24 of the 30 participants.

6.3) Eye movements during the encoding task

A one-way repeated measures ANOVA was used to analyse mean number of fixations per element (Table 7) and revealed that there was a main effect of the type of element being viewed (adjusted for phrase length) (F_{(2, 46)} = 68.962, p < 0.001).

Bonferroni post-hoc tests revealed that the element most frequently fixated was the name of the site. It was fixated on average 2.41 times, slightly but significantly (p = 0.12) more often than the 2.09 fixations that fell on the title of the article, and more than double the number fixations than on the section name (p < 0.001) which received 1.08 fixations. The title of the article was also fixated almost twice as frequently as the section name (p < 0.001).

A one-way repeated measures ANOVA was also used to analyse mean fixation duration (Table 8), and a main effect was found according to type of element being viewed (F_{(2, 46)} = 8.948, p = 0.001).

Bonferroni post-hoc tests revealed that the mean fixation duration on the name of the site was slightly longer at 241ms than on the title of the article at 225ms (p = 0.001) and longer than on the name of the section at 227ms (p = 0.021). The mean fixation durations on the article title and the section name were not significantly different.

6.4) Eye movements during the visual search task

In the search task, participants consistently scanned down the left had side of the bookmark menu, as has been found in similar studies of menu search (Altonen, Hyrskykari & Räihä, 1998). Fixations were largely concentrated in the second 8th of the bookmark menu, which corresponds to the first four letters of the first word of each entry (Table 9a). Saccades were also concentrated towards the left of the menu (Table 9b).

This result provides confirmation that the lead cue in the bookmark does in fact lie in a dominant position. For the purpose of analysis in the present study, we measure eye movements only on the lead cue, and assume that it is a fair proxy for the bookmark structure as a whole.

Number of fixations

A two-way repeated measures ANOVA was used to analyse the mean number of fixations on the leading cues in the bookmark (Table 10). In the present study, a higher number of fixations are an index of greater uncertainty in recognising the target. The analysis revealed that there was a main effect of the bookmark structure (top-down vs. bottom-up) (F_{(1, 23)} = 73.962, p < 0.001).

_{Note. Values enclosed in parentheses represent standard deviation.}

There was also a main effect of the number of fixations (adjusted for phrase length) on the leading cues when extra cues were added (F_{(2, 46)} = 12.259, p < 0.001).

Lastly, there was a significant interaction (F_{(2, 46)} = 4.620, p = 0.015) between the bookmark structure and the number of cues. The number of cues affected fixations differently depending on whether site name or article title was the leading cue.

Post-hoc tests were used to explore the interaction effect further. A stringent alpha level of p < .005 was set to accommodate the fact that multiple comparisons were being made.

When the name of the site was presented alone, it received 1.34 fixations, much higher than the 0.75 fixations that fell on article title when it was presented alone (F_{(1, 23)} = 44.362, p < 0.001). Adding another cue after site name reduced the number of fixations to 0.87 (F_{(1, 23)} = 20.828, p < 0.001), a much larger reduction than adding a cue to article title, which reduced slightly to 0.61 fixations (F_{(1, 23)} = 20.828, p < 0.001). Increasing the number of cues to 3 actually increased the number of fixations on both leading cues, though not significantly.

As indicated previously, bookmarks with a top-down structure received a significantly higher number of fixations at all levels than bookmarks with a bottom-up structure.

Mean fixation duration

In the present study, information which requires longer fixations is less meaningful to the person than information with shorter fixations. A two-way repeated measures ANOVA was used to analyse the mean fixation duration on the leading cues in the bookmark (Table 11). A main effect of the bookmark structure (top-down vs. bottom-up) was revealed (F_{(1, 23)} = 10.437, p = 0.004), as well as a main effect of the number of cues (F_{(2, 46)} = 5.742, p = 0.006).

_{Note. Values enclosed in parentheses represent standard deviation.}

Lastly, there was significant interaction between the bookmark structure and the number of cues (F_{(2, 46)} = 5.948, p = 0.005). The number of cues affected fixation duration differently depending on whether site name or article title was the leading cue.

Post-hoc tests were used to explore the interaction effect further. As before, a stringent alpha level of p < .005 was set to accommodate the fact that multiple comparisons were being made.

When the name of the site was presented alone, it received an average fixation duration of 335ms, 61ms higher than the 274ms that were spent fixating the article title when it was presented alone (F_{(1, 23)} = 44.362, p < 0.001). However, adding an extra cue brought fixation duration on site name down by 43ms to around the same level as for article title, but adding the 2nd and final cue increased the gap again, so that fixation duration on site name was 20ms longer than on the article title alone, although not significantly (F_{(1, 23)} = 8.150, p < 0.01). While fixation duration on the site name continued to be affected by the addition of extra cues, fixation duration on article title was not significantly affected at all.

6.5) Questionnaire results

Basic demographic information was collected through a questionnaire (Appendix D). Responses are described in section 4.1.

© Copyright 2004 - 2005 Alex Poole | Site updated: 7 April, 2005 www.alexpoole.info