Introduction

A task-based usability test was carried out on an ATM hosted by Crédit Lyonnais, a leading French high-street bank. The focus of the test was the task of transfering money from one account to another.

The test

Design

Carrying out a usability test on a real ATM can be problematic for several reasons:

• real customers can be obstructed from using the machine
• members of the public may interupt the test
• test participants may be less willing to take part if they have to travel to a test location (especially if they are not being paid)

The solution was to create paper mock-ups of the ATM screens (see materials). The advantages to this approach was that we could bring the test to the user and perform it in a controlled environment, while retaining the essential elements of the interaction.

The mock-ups were also very cheap to produce (around £2). The full portable testing kit included:

• paper mock-ups of the ATM screens
• a paper mock-up of a qwerty keyboard and number pad
• a cash card
• destination account details

Procedure

The test participants were asked to 'talk out loud' while interacting with the paper mock-ups so that the test moderator could better understand their actions.

They stepped through the process of transfering money by pressing 'buttons' on the paper mock-up screens, the test moderator presenting screens in the exact same sequence as they would appear on the real ATM.

Participants

Seven French users participated in the evaluations. Three users had accounts at Crédit Lyonnais and had used an ATM one or two times in the past to transfer money. The remaining four users did not have an account at Crédit Lyonnais and had never transferred money using an ATM.

Results

Issue

Screen 10 displays the statement "Veuillez indiquer le motif de votre operation" ("Please state the purpose of your transaction") - this caused significant hesitation and confusion in all users. This delay has the potential to annoy not just the customer making the transfer, but of course, all the other customers waiting in the queue behind them.

Analysis

The participants' verbal reports revealed that the meaning of the statement was consistently misinterpreted. Users reported that they thought they had to:

• Confirm that they really wanted to make a transfer, even though they had just done so in the previous screen
• State the personal reason why they were making the transfer, for the bank's records, even though this is none of the bank's business(!)

The text that the user enters is actually printed on their monthly statement to serve as a reminder of what the transfer was for.

Eventually, all users typed "Virement" ("Transfer") or a variation of the word, thereby adding useless information to their statements.

Recommendation

Change the wording of the phrase to clarify its meaning. Test with more rapid low-cost evaluations.

Comment

It's surprising that such a significant usability issue persisted in an ATM system that undoubtedly cost thousands of pounds to develop. It's even more surprising that we discovered it using a test that cost only £2 to produce!

It just goes to show that usability can be implemented cheaply and still produce valuable results.