Computer Science · Artificial Intelligence
Evaluating Evaluation: An Empirical Examination Of Novel And Conventional Usability Evaluation Methods
Jonathan Crellin And, Jonathan Crellin, Jenny Preece
See on the mapAbstract
This paper reports on the strengths and weaknesses of the methods employed for data collection, rather than on actual conclusions about the stimulus interfaces used. 2. METHODOLOGY 2.1. The Task The evaluation task is calculating the cost of a telephone call, using different versions of a prototype application. Each user is required to use all the different versions. Most people are familiar with telephone usage so that the task domain should not be unfamiliar to any user. However the method of call charging employed by BT is not particularly obvious, and so many people will not be familiar with exactly how telephone charges are calculated. BT charge calls by measuring the number of units used in each call. Each unit costs 5.06 pence. Some calls are more expensive than others, due to the distance called, or the amount of equipment used. For example overseas calls are typically much more expensive than inland calls and calls to mobile phones are particularly expensive. BT cope with this by allotting different amounts of time per unit to each distance band, hence a Local call may use one unit every 360 seconds, and a call to Hong Kong may use a unit every 3.5 seconds. Additionally BT vary the amount of time per unit according to the time of day, so that busy periods are more expensive. Inland calls usually have three charge rates, cheap, standard and peak, however some overseas calls have only one charge rate, and many have just two. British Telecom, (1987). In this study the functional part of each version of the prototype is clearly separated from the interface. There is considerable scope for variation in the way a user enters call parameters (distance and time of day). Also the way a user calls the different application functions provides considerable scope for interf...