Critical Evaluation

Interaction Evaluation

Designing an interaction can be a daunting task, notably when the design is finished and it is then difficult to assert the quality of the work. One way to address that is to conduct a thourough evaluation of the interaction. There are many evaluation techniques and methods, such as comparative studies, focus groups, interviews...

All methods however rely on the measuring of some parameters related to the interaction. The classical measurements are enshrined in a ISO standard (ISO 9241), the new measurements have been developed over the years to address my various projects and their specific requirements. These lists are not exhaustive, and should be amended to fit particular projects/interactions. Tthey should be used as guidance and inspiration.

Classical Measurements

The basic measurements (from ISO 9241):

  1. Effectiveness: How well has a task been performed, how easy was it to do so. It is about theaccuracy and completness with which the users achieve tasks
  2. Efficiency: Interaction duration, bandwidth and intensity, time-on-task, resources and modalities required. It relates to the resources used and necessary to achieve accuracy and completness in the performance of tasks by the users
  3. Affect: Effects of the interaction on the user emotions, moods, motivation and attitude, it is also about comfort

New Measurements

These measurements are more comprehensive and a better reflection of what users value:

  1. Learnability: Extent to which the interaction can be learned and memorised.
  2. Intuitiveness: How little cognitive effort is required to perform the interaction
  3. Discoverability: How litte cognitive and perception efforts are required to discover the interaction
  4. Experience: How does interaction combine with aesthetics, perception and action. Engagement, flow and attention focus are also evaluated
  5. Context: The granularity of the user perception of what is happening, as well as the overall time, location, domain, user and application combination
  6. Usability: Bugs and defects and ther effect on user confusion, error, delay and failure to perform a task. The severity, pervasiveness and number of users affected by usability issues is also measured
  7. Usefulness: How useful are the interaction and the task performed in relation to the user intents and motivations. The discrepency between usability and usefulness needs to be highlighted here
  8. Comfort & Egonomics: Is the user comfortable while performing an interaction task, how good is the ergonomics of the device/system.