This item is available under a Creative Commons License for non-commercial use only
Intelligent Virtual Agent (IVA) systems are notoriously difficult to evaluate, particularly due to the subjectivity involved. From the various efforts to develop standard evaluation schemes for IVA systems the scheme proposed by Isbister & Doyle, which evaluates systems across five categories, seems particularly appropriate. To examine how these categories are being used, the evaluations presented in the proceedings of IVA '07 and IVA '08 are summarised and the extent to which the five categories in the Isbister & Doyle scheme are used is highlighted. Finally, to illustrate how the full scheme can be used, an evaluation of an IVA system for computer game characters across the breadth of the five categories is described.
Mac Namee, B. & Dunne, M. (2009) Widening the Evaluation Net, 9th International Conference on Intelligent Virtual Agents (IVA '09), Amsterdam, 14-16 Sept. doi:10.1007/978-3-642-04380-2_74