The Evaluation of Creative Systems

As creative systems become more advanced, there is a growing need to assess their performance, whether during development or after implementation is complete. There is not yet any settled methodology for evaluation, despite a continuing debate. The task requires answers to two methodological questions: which properties of the behaviour of a creative computational system should be considered, and what are suitable ways of measuring these properties? It is essential to take into account the long-term aim of the work being evaluated, as different theoretical agendas may lead to variations in evaluation requirements. We review some of the theoretical and methodological suggestions that have been made, ranging from candidates for essential ingredients of creativity to practical matters about rating the output of programs. Many of the essential judgements about the success of a creative system can be made only subjectively, which means that it can be useful to borrow methods from experimental psychology. Nevertheless, evaluation within computational creativity has its own unique attributes, requiring specific approaches.