Evaluating Mixed-Initiative Systems: An Experimental Approach

Mixed-Initiative approaches to Planning and Scheduling are
being applied in different real world domains. While several
recent successful examples of such tools encourage a wider
use of this solving paradigm, research in mixed-initiative interaction
is still at an early stage and many important issues
need to be addressed. In particular, most of the work has
been dedicated to designing working prototypes and identifying
relevant features of the mixed-initiative interaction,
while much less attention has been given to the problem of
evaluating the approach as a whole. This article is aimed
at addressing some of the many diverse aspects involved in
Mixed-Initiative Planning and Scheduling system evaluation,
highlighting the need for a methodology to provide effective
evaluation studies for this class of tools.
In this paper we consider an established research methodology
in experimental psychology, and adopt it to investigate
specific aspects of mixed-initiative interaction. Specifically,
the experiments described in this article shed some light on
three aspects: (a) understanding users' attitude when choosing
between automated and mixed-initiative problem solving,
(b) investigating recourse to explanation services as a means
to foster the users' involvement in the solving process, and
(c) investigating possible individual differences (e.g., experts
vs. non-experts) in the choice of resolution strategy or access
to explanation.

Publication type: 
Contributo in volume
Author or Creator: 
AAAI Press, Menlo Park [CA], USA
. Menlo Park [CA]: AAAI Press, 2006
Resource Identifier: 
ISTC Author: 
Amedeo Cesta's picture
Real name: