DAGGER: Instance Selection for Combining Multiple Models Learnt from Disjoint Subsets

We introduce a novel instance selection method for combining multiple learned models. This technique results in a single comprehensible model. This is to be contrasted with current methods that typically combine models by voting. The core of the technique, the DAGGER (Disjoint Aggregation using Example Reduction) algorithm selects training example instances which provide evidence for each decision region within each local model. A single model is then learned from the union of these selected examples. We describe experiments on models learned from disjoint training sets which show that DAGGER performs as well as weighted voting on this task and that it extracts examples which are more informative than those that can be selected at random. The experiments were conducted on models learned from disjoint subsets generated with a uniform random distribution. DAGGER is actually designed for use on naturally distributed tasks, with non-random distribution. We discuss how one view of the experimental results suggests that DAGGER should work well on this type of problem.