Abstract
In this paper we illustrate how novel AI methods can improve the performance
of intelligence analysts. These analysts aim to make sense of — often conflicting or incomplete — information, weighing up competing hypotheses which
serve to explain an observed situation. Analysts have access to numerous visual analytic tools which support the temporal and/or conceptual structuring
of information and collection, and on the other hand other tools can be used to
evaluate alternative hypotheses. We believe, however, that there are currently
no tools or methods which allow them to combine the recording and interpretation of information, and that there is little understanding by analysts about how software tools can facilitate the hypothesis formation process. Following the identification of these requirements, we developed the CISpaces (Collaborative Intelligence Spaces) decision support tool in collaboration with professional intelligence analysts. CISpaces combines multiple AI-based methods including argumentation theory, crowdsourced Bayesian analysis, and provenance recording. We show that CISpaces is able to provide support to analysts by facilitating the interpretation of di↵erent types of evidence through argumentation-based reasoning, provenance analysis and crowdsourcing. We undertook an experimental analysis with intelligence analysts which highlights three key points. (1) The
novel, principled AI methods implemented in CISpaces advance performance in
intelligence analysis. (2) While designed as a research prototype (at TRL 3),
analysts benchmarked it against their existing software tools, and we provide
results suggesting intention to adopt CISpaces in analysts’ daily activities. (3)
Finally, the evaluation highlights some drawbacks in CISpaces. However, these
are not due to the technologies underpinning the tool, but rather its lack of
integration with existing organisational standards regarding input and output
formats. Our evaluation with intelligence analysts therefore demonstrates the
potential impact that an integrated tool building on state-of-the-art AI techniques can have on the process of understanding complex situations, and on how such a tool can help focus human e↵ort on identifying more credible interpretations of evidence.
of intelligence analysts. These analysts aim to make sense of — often conflicting or incomplete — information, weighing up competing hypotheses which
serve to explain an observed situation. Analysts have access to numerous visual analytic tools which support the temporal and/or conceptual structuring
of information and collection, and on the other hand other tools can be used to
evaluate alternative hypotheses. We believe, however, that there are currently
no tools or methods which allow them to combine the recording and interpretation of information, and that there is little understanding by analysts about how software tools can facilitate the hypothesis formation process. Following the identification of these requirements, we developed the CISpaces (Collaborative Intelligence Spaces) decision support tool in collaboration with professional intelligence analysts. CISpaces combines multiple AI-based methods including argumentation theory, crowdsourced Bayesian analysis, and provenance recording. We show that CISpaces is able to provide support to analysts by facilitating the interpretation of di↵erent types of evidence through argumentation-based reasoning, provenance analysis and crowdsourcing. We undertook an experimental analysis with intelligence analysts which highlights three key points. (1) The
novel, principled AI methods implemented in CISpaces advance performance in
intelligence analysis. (2) While designed as a research prototype (at TRL 3),
analysts benchmarked it against their existing software tools, and we provide
results suggesting intention to adopt CISpaces in analysts’ daily activities. (3)
Finally, the evaluation highlights some drawbacks in CISpaces. However, these
are not due to the technologies underpinning the tool, but rather its lack of
integration with existing organisational standards regarding input and output
formats. Our evaluation with intelligence analysts therefore demonstrates the
potential impact that an integrated tool building on state-of-the-art AI techniques can have on the process of understanding complex situations, and on how such a tool can help focus human e↵ort on identifying more credible interpretations of evidence.
| Original language | English |
|---|---|
| Article number | 200151 |
| Number of pages | 35 |
| Journal | Intelligent Systems with Applications (ISWA) |
| Volume | 17 |
| Early online date | 8 Dec 2022 |
| DOIs | |
| Publication status | Published - Feb 2023 |
Bibliographical note
AcknowledgementsWe would like to thank the professional analysts from the UK, US and international agencies for their support in developing this research. This research was sponsored by the U.S. Army Research Laboratory and the U.K. Ministry of Defence and was accomplished under Agreement Number W911NF-06-3-0001. The views and conclusions contained in this document are those of the author(s) and should not be interpreted as representing the social policies, either expressed or implied, of the U.S. Army Research Laboratory, the U.S. Government, the U.K. Ministry of Defence or the U.K. Government. The U.S. and U.K. Governments are authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon.