Applying Rule Extraction & Rule Refinement techniques to (Blackbox) Classifiers

Julius Cepukenas, Chenghua Lin, Derek Sleeman

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Black-box classifiers are able to classify unseen instances, once they have been trained on an appropriate (domain) dataset. Such classifiers have the advantage of being generally very efficient but the disadvantage of not being able to explain their processes to a user. For these reasons, over the last decade or so, a number of rule extraction algorithms have been developed which are able to extract a rule-set from classifiers. The focus of this project has been to re-implement a state-of-the-art rule extraction system, OSRE [1], and then to show that when the extracted rules are refined by the Knowledge Refinement system, FIXIT, that the refinement process, in virtually all cases, improves the fidelity of the refined rule-set when compared with the rule-set extracted by OSRE. A statistically significant difference between these two approaches has been demonstrated. We investigated 4 classifiers (2 blackbox (Neural Networks & SVM), 1 Bayesian classifier & 1 (Decision-Tree-based) whitebox) and 4 domains, so a total of 16 Classifier-Dataset combinations were considered. In only 1 case (6.25%) was the result slightly worse; 5 cases (31.25%) were the same (these could not be improved), and the remaining 10 cases (62.5%) show significant improvements. In the future, we intend using similar approaches to improve the accuracy of the classification; this study focuses on fidelity.
Original languageEnglish
Title of host publicationK-CAP 2015 Proceedings of the 8th International Conference on Knowledge Capture
PublisherACM
ISBN (Electronic)978-1-4503-3849-3
DOIs
Publication statusPublished - 2015
EventK-CAP 2015 - The 8th International Conference on Knowledge Capture - USA, New York, United States
Duration: 7 Oct 201510 Oct 2015

Conference

ConferenceK-CAP 2015 - The 8th International Conference on Knowledge Capture
CountryUnited States
CityNew York
Period7/10/1510/10/15

    Fingerprint

Cite this

Cepukenas, J., Lin, C., & Sleeman, D. (2015). Applying Rule Extraction & Rule Refinement techniques to (Blackbox) Classifiers. In K-CAP 2015 Proceedings of the 8th International Conference on Knowledge Capture [27] ACM. https://doi.org/10.1145/2815833.2816950