Developing the sensitivity of LIME for better machine learning explanation
Eun‐Jin Lee, Dave Braines, Mitchell Stiffler +2 more
Machine learning systems can provide outstanding results, but their black-box nature means that it’s hard to understand how the conclusion has been reached. Understanding how the results are determined is especially important in military and security contexts due to the importance of the decisions that may be made as a result. In this work, the reliability of LIME (Local Interpretable Model Agnostic Explanations), a method of interpretability, was analyzed and developed. A simple Convolutional Neural Network (CNN) model was trained using two classes of images of “gun-wielder” and “non-wielder". The sensitivity of LIME improved when multiple output weights for individual images were averaged and visualized. The resultant averaged images were compared to the individual images to analyze the variability and reliability of the two LIME methods. Without techniques such as those explored in…