DeepCompare: Visual and Interactive Comparison of Deep Learning Model Performance

作     者：Murugesan, Sugeerth Malik, Sana Du, Fan Koh, Eunyee Tuan Manh Lai 

作者机构：Univ Calif Davis Davis CA 95616 USA Adobe Res Cambridge MA USA Purdue Univ W Lafayette IN 47907 USA 

出 版 物：《IEEE COMPUTER GRAPHICS AND APPLICATIONS》 (IEEE Comput Graphics Appl)

年 卷 期：2019年第39卷第5期

页      面：47-59页

核心收录：

学科分类：0808[工学-电气工程] 08[工学] 0835[工学-软件工程] 

主　　题：Adaptation models Analytical models Computational modeling Computer architecture data visualization Deep learning human-centered computing machine learning Task analysis visual analytics 

摘      要：Deep learning models have become the state-of-the-art for many tasks, from text sentiment analysis to facial image recognition. However, understanding why certain models perform better than others or how one model learns differently than another is often difficult yet critical for increasing their effectiveness, improving prediction accuracy, and enabling fairness. Traditional methods for comparing models efficacy, such as accuracy, precision, and recall provide a quantitative view of performance;however, the qualitative intricacies of why one model performs better than another are hidden. In this paper, we interview machine learning practitioners to understand their evaluation and comparison workflow. From there, we iteratively design a visual analytic approach, DeepCompare, to systematically compare the results of deep learning models, in order to provide insight into the model behavior and interactively assess tradeoffs between two such models. The tool allows users to evaluate model results, identify and compare activation patterns form is classifications, and link the test results back to specific neurons. We conduct a preliminary evaluation through two real-world case studies to show that experts can make more informed decisions about the effectiveness of different types of models, understand in more detail the strengths and weaknesses of the models, and holistically evaluate the behavior of the models.