arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Su-In Lee

73 papers · 2000–2026

2000: 12001: 02002: 02003: 12004: 02005: 02006: 22007: 22008: 02009: 02010: 02011: 02012: 12013: 32014: 32015: 22016: 52017: 52018: 42019: 52020: 82021: 52022: 62023: 62024: 72025: 22026: 5
2000peak 8 in 20202026

Frequent co-authors

Papers

showing 1–25 of 73 RSS

A Unified Approach to Interpreting Model Predictions

Scott Lundberg, Su-In Lee

cs.AI · · cited 42,561× · arXiv ↗ · more like this ·

Understanding why a model makes a certain prediction can be as crucial as the prediction's accuracy in many applications. However, the highest accuracy for large modern datasets is often achieved by complex models that even experts struggle to interpret, such as ensemble or deep learning models, creating a tension between accuracy and interpretability. In response, various methods have recently been proposed to help users interpret the predictions of complex models, but it is often unclear how these methods are related and when one method is preferable over another. To address this problem, we present a unified framework for interpreting predictions, SHAP (SHapley Additive exPlanations). SHAP assigns each feature an importance value for a particular prediction. Its novel components include: (1) the identification of a new class of additive feature importance measures, and (2)…

Algorithms to estimate Shapley value feature attributions

Hugh Chen, Ian C. Covert, Scott M. Lundberg +1 more

cs.LG · · cited 521× · arXiv ↗ · more like this ·

Feature attributions based on the Shapley value are popular for explaining machine learning models; however, their estimation is complex from both a theoretical and computational standpoint. We disentangle this complexity into two factors: (1)~the approach to removing feature information, and (2)~the tractable estimation strategy. These two factors provide a natural lens through which we can better understand and compare 24 distinct algorithms. Based on the various feature removal approaches, we describe the multiple types of Shapley value feature attributions and methods to calculate each one. Then, based on the tractable estimation strategies, we characterize two distinct families of approaches: model-agnostic and model-specific approximations. For the model-agnostic approximations, we benchmark a wide class of estimation approaches and tie them to alternative yet equivalent…

Explaining by Removing: A Unified Framework for Model Explanation

Ian Covert, Scott Lundberg, Su-In Lee

cs.LG · · cited 404× · arXiv ↗ · more like this ·

Researchers have proposed a wide variety of model explanation approaches, but it remains unclear how most methods are related or when one method is preferable to another. We describe a new unified class of methods, removal-based explanations, that are based on the principle of simulating feature removal to quantify each feature's influence. These methods vary in several respects, so we develop a framework that characterizes each method along three dimensions: 1) how the method removes features, 2) what model behavior the method explains, and 3) how the method summarizes each feature's influence. Our framework unifies 26 existing methods, including several of the most widely used approaches: SHAP, LIME, Meaningful Perturbations, and permutation tests. This newly understood class of explanation methods has rich connections that we examine using tools that have been largely overlooked by…

Explainable AI for Trees: From Local Explanations to Global Understanding

Scott M. Lundberg, Gabriel Erion, Hugh Chen +7 more

cs.LG · · cited 400× · arXiv ↗ · more like this ·

Tree-based machine learning models such as random forests, decision trees, and gradient boosted trees are the most popular non-linear predictive models used in practice today, yet comparatively little attention has been paid to explaining their predictions. Here we significantly improve the interpretability of tree-based models through three main contributions: 1) The first polynomial time algorithm to compute optimal explanations based on game theory. 2) A new type of explanation that directly measures local feature interaction effects. 3) A new set of tools for understanding global model structure based on combining many local explanations of each prediction. We apply these tools to three medical machine learning problems and show how combining many high-quality local explanations allows us to represent global structure while retaining local faithfulness to the original model. These…

Explaining a Series of Models by Propagating Shapley Values

Hugh Chen, Scott M. Lundberg, Su-In Lee

cs.LG · · cited 286× · arXiv ↗ · more like this ·

Local feature attribution methods are increasingly used to explain complex machine learning models. However, current methods are limited because they are extremely expensive to compute or are not capable of explaining a distributed series of models where each model is owned by a separate institution. The latter is particularly important because it often arises in finance where explanations are mandated. Here, we present DeepSHAP, a tractable method to propagate local feature attributions through complex series of models based on a connection to the Shapley value. We evaluate DeepSHAP across biological, health, and financial datasets to show that it provides equally salient explanations an order of magnitude faster than existing model-agnostic attribution techniques and demonstrate its use in an important distributed series of models setting.

True to the Model or True to the Data?

Hugh Chen, Joseph D. Janizek, Scott Lundberg +1 more

cs.LG · · cited 218× · arXiv ↗ · more like this ·

A variety of recent papers discuss the application of Shapley values, a concept for explaining coalitional games, for feature attribution in machine learning. However, the correct way to connect a machine learning model to a coalitional game has been a source of controversy. The two main approaches that have been proposed differ in the way that they condition on known features, using either (1) an interventional or (2) an observational conditional expectation. While previous work has argued that one of the two approaches is preferable in general, we argue that the choice is application dependent. Furthermore, we argue that the choice comes down to whether it is desirable to be true to the model or true to the data. We use linear models to investigate this choice. After deriving an efficient method for calculating observational conditional expectation Shapley values for linear models, we…

Node-Based Learning of Multiple Gaussian Graphical Models

Karthik Mohan, Palma London, Maryam Fazel +2 more

stat.ML · · cited 217× · arXiv ↗ · more like this ·

We consider the problem of estimating high-dimensional Gaussian graphical models corresponding to a single set of variables under several distinct conditions. This problem is motivated by the task of recovering transcriptional regulatory networks on the basis of gene expression data {containing heterogeneous samples, such as different disease states, multiple species, or different developmental stages}. We assume that most aspects of the conditional dependence networks are shared, but that there are some structured differences between them. Rather than assuming that similarities and differences between networks are driven by individual edges, we take a node-based approach, which in many cases provides a more intuitive interpretation of the network differences. We consider estimation under two distinct assumptions: (1) differences between the K networks are due to individual nodes that…

Efficient Structure Learning of Markov Networks using L1-Regularization

Su-In Lee, Varun Ganapathi, Daphne Koller

Artificial Intelligence · The MIT Press eBooks · · cited 210× · DOI ↗ · more like this ·

Markov networks are commonly used in a wide variety of applications, ranging from computer vision, to natural language, to computational biology. In most current applications, even those that rely heavily on learned models, the structure of the Markov network is constructed by hand, due to the lack of effective algorithms for learning Markov network structure from data. In this paper, we provide a computationally efficient method for learning Markov network structure from data. Our method is based on the use of L1 regularization on the weights of the log-linear model, which has the effect of biasing the model towards solutions where many of the parameters are zero. This formulation converts the Markov network learning problem into a convex optimization problem in a continuous space, which can be solved using efficient gradient methods. A key issue in this setting is the (unavoidable)…

Explaining Explanations: Axiomatic Feature Interactions for Deep Networks

Joseph D. Janizek, Pascal Sturmfels, Su-In Lee

cs.LG · · cited 204× · arXiv ↗ · more like this ·

Recent work has shown great promise in explaining neural network behavior. In particular, feature attribution methods explain which features were most important to a model's prediction on a given input. However, for many tasks, simply knowing which features were important to a model's prediction may not provide enough insight to understand model behavior. The interactions between features within the model may better help us understand not only the model, but also why certain features are more important than others. In this work, we present Integrated Hessians, an extension of Integrated Gradients that explains pairwise feature interactions in neural networks. Integrated Hessians overcomes several theoretical limitations of previous methods to explain interactions, and unlike such previous methods is not limited to a specific architecture or class of neural network. Additionally, we find…

Consistent feature attribution for tree ensembles

Scott M. Lundberg, Su-In Lee

cs.AI · · cited 185× · arXiv ↗ · more like this ·

Note that a newer expanded version of this paper is now available at: arXiv:1802.03888 It is critical in many applications to understand what features are important for a model, and why individual predictions were made. For tree ensemble methods these questions are usually answered by attributing importance values to input features, either globally or for a single prediction. Here we show that current feature attribution methods are inconsistent, which means changing the model to rely more on a given feature can actually decrease the importance assigned to that feature. To address this problem we develop fast exact solutions for SHAP (SHapley Additive exPlanation) values, which were recently shown to be the unique additive feature attribution method based on conditional expectations that is both consistent and locally accurate. We integrate these improvements into the latest version of…

Learning a meta-level prior for feature relevance from multiple related tasks

Su-In Lee, Vassil Chatalbashev, David Vickrey +1 more

Artificial Intelligence · · cited 165× · DOI ↗ · more like this ·

In many prediction tasks, selecting relevant features is essential for achieving good generalization performance. Most feature selection algorithms consider all features to be a priori equally likely to be relevant. In this paper, we use transfer learning---learning on an ensemble of related tasks---to construct an informative prior on feature relevance. We assume that features themselves have meta-features that are predictive of their relevance to the prediction task, and model their relevance as a function of the meta-features using hyperparameters (called meta-priors). We present a convex optimization algorithm for simultaneously learning the meta-priors and feature weights from an ensemble of related prediction tasks which share a similar relevance structure. Our approach transfers the "meta-priors" among different tasks, which makes it possible to deal with settings where tasks…

An unexpected unity among methods for interpreting model predictions

Scott Lundberg, Su-In Lee

cs.AI · · cited 120× · arXiv ↗ · more like this ·

Understanding why a model made a certain prediction is crucial in many data science fields. Interpretable predictions engender appropriate trust and provide insight into how the model may be improved. However, with large modern datasets the best accuracy is often achieved by complex models even experts struggle to interpret, which creates a tension between accuracy and interpretability. Recently, several methods have been proposed for interpreting predictions from complex models by estimating the importance of input features. Here, we present how a model-agnostic additive representation of the importance of input features unifies current methods. This representation is optimal, in the sense that it is the only set of additive values that satisfies important properties. We show how we can leverage these properties to create novel visual explanations of model predictions. The thread of…

Learning Graphical Models With Hubs

Kean Ming Tan, Palma London, Karthik Mohan +3 more

stat.ML · · cited 106× · arXiv ↗ · more like this ·

We consider the problem of learning a high-dimensional graphical model in which certain hub nodes are highly-connected to many other nodes. Many authors have studied the use of an l1 penalty in order to learn a sparse graph in high-dimensional setting. However, the l1 penalty implicitly assumes that each edge is equally likely and independent of all other edges. We propose a general framework to accommodate more realistic networks with hub nodes, using a convex formulation that involves a row-column overlap norm penalty. We apply this general framework to three widely-used probabilistic graphical models: the Gaussian graphical model, the covariance graph model, and the binary Ising model. An alternating direction method of multipliers algorithm is used to solve the corresponding convex optimization problems. On synthetic data, we demonstrate that our proposed framework outperforms…

Interpretable machine learning prediction of all-cause mortality

Wei Qiu, Hugh Chen, Ayse B. Dincer +3 more

Artificial Intelligence · medRxiv · · cited 103× · DOI ↗ · more like this ·

Abstract Prior studies on all-cause mortality traditionally use linear models; however, growing field of explainable artificial intelligence (XAI) can improve prediction accuracy over traditional linear models using complex machine learning (ML) models while still revealing novel insights. We propose the IMPACT (Interpretable Machine learning Prediction of All-Cause morTality) framework that implements and explains complex, non-linear ML models by combining a tree ensemble mortality prediction model and a principled XAI technique. We apply IMPACT to the NHANES (1999-2014) dataset, which enables us to understand different subpopulations according to shorter or longer term mortality and younger and older individuals. Our IMPACT models have higher predictive accuracy than popular pre-existing mortality risk scores and biological ages. Using individualized feature importance scores, we…

Transparent medical image AI via an image–text foundation model grounded in medical literature

Chanwoo Kim, Soham Gadgil, Alex J. DeGrave +4 more

Artificial Intelligence · Nature Medicine · · cited 87× · DOI ↗ · more like this ·

Building trustworthy and transparent image-based medical artificial intelligence (AI) systems requires the ability to interrogate data and models at all stages of the development pipeline, from training models to post-deployment monitoring. Ideally, the data and associated AI systems could be described using terms already familiar to physicians, but this requires medical datasets densely annotated with semantically meaningful concepts. In the present study, we present a foundation model approach, named MONET (medical concept retriever), which learns how to connect medical images with text and densely scores images on concept presence to enable important tasks in medical AI development and deployment such as data auditing, model auditing and model interpretation. Dermatology provides a demanding use case for the versatility of MONET, due to the heterogeneity in diseases, skin tones and…

Checkpoint Ensembles: Ensemble Methods from a Single Training Process

Hugh Chen, Scott Lundberg, Su-In Lee

cs.LG · · cited 74× · arXiv ↗ · more like this ·

We present the checkpoint ensembles method that can learn ensemble models on a single training process. Although checkpoint ensembles can be applied to any parametric iterative learning technique, here we focus on neural networks. Neural networks' composable and simple neurons make it possible to capture many individual and interaction effects among features. However, small sample sizes and sampling noise may result in patterns in the training data that are not representative of the true relationship between the features and the outcome. As a solution, regularization during training is often used (e.g. dropout). However, regularization is no panacea -- it does not perfectly address overfitting. Even with methods like dropout, two methodologies are commonly used in practice. First is to utilize a validation set independent to the training set as a way to decide when to stop training.…

Learning to Estimate Shapley Values with Vision Transformers

Ian Covert, Chanwoo Kim, Su-In Lee

cs.CV · · cited 67× · arXiv ↗ · more like this ·

Transformers have become a default architecture in computer vision, but understanding what drives their predictions remains a challenging problem. Current explanation approaches rely on attention values or input gradients, but these provide a limited view of a model's dependencies. Shapley values offer a theoretically sound alternative, but their computational cost makes them impractical for large, high-dimensional models. In this work, we aim to make Shapley values practical for vision transformers (ViTs). To do so, we first leverage an attention masking approach to evaluate ViTs with partial information, and we then develop a procedure to generate Shapley value explanations via a separate, learned explainer model. Our experiments compare Shapley values to many baseline methods (e.g., attention rollout, GradCAM, LRP), and we find that our approach provides more accurate explanations…

Explainable machine learning prediction of synergistic drug combinations for precision cancer medicine

Joseph D. Janizek, Safiye Çelik, Su‐In Lee

Computational Theory and Mathematics · bioRxiv (Cold Spring Harbor Laboratory) · · cited 62× · DOI ↗ · more like this ·

Abstract Although combination therapy has been a mainstay of cancer treatment for decades, it remains challenging, both to identify novel effective combinations of drugs and to determine the optimal combination for a particular patient’s tumor. While there have been several recent efforts to test drug combinations in vitro , examining the immense space of possible combinations is far from being feasible. Thus, it is crucial to develop datadriven techniques to computationally identify the optimal drug combination for a patient. We introduce TreeCombo, an extreme gradient boosted tree-based approach to predict synergy of novel drug combinations, using chemical and physical properties of drugs and gene expression levels of cell lines as features. We find that TreeCombo significantly outperforms three other state-of-theart approaches, including the recently developed DeepSynergy, which uses…

Learning Explainable Models Using Attribution Priors

Gabriel Erion, Joseph D. Janizek, Pascal Sturmfels +2 more

Artificial Intelligence · · cited 57× · OpenAlex ↗ · more like this ·

Two important topics in deep learning both involve incorporating humans into the modeling process: Model priors transfer information from humans to a model by constraining the model's parameters; Model attributions transfer information from a model to humans by explaining the model's behavior. We propose connecting these topics with attribution priors (this https URL), which allow humans to use the common language of attributions to enforce prior expectations about a model's behavior during training. We develop a differentiable axiomatic feature attribution method called expected gradients and show how to directly regularize these attributions during training. We demonstrate the broad applicability of attribution priors ($\Omega$) by presenting three distinct examples that regularize models to behave more intuitively in three different domains: 1) on image data…

Feature Removal Is a Unifying Principle for Model Explanation Methods

Ian Covert, Scott Lundberg, Su-In Lee

cs.LG · · cited 45× · arXiv ↗ · more like this ·

Researchers have proposed a wide variety of model explanation approaches, but it remains unclear how most methods are related or when one method is preferable to another. We examine the literature and find that many methods are based on a shared principle of explaining by removing - essentially, measuring the impact of removing sets of features from a model. These methods vary in several respects, so we develop a framework for removal-based explanations that characterizes each method along three dimensions: 1) how the method removes features, 2) what model behavior the method explains, and 3) how the method summarizes each feature's influence. Our framework unifies 26 existing methods, including several of the most widely used approaches (SHAP, LIME, Meaningful Perturbations, permutation tests). Exposing the fundamental similarities between these methods empowers users to reason about…

Understanding Global Feature Contributions With Additive Importance Measures

Ian Covert, Scott Lundberg, Su-In Lee

cs.LG · · cited 26× · arXiv ↗ · more like this ·

Understanding the inner workings of complex machine learning models is a long-standing problem and most recent research has focused on local interpretability. To assess the role of individual input features in a global sense, we explore the perspective of defining feature importance through the predictive power associated with each feature. We introduce two notions of predictive power (model-based and universal) and formalize this approach with a framework of additive importance measures, which unifies numerous methods in the literature. We then propose SAGE, a model-agnostic method that quantifies predictive power while accounting for feature interactions. Our experiments show that SAGE can be calculated efficiently and that it assigns more accurate importance values than other methods.

Pathway Graphical Lasso

Maxim Grechkin, Maryam Fazel, Daniela Witten +1 more

Statistics and Probability · Proceedings of the AAAI Conference on Artificial Intelligence · · cited 24× · DOI ↗ · more like this ·

Graphical models provide a rich framework for summarizing the dependencies among variables. The graphical lasso approach attempts to learn the structure of a Gaussian graphical model (GGM) by maximizing the log likelihood of the data, subject to an l1 penalty on the elements of the inverse covariance matrix. Most algorithms for solving the graphical lasso problem do not scale to a very large number of variables. Furthermore, the learned network structure is hard to interpret. To overcome these challenges, we propose a novel GGM structure learning method that exploits the fact that for many real-world problems we have prior knowledge that certain edges are unlikely to be present. For example, in gene regulatory networks, a pair of genes that does not participate together in any of the cellular processes, typically referred to as pathways, is less likely to be connected. In computer…

Efficient Dimensionality Reduction for High-Dimensional Network Estimation

Safiye Çelik, Benjamin A. Logsdon, Su‐In Lee

Statistics and Probability · International Conference on Machine Learning · · cited 22× · OpenAlex ↗ · more like this ·

We propose module graphical lasso (MGL), an aggressive dimensionality reduction and network estimation technique for a high-dimensional Gaussian graphical model (GGM). MGL achieves scalability, interpretability and robustness by exploiting the modularity property of many real-world networks. Variables are organized into tightly coupled modules and a graph structure is estimated to determine the conditional independencies among modules. MGL iteratively learns the module assignment of variables, the latent variables, each corresponding to a module, and the parameters of the GGM of the latent variables. In synthetic data experiments, MGL outperforms the standard graphical lasso and three other methods that incorporate latent variables into GGMs. When applied to gene expression data from ovarian cancer, MGL outperforms standard clustering algorithms in identifying functionally coherent gene…

Associations Between Genetic Data and Quantitative Assessment of Normal Facial Asymmetry

Sara Rolfe, Su‐In Lee, Linda G. Shapiro

Geometry and Topology · Frontiers in Genetics · · cited 20× · DOI ↗ · more like this ·

Human facial asymmetry is due to a complex interaction of genetic and environmental factors. To identify genetic influences on facial asymmetry we developed a method for automated scoring that summarizes local morphology features and their spatial distribution. A genome-wide association study using asymmetry scores from two local symmetry features was conducted and significant genetic associations were identified for one asymmetry feature, including genes thought to play a role in craniofacial disorders and development: NFATC1, SOX5, NBAS, and TCF7L1. These results provide evidence that normal variation in facial asymmetry may be impacted by common genetic variants and further motivate the development of automated summaries of complex phenotypes.