arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Jacob Hinkle

38 papers · 2008–2024

2008: 12009: 02010: 02011: 02012: 12013: 12014: 22015: 02016: 32017: 12018: 32019: 62020: 82021: 32022: 62023: 12024: 2
2008peak 8 in 20202024

Frequent co-authors

Papers

showing 1–25 of 38 RSS

Intrinsic Polynomials for Regression on Riemannian Manifolds

Jacob Hinkle, P. Thomas Fletcher, Sarang Joshi

Geometry and Topology · Journal of Mathematical Imaging and Vision · · cited 89× · DOI ↗ · more like this ·

We develop a framework for polynomial regression on Riemannian manifolds. Unlike recently developed spline models on Riemannian manifolds, Riemannian polynomials offer the ability to model parametric polynomials of all integer orders, odd and even. An intrinsic adjoint method is employed to compute variations of the matching functional, and polynomial regression is accomplished using a gradient-based optimization scheme. We apply our polynomial regression framework in the context of shape analysis in Kendall shape space as well as in diffeomorphic landmark space. Our algorithm is shown to be particularly convenient in Riemannian manifolds with additional symmetry, such as Lie groups and homogeneous spaces with right or left invariant metrics. As a particularly important example, we also apply polynomial regression to time-series imaging data using a right invariant Sobolev metric on the…

Challenges in Markov chain Monte Carlo for Bayesian neural networks

Theodore Papamarkou, Jacob Hinkle, M. Todd Young +1 more

stat.ML · · cited 78× · arXiv ↗ · more like this ·

Markov chain Monte Carlo (MCMC) methods have not been broadly adopted in Bayesian neural networks (BNNs). This paper initially reviews the main challenges in sampling from the parameter posterior of a neural network via MCMC. Such challenges culminate to lack of convergence to the parameter posterior. Nevertheless, this paper shows that a non-converged Markov chain, generated via MCMC sampling from the parameter space of a neural network, can yield via Bayesian marginalization a valuable posterior predictive distribution of the output of the neural network. Classification examples based on multilayer perceptrons showcase highly accurate posterior predictive distributions. The postulate of limited scope for MCMC developments in BNNs is partially valid; an asymptotically exact parameter posterior seems less plausible, yet an accurate posterior predictive distribution is a tenable research…

A vector momenta formulation of diffeomorphisms for improved geodesic regression and atlas construction

Nikhil Singh, Jacob Hinkle, Sarang Joshi +1 more

Computer Vision and Pattern Recognition · PubMed · · cited 72× · DOI ↗ · more like this ·

This paper presents a novel approach for diffeomorphic image regression and atlas estimation that results in improved convergence and numerical stability. We use a vector momenta representation of a diffeomorphism's initial conditions instead of the standard scalar momentum that is typically used. The corresponding variational problem results in a closed-form update for template estimation in both the geodesic regression and atlas estimation problems. While we show that the theoretical optimal solution is equivalent to the scalar momenta case, the simplification of the optimization problem leads to more stable and efficient estimation in practice. We demonstrate the effectiveness of our method for atlas estimation and geodesic regression using synthetically generated shapes and 3D MRI brain scans.

Classifying cancer pathology reports with hierarchical self-attention networks

Shang Gao, John X. Qiu, Mohammed Alawad +9 more

Artificial Intelligence · Artificial Intelligence in Medicine · · cited 59× · DOI ↗ · more like this ·

We introduce a deep learning architecture, hierarchical self-attention networks (HiSANs), designed for classifying pathology reports and show how its unique architecture leads to a new state-of-the-art in accuracy, faster training, and clear interpretability. We evaluate performance on a corpus of 374,899 pathology reports obtained from the National Cancer Institute's (NCI) Surveillance, Epidemiology, and End Results (SEER) program. Each pathology report is associated with five clinical classification tasks - site, laterality, behavior, histology, and grade. We compare the performance of the HiSAN against other machine learning and deep learning approaches commonly used on medical text data - Naive Bayes, logistic regression, convolutional neural networks, and hierarchical attention networks (the previous state-of-the-art). We show that HiSANs are superior to other machine learning and…

Distributed Bayesian optimization of deep reinforcement learning algorithms

M. Todd Young, Jacob Hinkle, Ramakrishnan Kannan +1 more

Computational Theory and Mathematics · Journal of Parallel and Distributed Computing · · cited 41× · DOI ↗ · more like this ·

Significant strides have been made in supervised learning settings thanks to the successful application of deep learning. Now, recent work has brought the techniques of deep learning to bear on sequential decision processes in the area of deep reinforcement learning (DRL). Currently, little is known regarding hyperparameter optimization for DRL algorithms. Given that DRL algorithms are computationally intensive to train, and are known to be sample inefficient, optimizing model hyperparameters for DRL presents significant challenges to established techniques. We provide an open source, distributed Bayesian model-based optimization algorithm, HyperSpace, and show that it consistently outperforms standard hyperparameter optimization techniques across three DRL algorithms.

HyperSpace: Distributed Bayesian Hyperparameter Optimization

M. Todd Young, Jacob Hinkle, Arvind Ramanathan +1 more

Artificial Intelligence · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · · cited 19× · DOI ↗ · more like this ·

As machine learning models continue to increase in complexity, so does the potential number of free model parameters commonly known as hyperparameters. While there has been considerable progress toward finding optimal configurations of these hyperparameters, many optimization procedures are treated as black boxes. We believe optimization methods should not only return a set of optimized hyperparameters, but also give insight into the effects of model hyperparameter settings. To this end, we present HyperSpace, a parallel implementation of Bayesian sequential model-based optimization. HyperSpace leverages high performance computing (HPC) resources to better understand unknown, potentially non-convex hyperparameter search spaces. We show that it is possible to learn the dependencies between model hyperparameters through the optimization process. By partitioning large search spaces and…

Deep Transfer Learning Across Cancer Registries for Information Extraction from Pathology Reports

Mohammed Alawad, Shang Gao, John X. Qiu +9 more

Artificial Intelligence · PubMed · · cited 18× · DOI ↗ · more like this ·

Automated text information extraction from cancer pathology reports is an active area of research to support national cancer surveillance. A well-known challenge is how to develop information extraction tools with robust performance across cancer registries. In this study we investigated whether transfer learning (TL) with a convolutional neural network (CNN) can facilitate cross-registry knowledge sharing. Specifically, we performed a series of experiments to determine whether a CNN trained with single-registry data is capable of transferring knowledge to another registry or whether developing a cross-registry knowledge database produces a more effective and generalizable model. Using data from two cancer registries and primary tumor site and topography as the information extraction task of interest, our study showed that TL results in 6.90% and 17.22% improvement of classification…

Model Assumptions and Data Characteristics: Impacts on Domain Adaptation in Building Segmentation

Philipe Dias, Yuxin Tian, Shawn Newsam +3 more

Artificial Intelligence · IEEE Transactions on Geoscience and Remote Sensing · · cited 10× · DOI ↗ · more like this ·

Studies on domain adaptation (DA) for remote sensing (RS) imagery analysis lack consistency in selection and description of evaluation scenarios. Without properly characterizing datasets, model assumptions, and evaluation scenarios, it is difficult to objectively compare DA methods and reach conclusions about their suitability across different applications. With this motivation, this work seeks to empirically assess to which extent the interaction between data characteristics and model assumptions influence the effectiveness of DA methods. Using the widely explored task of building footprint segmentation as case study, we perform a large-scale study across over 200 domain adaptation scenarios that include variations across view angles, areas observed, and sensors used for data acquisition. Rather than adopting different model architectures or optimization criteria, we contrast the…

Polynomial Regression on Riemannian Manifolds

Jacob Hinkle, Prasanna Muralidharan, P. Thomas Fletcher +1 more

math.ST · · cited 9× · arXiv ↗ · more like this ·

In this paper we develop the theory of parametric polynomial regression in Riemannian manifolds and Lie groups. We show application of Riemannian polynomial regression to shape analysis in Kendall shape space. Results are presented, showing the power of polynomial regression on the classic rat skull growth data of Bookstein as well as the analysis of the shape changes associated with aging of the corpus callosum from the OASIS Alzheimer's study.

Wide Neural Networks with Bottlenecks are Deep Gaussian Processes

Devanshu Agrawal, Theodore Papamarkou, Jacob Hinkle

stat.ML · · cited 6× · arXiv ↗ · more like this ·

There has recently been much work on the "wide limit" of neural networks, where Bayesian neural networks (BNNs) are shown to converge to a Gaussian process (GP) as all hidden layers are sent to infinite width. However, these results do not apply to architectures that require one or more of the hidden layers to remain narrow. In this paper, we consider the wide limit of BNNs where some hidden layers, called "bottlenecks", are held at finite width. The result is a composition of GPs that we term a "bottleneck neural network Gaussian process" (bottleneck NNGP). Although intuitive, the subtlety of the proof is in showing that the wide limit of a composition of networks is in fact the composition of the limiting GPs. We also analyze theoretically a single-bottleneck NNGP, finding that the bottleneck induces dependence between the outputs of a multi-output network that persists through…

Toward Large-Scale Image Segmentation on Summit

Sudip K. Seal, Seung–Hwan Lim, Dali Wang +3 more

Computer Vision and Pattern Recognition · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · · cited 5× · DOI ↗ · more like this ·

Semantic segmentation of images is an important computer vision task that emerges in a variety of application domains such as medical imaging, robotic vision and autonomous vehicles to name a few. While these domain-specific image analysis tasks involve relatively small image sizes (∼ 102 × 102), there are many applications that need to train machine learning models on image data with extents that are orders of magnitude larger (∼ 104 × 104). Training deep neural network (DNN) models on large extent images is extremely memory-intensive and often exceeds the memory limitations of a single graphical processing unit, a hardware accelerator of choice for computer vision workloads. Here, an efficient, sample parallel approach to train U-Net models on large extent image data sets is presented. Its advantages and limitations are analyzed and near-linear strong-scaling speedup demonstrated on…

Extraction of Tumor Site from Cancer Pathology Reports using Deep Filters

Abhishek Dubey, Jacob Hinkle, J. Blair Christian +1 more

Artificial Intelligence · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · · cited 5× · DOI ↗ · more like this ·

\beginabstract Purpose: Pathology reports are the primary source of information concerning the millions of cancer cases across the United States. % Cancer registries manually process the pathology reports to extract the pertinent information including primary tumor site, behavior, histology, laterality, and grade. % Processing a large volume of the pathology reports in a timely manner is a continuing challenge for cancer registries. % The purpose of this study is to develop an information extraction pipeline to reliably and efficiently extract reportable information.\\ Method: % We have developed a novel inverse-regression (IR) based information extraction pipeline. % The inverse-regression based supervised filter has been successfully applied to many application domains. % However, its application to the information extraction from unstructured text is hindered primarily by the extreme…

An efficient parallel algorithm for hierarchical geodesic models in diffeomorphisms

Nikhil Singh, Jacob Hinkle, Sarang Joshi +1 more

Geometry and Topology · · cited 3× · DOI ↗ · more like this ·

We present a novel algorithm for computing hierarchical geodesic models (HGMs) for diffeomorphic longitudinal shape analysis. The proposed algorithm exploits the inherent parallelism arising out of the independence in the contributions of individual geodesics to the group geodesic. The previous serial implementation severely limits the use of HGMs to very small population sizes due to computation time and massive memory requirements. The conventional method makes it impossible to estimate the parameters of HGMs on large datasets due to limited memory available onboard current GPU computing devices. The proposed parallel algorithm easily scales to solve HGMs on a large collection of 3D images of several individuals. We demonstrate its effectiveness on longitudinal datasets of synthetically generated shapes and 3D magnetic resonance brain images (MRI).

Scaling Resolution of Gigapixel Whole Slide Images Using Spatial Decomposition on Convolutional Neural Networks

Aristeidis Tsaris, Joshua Romero, Thorsten Kurth +5 more

Computer Vision and Pattern Recognition · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · · cited 2× · DOI ↗ · more like this ·

Gigapixel images are prevalent in scientific domains ranging from remote sensing, and satellite imagery to microscopy, etc. However, training a deep learning model at the natural resolution of those images has been a challenge in terms of both, overcoming the resource limit (e.g. HBM memory constraints), as well as scaling up to a large number of GPUs. In this paper, we trained Residual neural Networks (ResNet) on 22,528 x 22,528-pixel size images using a distributed spatial decomposition method on 2,304 GPUs on the Summit Supercomputer. We applied our method on a Whole Slide Imaging (WSI) dataset from The Cancer Genome Atlas (TCGA) database. WSI images can be in the size of 100,000 x 100,000 pixels or even larger, and in this work we studied the effect of image resolution on a classification task, while achieving state-of-the-art AUC scores. Moreover, our approach doesn't need…

Distributed Training for High Resolution Images: A Domain and Spatial Decomposition Approach

Aristeidis Tsaris, Jacob Hinkle, Dalton Lunga +1 more

Computer Vision and Pattern Recognition · · cited 2× · DOI ↗ · more like this ·

In this work we developed two Pytorch libraries using the PyTorch RPC interface for distributed deep learning approaches on high resolution images. The spatial decomposition library allows for distributed training on very large images, which otherwise wouldn’t be possible on a single GPU. The domain parallelism library allows for distributed training across multiple domain unlabeled data, by leveraging the domain separation architecture. Both of those libraries where tested on the Summit supercomputer at Oak Ridge National Laboratory at a moderate scale.

VISION: Toward a Standardized Process for Radiology Image Management at the National Level

Kathryn Knight, Ioana Danciu, Olga Ovchinnikova +14 more

cs.CV · · cited 1× · arXiv ↗ · more like this ·

The compilation and analysis of radiological images poses numerous challenges for researchers. The sheer volume of data as well as the computational needs of algorithms capable of operating on images are extensive. Additionally, the assembly of these images alone is difficult, as these exams may differ widely in terms of clinical context, structured annotation available for model training, modality, and patient identifiers. In this paper, we describe our experiences and challenges in establishing a trusted collection of radiology images linked to the United States Department of Veterans Affairs (VA) electronic health record database. We also discuss implications in making this repository research-ready for medical investigators. Key insights include uncovering the specific procedures required for transferring images from a clinical to a research-ready environment, as well as roadblocks…

Distilling Knowledge from Ensembles of Cluster-Constrained-Attention Multiple-Instance Learners for Whole Slide Image Classification

Folami Alamudun, Jacob Hinkle, Sajal Dash +3 more

Artificial Intelligence · 2022 IEEE International Conference on Big Data (Big Data) · · cited 0× · DOI ↗ · more like this ·

The peculiar nature of whole slide imaging (WSI), digitizing conventional glass slides to obtain multiple high resolution images which capture microscopic details of a patient’s histopathological features, has garnered increased interest from the computer vision research community over the last two decades. Given the unique computational space and time complexity inherent to gigapixel-size whole slide image data, researchers have proposed novel machine learning algorithms to aid in the performance of diagnostic tasks in clinical pathology. One effective algorithm represents a Whole slide image as a bag of smaller image patches, which can be represented as low-dimension image patch embeddings. Weakly supervised deep-learning methods, such as cluster-constrained-attention multiple instance learning (CLAM), have shown promising results when combined with image patch embeddings. While…

Lossless Image Compression for 4D-STEM Datasets

Jacob Hinkle, Debangshu Mukherjee, Kevin M. Roccapriore +8 more

Artificial Intelligence · Microscopy and Microanalysis · · cited 0× · DOI ↗ · more like this ·

Recent progress in scanning transmission electron microscopy has enabled the collection of fourdimensional datasets (4D-STEM) that are incredibly rich in information at atomistic levels of detail. These datasets typically occupy 20-50 GiB on disk. Montaging and related automated acquisition methods promise to increase these data sizes many-fold in the near term. As data collection has advanced, so has our statistical modeling and computational capacity. Electron ptychography via iterative optimization is a computationally intensive method that uses 4D-STEM datasets to reconstruct 2D and 3D images with sufficient resolution to resolve individual atoms. Modern approaches to ptychography use high-performance computing with distributed computation to compute these image reconstructions in an acceptable timeframe. This necessitates transferring of tens to hundreds of gigabytes of data…

Deep Ensemble Kernel Learning

Devanshu Agrawal, Jacob Hinkle

Artificial Intelligence · · cited 0× · OpenAlex ↗ · more like this ·

Gaussian processes (GPs) are nonparametric Bayesian models that are both flexible and robust to overfitting. One of the main challenges of GP methods is selecting the kernel. In the deep kernel learning (DKL) paradigm, a deep neural network or ``feature network'' is used to map inputs into a latent feature space, where a GP with a ``base kernel'' acts; the resulting model is then trained in an end-to-end fashion. In this work, we introduce the ``deep ensemble kernel learning'' (DEKL) model, which is a special case of DKL. In DEKL, a linear base kernel is used, enabling exact optimization of the base kernel hyperparameters and a scalable inference method that does not require approximation by inducing points. We also represent the feature network as a concatenation of an ensemble of learner networks with a common architecture, allowing for easy model parallelism. We show that DEKL is…

Model Reduction of Shallow CNN Model for Reliable Deployment of Information Extraction from Medical Reports

Abhishek K Dubey, Alina Peluso, Jacob Hinkle +2 more

cs.CL · · cited 0× · arXiv ↗ · more like this ·

Shallow Convolution Neural Network (CNN) is a time-tested tool for the information extraction from cancer pathology reports. Shallow CNN performs competitively on this task to other deep learning models including BERT, which holds the state-of-the-art for many NLP tasks. The main insight behind this eccentric phenomenon is that the information extraction from cancer pathology reports require only a small number of domain-specific text segments to perform the task, thus making the most of the texts and contexts excessive for the task. Shallow CNN model is well-suited to identify these key short text segments from the labeled training set; however, the identified text segments remain obscure to humans. In this study, we fill this gap by developing a model reduction tool to make a reliable connection between CNN filters and relevant text segments by discarding the spurious connections. We…

Hierarchical Geodesic Models in Diffeomorphisms

Sarang Joshi, Nikhil Singh, P. Thomas Fletcher +1 more

Mathematical Physics · UNC Libraries · · cited 0× · DOI ↗ · more like this ·

Hierarchical linear models (HLMs) are a standard approach for analyzing data where individuals are measured repeatedly over time. However, such models are only applicable to longitudinal studies of Euclidean data. This paper develops the theory of hierarchical geodesic models (HGMs), which generalize HLMs to the manifold setting. Our proposed model quantifies longitudinal trends in shapes as a hierarchy of geodesics in the group of diffeomorphisms. First, individual-level geodesics represent the trajectory of shape changes within individuals. Second, a group-level geodesic represents the average trajectory of shape changes for the population. Our proposed HGM is applicable to longitudinal data from unbalanced designs, i.e., varying numbers of timepoints for individuals, which is typical in medical studies. We derive the solution of HGMs on diffeomorphisms to estimate individual-level…

Computer-aided detection using non-convolutional neural network Gaussian processes

Devanshu Agrawal, Hong‐Jun Yoon, Georgia D. Tourassi +1 more

Artificial Intelligence · Medical Imaging 2019: Computer-Aided Diagnosis · · cited 0× · DOI ↗ · more like this ·

Deep convolutional neural networks (CNNs) have in recent years achieved record-breaking performance on many image classification tasks and are therefore well-suited for computer aided detection (CAD). The need for uncertainty quantification for CAD motivates the need for a probabilistic framework for deep learning. The most well-known probabilistic neural network model is the Bayesian neural network (BNN), but BNNs are notoriously difficult to sample for large complex network architectures, and as such their use is restricted to small problems. It is known that the limit of BNNs as their widths increase toward infinity is a Gaussian process (GP), and there has been considerable research interest in these infinitely wide BNNs. Recently, this classic result has been extended to deep architectures in what is termed the neural network Gaussian process (NNGP) model. In this work, we…

DeepAnnotator

James Blair Christian, Mohammed W. Alawad, Jacob Hinkle +5 more

Artificial Intelligence · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · · cited 0× · DOI ↗ · more like this ·

Given a collection of pairs ((A) written material such as reports, phrases, bulleted lists, etc., (B) structured labels called annotations) called the training data, we want to create software to take future written material (A) and correctly predict (B). In this case we have used medical reports and medical coder's structured labels as our training data. The software builds a machine learning model of type deep learning, specifically a convolutional neural network (CNN) using the training data so that when new data arrives in the future the annotations can be predicted.

daft [SWR-17-53]

Brian Bush, Jacob Hinkle

Computer Networks and Communications · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · · cited 0× · DOI ↗ · more like this ·

Daft is a Haskell library that implements the key constructs of relational algebra via static structural typing. Its DataCube typeclass supports selection, projection, aggregation, disaggregation, natural joins, semijoins, and antijoins. Instances of DataCube include both map- and function-based representations. An additional abstract data type provides Vinyl-style, keyed relational tables for which an embedded domain-specific language compactly represents relational operations. Utility types and functions support serialization to tab-separated-value data files, JSON arrays, MongoDB documents, and PostgreSQL tables, including automatic detection and verification of the schema for the input data.

A map estimation algorithm for Bayesian polynomial regression on riemannian manifolds

Prasanna Muralidharan, Jacob Hinkle, P. Thomas Fletcher

Geometry and Topology · · cited 0× · DOI ↗ · more like this ·

In this paper, we present a Bayesian formulation of polynomial regression on a Riemannian manifold. Previous methods for fitting a curve to manifold-valued data have been formulated as geometric, least-squares estimation problems. We show that least-squares estimation on manifolds, much like the familiar Euclidean case, suffers from overfitting when using higher-order polynomials. Our Bayesian model mitigates this overfitting by placing a prior on the polynomial coefficients that shrinks their magnitude, analogous to Bayesian Euclidean regression with a Gaussian prior on the coefficients. We develop an algorithm for computing maximum a posteriori estimates of polynomial coefficients and the noise variance. Experiments on synthetically generated sphere data and a real shape regression problem demonstrate the advantages of our approach.