arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Davis

7,995 papers · 1827–2026

1967: 101968: 141969: 71970: 141971: 181972: 121973: 181974: 241975: 281976: 191977: 241978: 291979: 281980: 261981: 291982: 361983: 381984: 401985: 431986: 261987: 341988: 521989: 571990: 531991: 611992: 491993: 691994: 771995: 861996: 881997: 901998: 1111999: 1182000: 1112001: 1552002: 2092003: 1752004: 2162005: 2162006: 2362007: 2492008: 2442009: 2352010: 2832011: 3842012: 3302013: 4162014: 4402015: 4472016: 2992017: 2792018: 3132019: 3292020: 3602021: 3532022: 1782023: 22024: 12025: 02026: 1
1967peak 447 in 20152026

Frequent co-authors

Papers

showing 1–25 of 7,995 RSS

TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems

Martín Abadi, Ashish Agarwal, Paul Barham +37 more

cs.DC · · cited 11,866× · arXiv ↗ · more like this ·

TensorFlow is an interface for expressing machine learning algorithms, and an implementation for executing such algorithms. A computation expressed using TensorFlow can be executed with little or no change on a wide variety of heterogeneous systems, ranging from mobile devices such as phones and tablets up to large-scale distributed systems of hundreds of machines and thousands of computational devices such as GPU cards. The system is flexible and can be used to express a wide variety of algorithms, including training and inference algorithms for deep neural network models, and it has been used for conducting research and for deploying machine learning systems into production across more than a dozen areas of computer science and other fields, including speech recognition, computer vision, robotics, information retrieval, natural language processing, geographic information extraction…

TensorFlow: A system for large-scale machine learning

Martín Abadi, Paul Barham, Jianmin Chen +19 more

cs.DC · · cited 8,825× · arXiv ↗ · more like this ·

TensorFlow is a machine learning system that operates at large scale and in heterogeneous environments. TensorFlow uses dataflow graphs to represent computation, shared state, and the operations that mutate that state. It maps the nodes of a dataflow graph across many machines in a cluster, and within a machine across multiple computational devices, including multicore CPUs, general-purpose GPUs, and custom designed ASICs known as Tensor Processing Units (TPUs). This architecture gives flexibility to the application developer: whereas in previous "parameter server" designs the management of shared state is built into the system, TensorFlow enables developers to experiment with novel optimizations and training algorithms. TensorFlow supports a variety of applications, with particularly strong support for training and inference on deep neural networks. Several Google services use…

On the Opportunities and Risks of Foundation Models

Rishi Bommasani, Drew A. Hudson, Ehsan Adeli +111 more

cs.LG · · cited 7,452× · arXiv ↗ · more like this ·

AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles(e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent…

The relationship between Precision-Recall and ROC curves

Jesse Davis, Mark Goadrich

Artificial Intelligence · Minds at UW (University of Wisconsin) · · cited 6,255× · DOI ↗ · more like this ·

Receiver Operator Characteristic (ROC) curves are commonly used to present results for binary decision problems in machine learning. However, when dealing with highly skewed datasets, Precision-Recall (PR) curves give a more informative picture of an algorithm's performance. We show that a deep connection exists between ROC space and PR space, such that a curve dominates in ROC space if and only if it dominates in PR space. A corollary is the notion of an achievable PR curve, which has properties much like the convex hull in ROC space; we show an efficient algorithm for computing this curve. Finally, we also note differences in the two types of curves are significant for algorithm design. For example, in PR space it is incorrect to linearly interpolate between points. Furthermore, algorithms that optimize the area under the ROC curve are not guaranteed to optimize the area under the PR…

Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer

Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz +4 more

cs.LG · · cited 5,777× · arXiv ↗ · more like this ·

The capacity of a neural network to absorb information is limited by its number of parameters. Conditional computation, where parts of the network are active on a per-example basis, has been proposed in theory as a way of dramatically increasing model capacity without a proportional increase in computation. In practice, however, there are significant algorithmic and performance challenges. In this work, we address these challenges and finally realize the promise of conditional computation, achieving greater than 1000x improvements in model capacity with only minor losses in computational efficiency on modern GPU clusters. We introduce a Sparsely-Gated Mixture-of-Experts layer (MoE), consisting of up to thousands of feed-forward sub-networks. A trainable gating network determines a sparse combination of these experts to use for each example. We apply the MoE to the tasks of language…

Planck 2015 results. XII. Full Focal Plane simulations

Planck Collaboration, P. A. R. Ade, N. Aghanim +228 more

astro-ph.CO · · cited 4,910× · arXiv ↗ · more like this ·

We present the 8th Full Focal Plane simulation set (FFP8), deployed in support of the Planck 2015 results. FFP8 consists of 10 fiducial mission realizations reduced to 18144 maps, together with the most massive suite of Monte Carlo realizations of instrument noise and CMB ever generated, comprising $10^4$ mission realizations reduced to about $10^6$ maps. The resulting maps incorporate the dominant instrumental, scanning, and data analysis effects; remaining subdominant effects will be included in future updates. Generated at a cost of some 25 million CPU-hours spread across multiple high-performance-computing (HPC) platforms, FFP8 is used for the validation and verification of analysis algorithms, as well as their implementations, and for removing biases from and quantifying uncertainties in the results of analyses of the real data.

Common Voice: A Massively-Multilingual Speech Corpus

Rosana Ardila, Megan Branson, Kelly Davis +7 more

cs.CL · · cited 2,681× · arXiv ↗ · more like this ·

The Common Voice corpus is a massively-multilingual collection of transcribed speech intended for speech technology research and development. Common Voice is designed for Automatic Speech Recognition purposes but can be useful in other domains (e.g. language identification). To achieve scale and sustainability, the Common Voice project employs crowdsourcing for both data collection and data validation. The most recent release includes 29 languages, and as of November 2019 there are a total of 38 languages collecting data. Over 50,000 individuals have participated so far, resulting in 2,500 hours of collected audio. To our knowledge this is the largest audio corpus in the public domain for speech recognition, both in terms of number of hours and number of languages. As an example use case for Common Voice, we present speech recognition experiments using Mozilla's DeepSpeech…

Rethinking Attention with Performers

Krzysztof Choromanski, Valerii Likhosherstov, David Dohan +10 more

cs.LG · · cited 2,501× · arXiv ↗ · more like this ·

We introduce Performers, Transformer architectures which can estimate regular (softmax) full-rank-attention Transformers with provable accuracy, but using only linear (as opposed to quadratic) space and time complexity, without relying on any priors such as sparsity or low-rankness. To approximate softmax attention-kernels, Performers use a novel Fast Attention Via positive Orthogonal Random features approach (FAVOR+), which may be of independent interest for scalable kernel methods. FAVOR+ can be also used to efficiently model kernelizable attention mechanisms beyond softmax. This representational power is crucial to accurately compare softmax with other kernels for the first time on large-scale tasks, beyond the reach of regular Transformers, and investigate optimal attention-kernels. Performers are linear architectures fully compatible with regular Transformers and with strong…

Soft-NMS -- Improving Object Detection With One Line of Code

Navaneeth Bodla, Bharat Singh, Rama Chellappa +1 more

cs.CV · · cited 2,150× · arXiv ↗ · more like this ·

Non-maximum suppression is an integral part of the object detection pipeline. First, it sorts all detection boxes on the basis of their scores. The detection box M with the maximum score is selected and all other detection boxes with a significant overlap (using a pre-defined threshold) with M are suppressed. This process is recursively applied on the remaining boxes. As per the design of the algorithm, if an object lies within the predefined overlap threshold, it leads to a miss. To this end, we propose Soft-NMS, an algorithm which decays the detection scores of all other objects as a continuous function of their overlap with M. Hence, no object is eliminated in this process. Soft-NMS obtains consistent improvements for the coco-style mAP metric on standard datasets like PASCAL VOC 2007 (1.7% for both R-FCN and Faster-RCNN) and MS-COCO (1.3% for R-FCN and 1.1% for Faster-RCNN) by just…

The Rise of Social Bots

Emilio Ferrara, Onur Varol, Clayton Davis +2 more

cs.SI · · cited 2,128× · arXiv ↗ · more like this ·

The Turing test aimed to recognize the behavior of a human from that of a computer algorithm. Such challenge is more relevant than ever in today's social media context, where limited attention and technology constrain the expressive power of humans, while incentives abound to develop software agents mimicking humans. These social bots interact, often unnoticed, with real people in social media ecosystems, but their abundance is uncertain. While many bots are benign, one can design harmful bots with the goals of persuading, smearing, or deceiving. Here we discuss the characteristics of modern, sophisticated social bots, and how their presence can endanger online ecosystems and our society. We then review current efforts to detect social bots on Twitter. Features related to content, network, sentiment, and temporal patterns of activity are imitated by bots but at the same time can help…

Learning Temporal Regularity in Video Sequences

Mahmudul Hasan, Jonghyun Choi, Jan Neumann +2 more

cs.CV · · cited 1,410× · arXiv ↗ · more like this ·

Perceiving meaningful activities in a long video sequence is a challenging problem due to ambiguous definition of 'meaningfulness' as well as clutters in the scene. We approach this problem by learning a generative model for regular motion patterns, termed as regularity, using multiple sources with very limited supervision. Specifically, we propose two methods that are built upon the autoencoders for their ability to work with little to no supervision. We first leverage the conventional handcrafted spatio-temporal local features and learn a fully connected autoencoder on them. Second, we build a fully convolutional feed-forward autoencoder to learn both the local features and the classifiers as an end-to-end learning framework. Our model can capture the regularities from multiple datasets. We evaluate our methods in both qualitative and quantitative ways - showing the learned regularity…

What is the State of Neural Network Pruning?

Davis Blalock, Jose Javier Gonzalez Ortiz, Jonathan Frankle +1 more

cs.LG · · cited 1,313× · arXiv ↗ · more like this ·

Neural network pruning---the task of reducing the size of a network by removing parameters---has been the subject of a great deal of work in recent years. We provide a meta-analysis of the literature, including an overview of approaches to pruning and consistent findings in the literature. After aggregating results across 81 papers and pruning hundreds of models in controlled conditions, our clearest finding is that the community suffers from a lack of standardized benchmarks and metrics. This deficiency is substantial enough that it is hard to compare pruning techniques to one another or determine how much progress the field has made over the past three decades. To address this situation, we identify issues with current practices, suggest concrete remedies, and introduce ShrinkBench, an open-source framework to facilitate standardized evaluations of pruning methods. We use ShrinkBench…

The Rotation of Eigenvectors by a Perturbation. III

Chandler Davis, W. Kahan

Computational Theory and Mathematics · SIAM Journal on Numerical Analysis · · cited 1,212× · DOI ↗ · more like this ·

When a Hermitian linear operator is slightly perturbed, by how much can its invariant subspaces change? Given some approximations to a cluster of neighboring eigenvalues and to the corresponding eigenvectors of a real symmetric matrix, and given an estimate for the gap that separates the cluster from all other eigenvalues, how much can the subspace spanned by the eigenvectors differ from the subspace spanned by our approximations? These questions are closely related; both are investigated here. The difference between the two subspaces is characterized in terms of certain angles through which one subspace must be rotated in order most directly to reach the other. These angles unify the treatment of natural geometric, operator-theoretic and error-analytic questions concerning those subspaces. Sharp bounds upon trigonometric functions of these angles are obtained from the gap and from…

Online Human-Bot Interactions: Detection, Estimation, and Characterization

Onur Varol, Emilio Ferrara, Clayton A. Davis +2 more

cs.SI · · cited 1,140× · arXiv ↗ · more like this ·

Increasing evidence suggests that a growing amount of social media content is generated by autonomous entities known as social bots. In this work we present a framework to detect such entities on Twitter. We leverage more than a thousand features extracted from public data and meta-data about users: friends, tweet content and sentiment, network patterns, and activity time series. We benchmark the classification framework by using a publicly available dataset of Twitter bots. This training data is enriched by a manually annotated collection of active Twitter users that include both humans and bots of varying sophistication. Our models yield high accuracy and agreement with each other and can detect bots of different nature. Our estimates suggest that between 9% and 15% of active Twitter accounts are bots. Characterizing ties among accounts, we observe that simple bots tend to interact…

DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker

Matej Moravčík, Martin Schmid, Neil Burch +7 more

cs.AI · · cited 1,043× · arXiv ↗ · more like this ·

Artificial intelligence has seen several breakthroughs in recent years, with games often serving as milestones. A common feature of these games is that players have perfect information. Poker is the quintessential game of imperfect information, and a longstanding challenge problem in artificial intelligence. We introduce DeepStack, an algorithm for imperfect information settings. It combines recursive reasoning to handle information asymmetry, decomposition to focus computation on the relevant decision, and a form of intuition that is automatically learned from self-play using deep learning. In a study involving 44,000 hands of poker, DeepStack defeated with statistical significance professional poker players in heads-up no-limit Texas hold'em. The approach is theoretically sound and is shown to produce more difficult to exploit strategies than prior approaches.

Training feedforward neural networks using genetic algorithms

David J. Montana, Lawrence Davis

Artificial Intelligence · International Joint Conference on Artificial Intelligence · · cited 999× · OpenAlex ↗ · more like this ·

Multilayered feedforward neural networks possess a number of properties which make them particularly suited to complex pattern classification problems. However, their application to some realworld problems has been hampered by the lack of a training algonthm which reliably finds a nearly globally optimal set of weights in a relatively short time. Genetic algorithms are a class of optimization procedures which are good at exploring a large and complex space in an intelligent way to find values close to the global optimum. Hence, they are well suited to the problem of training feedforward networks. In this paper, we describe a set of experiments performed on data from a sonar image classification problem. These experiments both 1) illustrate the improvements gained by using a genetic algorithm rather than backpropagation and 2) chronicle the evolution of the performance of the genetic…

Children's Time With Fathers in Intact Families

Wei‐Jun Jean Yeung, John Sandberg, Pamela Davis‐Kean +1 more

Gender Studies · Journal of Marriage and the Family · · cited 973× · DOI ↗ · more like this ·

This paper uses the Panel Study of Income Dynamics to examine children's involvement with their fathers in intact families as measured through time spent together. Our findings suggest that although mothers still shoulder the lion's share of the parenting, fathers' involvement relative to that of mothers appears to be on the increase. A “new father” role is emerging on weekends in intact families. Different determinants of fathers' involvement were found on weekdays and on weekends. Fathers' wages and work hours have a negative relationship with the time they spend with a child on weekdays, but not on weekends. Mothers' work hours have no effect on children's time with fathers. On weekends, Black fathers were found to be less involved and Latino fathers more involved with their children than are White fathers. The weekday‐weekend differential suggests that a simple gender inequality…

BotOrNot: A System to Evaluate Social Bots

Clayton A. Davis, Onur Varol, Emilio Ferrara +2 more

cs.SI · · cited 938× · arXiv ↗ · more like this ·

While most online social media accounts are controlled by humans, these platforms also host automated agents called social bots or sybil accounts. Recent literature reported on cases of social bots imitating humans to manipulate discussions, alter the popularity of users, pollute content and spread misinformation, and even perform terrorist propaganda and recruitment actions. Here we present BotOrNot, a publicly-available service that leverages more than one thousand features to evaluate the extent to which a Twitter account exhibits similarity to the known characteristics of social bots. Since its release in May 2014, BotOrNot has served over one million requests via our website and APIs.

NISP: Pruning Networks using Neuron Importance Score Propagation

Ruichi Yu, Ang Li, Chun-Fu Chen +6 more

cs.CV · · cited 889× · arXiv ↗ · more like this ·

To reduce the significant redundancy in deep Convolutional Neural Networks (CNNs), most existing methods prune neurons by only considering statistics of an individual layer or two consecutive layers (e.g., prune one layer to minimize the reconstruction error of the next layer), ignoring the effect of error propagation in deep networks. In contrast, we argue that it is essential to prune neurons in the entire neuron network jointly based on a unified goal: minimizing the reconstruction error of important responses in the "final response layer" (FRL), which is the second-to-last layer before classification, for a pruned network to retrain its predictive power. Specifically, we apply feature ranking techniques to measure the importance of each neuron in the FRL, and formulate network pruning as a binary integer optimization problem and derive a closed-form solution to it for pruning…

Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification

Le Hou, Dimitris Samaras, Tahsin M. Kurc +3 more

cs.CV · · cited 882× · arXiv ↗ · more like this ·

Convolutional Neural Networks (CNN) are state-of-the-art models for many image classification tasks. However, to recognize cancer subtypes automatically, training a CNN on gigapixel resolution Whole Slide Tissue Images (WSI) is currently computationally impossible. The differentiation of cancer subtypes is based on cellular-level visual features observed on image patch scale. Therefore, we argue that in this situation, training a patch-level classifier on image patches will perform better than or similar to an image-level classifier. The challenge becomes how to intelligently combine patch-level classification results and model the fact that not all patches will be discriminative. We propose to train a decision fusion model to aggregate patch-level predictions given by patch-level CNNs, which to the best of our knowledge has not been shown before. Furthermore, we formulate a novel…

#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning

Haoran Tang, Rein Houthooft, Davis Foote +6 more

cs.AI · · cited 878× · arXiv ↗ · more like this ·

Count-based exploration algorithms are known to perform near-optimally when used in conjunction with tabular reinforcement learning (RL) methods for solving small discrete Markov decision processes (MDPs). It is generally thought that count-based methods cannot be applied in high-dimensional state spaces, since most states will only occur once. Recent deep RL exploration strategies are able to deal with high-dimensional continuous state spaces through complex heuristics, often relying on optimism in the face of uncertainty or intrinsic motivation. In this work, we describe a surprising finding: a simple generalization of the classic count-based approach can reach near state-of-the-art performance on various high-dimensional and/or continuous deep RL benchmarks. States are mapped to hash codes, which allows to count their occurrences with a hash table. These counts are then used to…

Determinants of Risk: Exposure and Vulnerability

Omar D. Cardona, Maarten van Aalst, Jörn Birkmann +9 more

Sociology and Political Science · Cambridge University Press eBooks · · cited 853× · DOI ↗ · more like this ·

Many climate change adaptation efforts aim to address the implications of potential changes in the frequency, intensity, and duration of weather and climate events that affect the risk of extreme impacts on human society. That risk is determined not only by the climate and weather events (the hazards) but also by the exposure and vulnerability to these hazards. Therefore, effective adaptation and disaster risk management strategies and practices also depend on a rigorous understanding of the dimensions of exposure and vulnerability, as well as a proper assessment of changes in those dimensions. This chapter aims to provide that understanding and assessment, by further detailing the determinants of risk as presented in Chapter 1. \n\nThe first sections of this chapter elucidate the concepts that are needed to define and understand risk, and show that risk originates from a combination of…

An Analysis of Scale Invariance in Object Detection - SNIP

Bharat Singh, Larry S. Davis

cs.CV · · cited 850× · arXiv ↗ · more like this ·

An analysis of different techniques for recognizing and detecting objects under extreme scale variation is presented. Scale specific and scale invariant design of detectors are compared by training them with different configurations of input data. By evaluating the performance of different network architectures for classifying small objects on ImageNet, we show that CNNs are not robust to changes in scale. Based on this analysis, we propose to train and test detectors on the same scales of an image-pyramid. Since small and large objects are difficult to recognize at smaller and larger scales respectively, we present a novel training scheme called Scale Normalization for Image Pyramids (SNIP) which selectively back-propagates the gradients of object instances of different sizes as a function of the image scale. On the COCO dataset, our single model performance is 45.7% and an ensemble of…

Conceptualizing the Role and Influence of Student-Teacher Relationships on Children's Social and Cognitive Development

Heather A. Davis

Education · Educational Psychologist · · cited 815× · DOI ↗ · more like this ·

Abstract The purpose of this article is to add to our current understanding of the social contexts of education by synthesizing research on the nature and influence of relationships between students and their teachers. I was guided by 3 questions. First, how have we conceptualized students' relationships with teachers? Second, how have our approaches to studying relationships shaped our understanding of the phenomenon? Third, looking across these different approaches to studying student-teacher relationships, what do we know about the nature and influence of student-teacher relationships developmentally? Specifically, I review in 3 broad themes the multiple conceptions, or approaches, to the study of student-teacher relationships. These include student-teacher relationships from attachment perspectives, from motivation perspectives, and from sociocultural perspectives. I view each…

MLPerf Inference Benchmark

Vijay Janapa Reddi, Christine Cheng, David Kanter +44 more

cs.LG · · cited 787× · arXiv ↗ · more like this ·

Machine-learning (ML) hardware and software system demand is burgeoning. Driven by ML applications, the number of different ML inference systems has exploded. Over 100 organizations are building ML inference chips, and the systems that incorporate existing models span at least three orders of magnitude in power consumption and five orders of magnitude in performance; they range from embedded devices to data-center solutions. Fueling the hardware are a dozen or more software frameworks and libraries. The myriad combinations of ML hardware and ML software make assessing ML-system performance in an architecture-neutral, representative, and reproducible manner challenging. There is a clear need for industry-wide standard ML benchmarking and evaluation criteria. MLPerf Inference answers that call. In this paper, we present our benchmarking method for evaluating ML inference systems. Driven…