arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Edward

7,997 papers · 1835–2024

1965: 101966: 141967: 111968: 111969: 291970: 211971: 221972: 251973: 341974: 321975: 431976: 451977: 391978: 391979: 251980: 431981: 441982: 421983: 541984: 501985: 501986: 591987: 571988: 511989: 481990: 591991: 671992: 691993: 931994: 1221995: 1061996: 1331997: 1261998: 1501999: 1502000: 1832001: 1682002: 2272003: 2182004: 2312005: 2352006: 2542007: 2692008: 2702009: 2952010: 3572011: 3452012: 3072013: 3292014: 4072015: 3622016: 3562017: 3362018: 4132019: 2602020: 132021: 12022: 32023: 52024: 2
1965peak 413 in 20182024

Frequent co-authors

Papers

showing 1–25 of 7,997 RSS

PyTorch: An Imperative Style, High-Performance Deep Learning Library

Adam Paszke, Sam Gross, Francisco Massa +18 more

cs.LG · · cited 55,920× · arXiv ↗ · more like this ·

Deep learning frameworks have often focused on either usability or speed, but not both. PyTorch is a machine learning library that shows that these two goals are in fact compatible: it provides an imperative and Pythonic programming style that supports code as a model, makes debugging easy and is consistent with other popular scientific computing libraries, while remaining efficient and supporting hardware accelerators such as GPUs. In this paper, we detail the principles that drove the implementation of PyTorch and how they are reflected in its architecture. We emphasize that every aspect of PyTorch is a regular Python program under the full control of its user. We also explain how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance. We demonstrate the efficiency of individual subsystems, as well…

A Quantum Approximate Optimization Algorithm

Edward Farhi, Jeffrey Goldstone, Sam Gutmann

quant-ph · · cited 4,674× · arXiv ↗ · more like this ·

We introduce a quantum algorithm that produces approximate solutions for combinatorial optimization problems. The algorithm depends on a positive integer p and the quality of the approximation improves as p is increased. The quantum circuit that implements the algorithm consists of unitary gates whose locality is at most the locality of the objective function whose optimum is sought. The depth of the circuit grows linearly with p times (at worst) the number of constraints. If p is fixed, that is, independent of the input size, the algorithm makes use of efficient classical preprocessing. If p grows with the input size a different strategy is proposed. We study the algorithm as applied to MaxCut on regular graphs and analyze its performance on 2-regular and 3-regular graphs for fixed p. For p = 1, on 3-regular graphs the quantum algorithm always finds a cut that is at least 0.6924 times…

Teaching Machines to Read and Comprehend

Karl Moritz Hermann, Tomáš Kočiský, Edward Grefenstette +4 more

cs.CL · · cited 3,939× · arXiv ↗ · more like this ·

Teaching machines to read natural language documents remains an elusive challenge. Machine reading systems can be tested on their ability to answer questions posed on the contents of documents that they have seen, but until now large scale training and test datasets have been missing for this type of evaluation. In this work we define a new methodology that resolves this bottleneck and provides large scale supervised reading comprehension data. This allows us to develop a class of attention based deep neural networks that learn to read real documents and answer complex questions with minimal prior knowledge of language structure.

A Convolutional Neural Network for Modelling Sentences

Nal Kalchbrenner, Edward Grefenstette, Phil Blunsom

cs.CL · · cited 3,663× · arXiv ↗ · more like this ·

The ability to accurately represent sentences is central to language understanding. We describe a convolutional architecture dubbed the Dynamic Convolutional Neural Network (DCNN) that we adopt for the semantic modelling of sentences. The network uses Dynamic k-Max Pooling, a global pooling operation over linear sequences. The network handles input sentences of varying length and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations. The network does not rely on a parse tree and is easily applicable to any language. We test the DCNN in four experiments: small scale binary and multi-class sentiment prediction, six-way question classification and Twitter sentiment prediction by distant supervision. The network achieves excellent performance in the first three tasks and a greater than 25% error reduction in the last task with…

Ultrathin epitaxial graphite: 2D electron gas properties and a route toward graphene-based nanoelectronics

Claire Berger, Zhimin Song, Tianbo Li +8 more

cond-mat.mtrl-sci · · cited 3,274× · arXiv ↗ · more like this ·

We have produced ultrathin epitaxial graphite films which show remarkable 2D electron gas (2DEG) behavior. The films, composed of typically 3 graphene sheets, were grown by thermal decomposition on the (0001) surface of 6H-SiC, and characterized by surface-science techniques. The low-temperature conductance spans a range of localization regimes according to the structural state (square resistance 1.5 kOhm to 225 kOhm at 4 K, with positive magnetoconductance). Low resistance samples show characteristics of weak-localization in two dimensions, from which we estimate elastic and inelastic mean free paths. At low field, the Hall resistance is linear up to 4.5 T, which is well-explained by n-type carriers of density 10^{12} cm^{-2} per graphene sheet. The most highly-ordered sample exhibits Shubnikov - de Haas oscillations which correspond to nonlinearities observed in the Hall resistance…

BART: Bayesian additive regression trees

Hugh A. Chipman, Edward I. George, Robert E. McCulloch

stat.ME · · cited 2,104× · arXiv ↗ · more like this ·

We develop a Bayesian "sum-of-trees" model where each tree is constrained by a regularization prior to be a weak learner, and fitting and inference are accomplished via an iterative Bayesian backfitting MCMC algorithm that generates samples from a posterior. Effectively, BART is a nonparametric Bayesian regression approach which uses dimensionally adaptive random basis elements. Motivated by ensemble methods in general, and boosting algorithms in particular, BART is defined by a statistical model: a prior and a likelihood. This approach enables full posterior inference including point and interval estimates of the unknown regression function as well as the marginal effects of potential predictors. By keeping track of predictor inclusion frequencies, BART can also be used for model-free variable selection. BART's many features are illustrated with a bake-off against competing methods on…

A Quantum Adiabatic Evolution Algorithm Applied to Random Instances of an NP-Complete Problem

Edward Farhi, Jeffrey Goldstone, Sam Gutmann +3 more

quant-ph · · cited 1,952× · arXiv ↗ · more like this ·

A quantum system will stay near its instantaneous ground state if the Hamiltonian that governs its evolution varies slowly enough. This quantum adiabatic behavior is the basis of a new class of algorithms for quantum computing. We test one such algorithm by applying it to randomly generated, hard, instances of an NP-complete problem. For the small examples that we can simulate, the quantum adiabatic algorithm works well, and provides evidence that quantum computers (if large ones can be built) may be able to outperform ordinary computers on hard sets of instances of NP-complete problems.

Quantum Computation by Adiabatic Evolution

Edward Farhi, Jeffrey Goldstone, Sam Gutmann +1 more

quant-ph · · cited 1,858× · arXiv ↗ · more like this ·

We give a quantum algorithm for solving instances of the satisfiability problem, based on adiabatic evolution. The evolution of the quantum state is governed by a time-dependent Hamiltonian that interpolates between an initial Hamiltonian, whose ground state is easy to construct, and a final Hamiltonian, whose ground state encodes the satisfying assignment. To ensure that the system evolves to the desired final ground state, the evolution time must be big enough. The time required depends on the minimum energy difference between the two lowest states of the interpolating Hamiltonian. We are unable to estimate this gap in general. We give some special symmetric cases of the satisfiability problem where the symmetry allows us to estimate the gap and we show that, in these cases, our algorithm runs in polynomial time.

RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism

Edward Choi, Mohammad Taha Bahadori, Joshua A. Kulas +3 more

cs.LG · · cited 1,608× · arXiv ↗ · more like this ·

Accuracy and interpretability are two dominant features of successful predictive models. Typically, a choice must be made in favor of complex black box models such as recurrent neural networks (RNN) for accuracy versus less accurate but more interpretable traditional models such as logistic regression. This tradeoff poses challenges in medicine where both accuracy and interpretability are important. We addressed this challenge by developing the REverse Time AttentIoN model (RETAIN) for application to Electronic Health Records (EHR) data. RETAIN achieves high accuracy while remaining clinically interpretable and is based on a two-level neural attention model that detects influential past visits and significant clinical variables within those visits (e.g. key diagnoses). RETAIN mimics physician practice by attending the EHR data in a reverse time order so that recent clinical visits are…

On choosing and bounding probability metrics

Alison L. Gibbs, Francis Edward Su

math.PR · · cited 1,535× · arXiv ↗ · more like this ·

When studying convergence of measures, an important issue is the choice of probability metric. In this review, we provide a summary and some new results concerning bounds among ten important probability metrics/distances that are used by statisticians and probabilists. We focus on these metrics because they are either well-known, commonly used, or admit practical bounding techniques. We summarize these relationships in a handy reference diagram, and also give examples to show how rates of convergence can depend on the metric chosen.

NLTK: The Natural Language Toolkit

Edward Loper, Steven Bird

cs.CL · · cited 1,473× · arXiv ↗ · more like this ·

NLTK, the Natural Language Toolkit, is a suite of open source program modules, tutorials and problem sets, providing ready-to-use computational linguistics courseware. NLTK covers symbolic and statistical natural language processing, and is interfaced to annotated corpora. Students augment and replace existing components, learn structured programming by example, and manipulate sophisticated models from the outset.

End-to-End Multi-Task Learning with Attention

Shikun Liu, Edward Johns, Andrew J. Davison

cs.CV · · cited 1,441× · arXiv ↗ · more like this ·

We propose a novel multi-task learning architecture, which allows learning of task-specific feature-level attention. Our design, the Multi-Task Attention Network (MTAN), consists of a single shared network containing a global feature pool, together with a soft-attention module for each task. These modules allow for learning of task-specific features from the global features, whilst simultaneously allowing for features to be shared across different tasks. The architecture can be trained end-to-end and can be built upon any feed-forward neural network, is simple to implement, and is parameter efficient. We evaluate our approach on a variety of datasets, across both image-to-image predictions and image classification tasks. We show that our architecture is state-of-the-art in multi-task learning compared to existing methods, and is also less sensitive to various weighting schemes in the…

Doctor AI: Predicting Clinical Events via Recurrent Neural Networks

Edward Choi, Mohammad Taha Bahadori, Andy Schuetz +2 more

cs.LG · · cited 1,286× · arXiv ↗ · more like this ·

Leveraging large historical data in electronic health record (EHR), we developed Doctor AI, a generic predictive model that covers observed medical conditions and medication uses. Doctor AI is a temporal model using recurrent neural networks (RNN) and was developed and applied to longitudinal time stamped EHR data from 260K patients over 8 years. Encounter records (e.g. diagnosis codes, medication codes or procedure codes) were input to RNN to predict (all) the diagnosis and medication categories for a subsequent visit. Doctor AI assesses the history of patients to make multilabel predictions (one label for each diagnosis or medication category). Based on separate blind test set evaluation, Doctor AI can perform differential diagnosis with up to 79% recall@30, significantly higher than several baselines. Moreover, we demonstrate great generalizability of Doctor AI by adapting the…

A Strong Coupling Test of S-Duality

Cumrun Vafa, Edward Witten

hep-th · · cited 1,237× · arXiv ↗ · more like this ·

By studying the partition function of $N=4$ topologically twisted supersymmetric Yang-Mills on four-manifolds, we make an exact strong coupling test of the Montonen-Olive strong-weak duality conjecture. Unexpected and exciting links are found with two-dimensional rational conformal field theory.

APPROACHES FOR BAYESIAN VARIABLE SELECTION

Edward I. George, Robert E. McCulloch

Artificial Intelligence · Statistica Sinica · · cited 1,159× · OpenAlex ↗ · more like this ·

Abstract: This paper describes and compares various hierarchical mixture prior formulations of variable selection uncertainty in normal linear regression models. These include the nonconjugate SSVS formulation of George and McCulloch (1993), as well as conjugate formulations which allow for analytical simplification. Hyperparameter settings which base selection on practical significance, and the implications of using mixtures with point priors are discussed. Computational methods for posterior evaluation and exploration are considered. Rapid updating methods are seen to provide feasible methods for exhaustive evaluation using Gray Code sequencing in moderately sized problems, and fast Markov Chain Monte Carlo exploration in large problems. Estimation of normalization constants is seen to provide improved posterior estimates of individual model probabilities and the total visited…

Electric-Magnetic Duality And The Geometric Langlands Program

Anton Kapustin, Edward Witten

hep-th · · cited 1,137× · arXiv ↗ · more like this ·

The geometric Langlands program can be described in a natural way by compactifying on a Riemann surface C a twisted version of N=4 super Yang-Mills theory in four dimensions. The key ingredients are electric-magnetic duality of gauge theory, mirror symmetry of sigma-models, branes, Wilson and 't Hooft operators, and topological field theory. Seemingly esoteric notions of the geometric Langlands program, such as Hecke eigensheaves and D-modules, arise naturally from the physics.

The NarrativeQA Reading Comprehension Challenge

Tomáš Kočiský, Jonathan Schwarz, Phil Blunsom +4 more

cs.CL · · cited 1,115× · arXiv ↗ · more like this ·

Reading comprehension (RC)---in contrast to information retrieval---requires integrating information and reasoning about events, entities, and their relations across a full document. Question answering is conventionally used to assess RC ability, in both artificial agents and children learning to read. However, existing RC datasets and tasks are dominated by questions that can be solved by selecting answers using superficial information (e.g., local context similarity or global term frequency); they thus fail to test for the essential integrative aspect of RC. To encourage progress on deeper comprehension of language, we present a new dataset and set of tasks in which the reader must answer questions about stories by reading entire books or movie scripts. These tasks are designed so that successfully answering their questions requires understanding the underlying narrative rather than…

Quantum Computation and Decision Trees

Edward Farhi, Sam Gutmann

quant-ph · · cited 1,104× · arXiv ↗ · more like this ·

Many interesting computational problems can be reformulated in terms of decision trees. A natural classical algorithm is to then run a random walk on the tree, starting at the root, to see if the tree contains a node n levels from the root. We devise a quantum mechanical algorithm that evolves a state, initially localized at the root, through the tree. We prove that if the classical strategy succeeds in reaching level n in time polynomial in n, then so does the quantum algorithm. Moreover, we find examples of trees for which the classical algorithm requires time exponential in n, but for which the quantum algorithm succeeds in polynomial time. The examples we have so far, however, could also be solved in polynomial time by different classical algorithms.

Low Dimensional Behavior of Large Systems of Globally Coupled Oscillators

Edward Ott, Thomas M. Antonsen

nlin.CD · · cited 1,080× · arXiv ↗ · more like this ·

It is shown that, in the infinite size limit, certain systems of globally coupled phase oscillators display low dimensional dynamics. In particular, we derive an explicit finite set of nonlinear ordinary differential equations for the macroscopic evolution of the systems considered. For example, an exact, closed form solution for the nonlinear time evolution of the Kuramoto problem with a Lorentzian oscillator frequency distribution function is obtained. Low dimensional behavior is also demonstrated for several prototypical extensions of the Kuramoto model, and time-delayed coupling is also considered.

Classification with Quantum Neural Networks on Near Term Processors

Edward Farhi, Hartmut Neven

quant-ph · · cited 1,068× · arXiv ↗ · more like this ·

We introduce a quantum neural network, QNN, that can represent labeled data, classical or quantum, and be trained by supervised learning. The quantum circuit consists of a sequence of parameter dependent unitary transformations which acts on an input quantum state. For binary classification a single Pauli operator is measured on a designated readout qubit. The measured output is the quantum neural network's predictor of the binary label of the input state. First we look at classifying classical data sets which consist of n-bit strings with binary labels. The input quantum state is an n-bit computational basis state corresponding to a sample string. We show how to design a circuit made from two qubit unitaries that can correctly represent the label of any Boolean function of n bits. For certain label functions the circuit is exponentially long. We introduce parameter dependent unitaries…

Exponential algorithmic speedup by quantum walk

Andrew M. Childs, Richard Cleve, Enrico Deotto +3 more

quant-ph · · cited 963× · arXiv ↗ · more like this ·

We construct an oracular (i.e., black box) problem that can be solved exponentially faster on a quantum computer than on a classical computer. The quantum algorithm is based on a continuous time quantum walk, and thus employs a different technique from previous quantum algorithms based on quantum Fourier transforms. We show how to implement the quantum walk efficiently in our oracular setting. We then show how this quantum walk can be used to solve our problem by rapidly traversing a graph. Finally, we prove that no classical algorithm can solve this problem with high probability in subexponential time.

Efficient Neural Audio Synthesis

Nal Kalchbrenner, Erich Elsen, Karen Simonyan +7 more

cs.SD · · cited 944× · arXiv ↗ · more like this ·

Sequential models achieve state-of-the-art results in audio, visual and textual domains with respect to both estimating the data distribution and generating high-quality samples. Efficient sampling for this class of models has however remained an elusive problem. With a focus on text-to-speech synthesis, we describe a set of general techniques for reducing sampling time while maintaining high output quality. We first describe a single-layer recurrent neural network, the WaveRNN, with a dual softmax layer that matches the quality of the state-of-the-art WaveNet model. The compact form of the network makes it possible to generate 24kHz 16-bit audio 4x faster than real time on a GPU. Second, we apply a weight pruning technique to reduce the number of weights in the WaveRNN. We find that, for a constant number of parameters, large sparse networks perform better than small dense networks and…

Parallel WaveNet: Fast High-Fidelity Speech Synthesis

Aaron van den Oord, Yazhe Li, Igor Babuschkin +19 more

cs.LG · · cited 918× · arXiv ↗ · more like this ·

The recently-developed WaveNet architecture is the current state of the art in realistic speech synthesis, consistently rated as more natural sounding for many different languages than any previous system. However, because WaveNet relies on sequential generation of one audio sample at a time, it is poorly suited to today's massively parallel computers, and therefore hard to deploy in a real-time production setting. This paper introduces Probability Density Distillation, a new method for training a parallel feed-forward network from a trained WaveNet with no significant difference in quality. The resulting system is capable of generating high-fidelity speech samples at more than 20 times faster than real-time, and is deployed online by Google Assistant, including serving multiple English and Japanese voices.

The Holographic Bound in Anti-de Sitter Space

L. Susskind, Edward Witten

hep-th · · cited 902× · arXiv ↗ · more like this ·

The correspondence between string theory in Anti-de Sitter space and super Yang Mills theory is an example of the Holographic principle according to which a quantum theory with gravity must be describable by a boundary theory. However, arguments given so far are incomplete because, while the bulk theory has been related to a boundary theory, the holographic bound saying that the boundary theory has only one bit of information per Planck area has not been justified. We show here that this bound is the physical interpretation of one of the unusual aspects of the correspondence between Anti-de Sitter space and the boundary conformal field theory, which is that infrared effects in the bulk theory are reflected as ultraviolet effects in the boundary theory.

Chern-Simons Gauge Theory As A String Theory

Edward Witten

hep-th · · cited 860× · arXiv ↗ · more like this ·

Certain two dimensional topological field theories can be interpreted as string theory backgrounds in which the usual decoupling of ghosts and matter does not hold. Like ordinary string models, these can sometimes be given space-time interpretations. For instance, three-dimensional Chern-Simons gauge theory can arise as a string theory. The world-sheet model in this case involves a topological sigma model. Instanton contributions to the sigma model give rise to Wilson line insertions in the space-time Chern-Simons theory. A certain holomorphic analog of Chern-Simons theory can also arise as a string theory.