arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Luc Devroye

268 papers · 1973–2026

1973: 11974: 11975: 01976: 41977: 31978: 21979: 51980: 21981: 51982: 41983: 11984: 31985: 31986: 101987: 31988: 31989: 51990: 51991: 21992: 91993: 31994: 41995: 51996: 51997: 71998: 51999: 82000: 62001: 52002: 52003: 42004: 62005: 72006: 22007: 82008: 72009: 102010: 82011: 72012: 72013: 92014: 72015: 92016: 22017: 52018: 82019: 22020: 52021: 42022: 32023: 52024: 92025: 42026: 6
1973peak 10 in 19862026

Frequent co-authors

Papers

showing 1–25 of 268 RSS

Nonparametric Density Estimation: The L 1 View.

James R. Thompson, Luc Devroye, László Györfi

Statistics and Probability · Journal of the American Statistical Association · · cited 804× · DOI ↗ · more like this ·

Differentiation of Integrals Consistency Lower Bounds for Rates of Convergence Rates of Convergence in L1 The Automatic Kernel Estimate: L1 and Pointwise Convergence Estimates Related to the Kernel Estimate and the Histogram Estimate Simulation, Inequalities, and Random Variate Generation The Transformed Kernel Estimate Applications in Discrimination Operations on Density Estimates Estimators Based on Orthogonal Series Index.

Consistency of Random Forests and Other Averaging Classifiers

Gérard Biau, Luc Devroye, Gábor Lugosi

Computational Theory and Mathematics · HAL (Le Centre pour la Communication Scientifique Directe) · · cited 467× · OpenAlex ↗ · more like this ·

In the last years of his life, Leo Breiman promoted random forests for use in classification. He suggested using averaging as a means of obtaining good discrimination rules. The base classifiers used for averaging are simple and randomized, often based on random samples from the data. He left a few questions unanswered regarding the consistency of such rules. In this paper, we give a number of theorems that establish the universal consistency of averaging rules. We also show that some popular classifiers, including one suggested by Breiman, are not universally consistent.

On the Strong Universal Consistency of Nearest Neighbor Regression Function Estimates

Luc Devroye, László Györfi, Adam Krzyżak +1 more

Statistics and Probability · The Annals of Statistics · · cited 305× · DOI ↗ · more like this ·

Two results are presented concerning the consistency of the $k$-nearest neighbor regression estimate. We show that all modes of convergence in $L_1$ (in probability, almost sure, complete) are equivalent if the regression variable is bounded. Under the additional conditional $k/\log n \rightarrow \infty$ we also obtain the strong universal consistency of the estimate.

A note on the height of binary search trees

Luc Devroye

Discrete Mathematics and Combinatorics · Journal of the ACM · · cited 234× · DOI ↗ · more like this ·

Let H n be the height of a binary search tree with n nodes constructed by standard insertions from a random permutation of 1, … , n . It is shown that H n /log n → c = 4.31107 … in probability as n → ∞, where c is the unique solution of c log((2 e )/ c ) = 1, c ≥ 2. Also, for all p > 0, lim n →∞ E ( H p n )/ log p n = c p . Finally, it is proved that S n /log n → c * = 0.3733 … , in probability, where c * is defined by c log((2 e )/ c ) = 1, c ≤ 1, and S n is the saturation level of the same tree, that is, the number of full levels in the tree.

Distribution-free performance bounds for potential function rules

Luc Devroye, T. Wagner

Artificial Intelligence · IEEE Transactions on Information Theory · · cited 230× · DOI ↗ · more like this ·

In the discrimination problem the random variable <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">\theta</tex> , known to take values in <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">{1, \cdots ,M}</tex> , is estimated from the random vector <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X</tex> . All that is known about the joint distribution of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(X, \theta)</tex> is that which can be inferred from a sample <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(X_{1}, \theta_{1}), \cdots ,(X_{n}, \theta_{n})</tex> of size <tex xmlns:mml="http://www.w3.org/1998/Math/MathML"…

Distribution-Free Consistency Results in Nonparametric Discrimination and Regression Function Estimation

Luc Devroye, T. Wagner

Statistics and Probability · The Annals of Statistics · · cited 228× · DOI ↗ · more like this ·

Let $(X, Y)$ be an $\mathbb{R}^d \times \mathbb{R}$-valued random vector and let $(X_1, Y_1), \cdots, (X_n, Y_n)$ be a random sample drawn from its distribution. We study the consistency properties of the kernel estimate $m_n(x)$ of the regression function $m(x) = E\{Y\mid X = x\}$ that is defined by $m_n(x) = \Sigma^n_{i=1} Y_ik((X_i - x)/h_n)/\Sigma^n_{i=1}k((X_i - x)/h_n)$ where $k$ is a bounded nonnegative function on $\mathbb{R}^d$ with compact support and $\{h_n\}$ is a sequence of positive numbers satisfying $h_n \rightarrow_n0, nh^d_n \rightarrow_n\infty$. It is shown that $E\{\int|m_n(x) - m(x)|^p\mu(dx)\} \rightarrow_n 0$ whenever $E\{|Y|^p\} < \infty(p \geqslant 1)$. No other restrictions are placed on the distribution of $(X, Y)$. The result is applied to verify the Bayes risk consistency of the corresponding discrimination rules.

On the Almost Everywhere Convergence of Nonparametric Regression Function Estimates

Luc Devroye

Statistics and Probability · The Annals of Statistics · · cited 215× · DOI ↗ · more like this ·

Let $(X, Y), (X_1, Y_1), \cdots, (X_n, Y_n)$ be independent identically distributed random vectors from $R^d \times R$, and let $E(|Y|^p) < \infty$ for some $p \geq 1$. We wish to estimate the regression function $m(x) = E(Y \mid X = x)$ by $m_n(x)$, a function of $x$ and $(X_1, Y_1), \cdots, (X_n, Y_n)$. For large classes of kernel estimates and nearest neighbor estimates, sufficient conditions are given for $E\{|m_n(x) - m(x)|^p\} \rightarrow 0$ as $n \rightarrow \infty$, almost all $x$. No additional conditions are imposed on the distribution of $(X, Y)$. As a by-product, just assuming the boundedness of $Y$, the almost sure convergence to 0 of $E\{|m_n(X) - m(X)\| X_1, Y_1, \cdots, X_n, Y_n\}$ is established for the same estimates. Finally, the weak and strong Bayes risk consistency of the corresponding nonparametric discrimination rules is proved for all possible distributions of…

Sample-based non-uniform random variate generation

Luc Devroye

Artificial Intelligence · NCSU Libraries Repository (North Carolina State University Libraries) · · cited 205× · DOI ↗ · more like this ·

A sample of n lid random variables with a given unknown density is given. We discuss several issues related to the problem or generating a new sample of lid random variables with almost the same density. In particular, we look at sample independence, consistency, sample indistinguishability, moment matching and generator efficiency. We also introduce the notion of a replacement number, the minimum number of observations in a given sample that have to be replaced to obtain a sample with a given density.

Detection of Abnormal Behavior Via Nonparametric Estimation of the Support

Luc Devroye, G.L. Wise

Computer Networks and Communications · SIAM Journal on Applied Mathematics · · cited 179× · DOI ↗ · more like this ·

In this paper two problems are considered, both involving the nonparametric estimation of the support of a random vector from a sequence of independent identically distributed observations. In the first problem, after observing n independent random vectors with a common unknown distribution $\mu $, we are given one new measurement and we wish to know whether or not it belongs to the support of $\mu $. In the second problem, after observing the n independent random vectors with a common unknown distribution $\mu $, we then observe n additional independent random vectors with a common unknown distribution $\nu $. In this case we wish to know whether or not the support of $\nu $ is completely contained within the support of $\mu $. Decision schemes are presented and then convergence properties are established.

The Strong Uniform Consistency of Nearest Neighbor Density Estimates

Luc Devroye, T. Wagner

Statistics and Probability · The Annals of Statistics · · cited 148× · DOI ↗ · more like this ·

Let $X_1,\cdots, X_n$ be independent, identically distributed random vectors with values in $\mathbb{R}^d$ and with a common probability density $f$. If $V_k(x)$ is the volume of the smallest sphere centered at $x$ and containing at least $k$ of the $X_1,\cdots, X_n$ then $f_n(x) = k/(nV_k(x))$ is a nearest neighbor density estimate of $f$. We show that if $k = k(n)$ satisfies $k(n)/n \rightarrow 0$ and $k(n)/\log n \rightarrow \infty$ then $\sup_x|f_n(x) - f(x)|\rightarrow 0$ w.p. 1 when $f$ is uniformly continuous on $\mathbb{R}^d$.

The Equivalence of Weak, Strong and Complete Convergence in $L_1$ for Kernel Density Estimates

Luc Devroye

Numerical Analysis · The Annals of Statistics · · cited 146× · DOI ↗ · more like this ·

Let $f$ be a density on $R^d$, and let $f_n$ be the kernel estimate of $f$, $f_n(x) = (nh^d)^{-1} \sum^n_{i=1} K((x - X_i)/h)$ where $h = h_n$ is a sequence of positive numbers, and $K$ is an absolutely integrable function with $\int K(x) dx = 1$. Let $J_n = \int |f_n(x) - f(x)| dx$. We show that when $\lim_nh = 0$ and $\lim_nnh^d = \infty$, then for every $\varepsilon > 0$ there exist constants $r, n_0 > 0$ such that $P(J_n \geq \varepsilon) \leq \exp(-rn), n \geq n_0$. Also, when $J_n \rightarrow 0$ in probability as $n \rightarrow \infty$ and $K$ is a density, then $\lim_nh = 0$ and $\lim_nnh^d = \infty$.

Automatic pattern recognition: a study of the probability of error

Luc Devroye

Artificial Intelligence · IEEE Transactions on Pattern Analysis and Machine Intelligence · · cited 144× · DOI ↗ · more like this ·

A test sequence is used to select the best rule from a class of discrimination rules defined in terms of the training sequence. The Vapnik-Chervonenkis and related inequalities are used to obtain distribution-free bounds on the difference between the probability of error of the selected rule and the probability of error of the best rule in the given class. The bounds are used to prove the consistency and asymptotic optimality for several popular classes, including linear discriminators, nearest-neighbor rules, kernel-based rules, histogram rules, binary tree classifiers, and Fourier series classifiers. In particular, the method can be used to choose the smoothing parameter in kernel-based rules, to choose k in the k-nearest neighbor rule, and to choose between parametric and nonparametric rules.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML"…

The uniform convergence of nearest neighbor regression function estimators and their application in optimization

Luc Devroye

Statistics and Probability · IEEE Transactions on Information Theory · · cited 141× · DOI ↗ · more like this ·

A class of nonparametric regression function estimates generalizing the nearest neighbor estimate of Cover [ 12] is presented. Under various noise conditions, it is shown that the estimates are strongly uniformly consistent. The uniform convergence of the estimates can be exploited to design a simple random search algorithm for the global minimization of the regression function.

On the Performance of Clustering in Hilbert Spaces

Gérard Biau, Luc Devroye, Gábor Lugosi

Computer Vision and Pattern Recognition · IEEE Transactions on Information Theory · · cited 134× · DOI ↗ · more like this ·

Based on randomly drawn vectors in a separable Hilbert space, one may construct a k-means clustering scheme by minimizing an empirical squared error. We investigate the risk of such a clustering scheme, defined as the expected squared distance of a random vector X from the set of cluster centers. Our main result states that, for an almost surely bounded , the expected excess clustering risk is O(¿1/n) . Since clustering in high (or even infinite)-dimensional spaces may lead to severe computational problems, we examine the properties of a dimension reduction strategy for clustering based on Johnson-Lindenstrauss-type random projections. Our results reflect a tradeoff between accuracy and computational complexity when one uses k-means clustering after random projection of the data to a low-dimensional space. We argue that random projections work better than other simplistic dimension…

Consistent deconvolution in density estimation

Luc Devroye

Statistics and Probability · Canadian Journal of Statistics · · cited 128× · DOI ↗ · more like this ·

Abstract Suppose we have n observations from X = Y + Z , where Z is a noise component with known distribution, and Y has an unknown density f . When the characteristic function of Z is nonzero almost everywhere, we show that it is possible to construct a density estimate f n such that for all f , Iim n | |=0.

Laws of the Iterated Logarithm for Order Statistics of Uniform Spacings

Luc Devroye

Mathematical Physics · The Annals of Probability · · cited 127× · DOI ↗ · more like this ·

Let $X_1, X_2, \cdots$ be a sequence of independent uniformly distributed random variables on $\lbrack 0, 1\rbrack$, and let $K_n$ be the $k$th largest spacing induced by the order statistics of $X_1, \cdots, X_{n - 1}$. We show that $\lim \sup(nK_n - \log n)/2 \log_2n = 1/k \quad\text{almost surely},$ and $\lim \inf(nK_n - \log n + \log_3n) = c \quad\text{almost surely},$ where $-\log 2 \leq c \leq 0$, and $\log_j$ is the $j$ times iterated logarithm.

On combinatorial testing problems

Louigi Addario-Berry, Nicolas Broutin, Luc Devroye +1 more

math.ST · · cited 121× · arXiv ↗ · more like this ·

We study a class of hypothesis testing problems in which, upon observing the realization of an $n$-dimensional Gaussian vector, one has to decide whether the vector was drawn from a standard normal distribution or, alternatively, whether there is a subset of the components belonging to a certain given class of sets whose elements have been ``contaminated,'' that is, have a mean different from zero. We establish some general conditions under which testing is possible and others under which testing is hopeless with a small risk. The combinatorial and geometric structure of the class of sets is shown to play a crucial role. The bounds are illustrated on various examples.

Distribution-free inequalities for the deleted and holdout error estimates

Luc Devroye, T. Wagner

Artificial Intelligence · IEEE Transactions on Information Theory · · cited 120× · DOI ↗ · more like this ·

In the discrimination problem the random variable <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">\theta</tex> , known to take values in <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">{1 ,\ldots ,M}</tex> , is estimated from the random vector <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X</tex> taking values in <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">{\bfR}^{d}</tex> . Ali that is known about the joint distribution of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(X,O)</tex> is that which can be inferred from a sample <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(X_{1} , \theta_{1}, \ldots…

Random variate generation for exponentially and polynomially tilted stable distributions

Luc Devroye

Artificial Intelligence · ACM Transactions on Modeling and Computer Simulation · · cited 117× · DOI ↗ · more like this ·

We develop exact random variate generators for the polynomially and exponentially tilted unilateral stable distributions. The algorithms, which generalize Kanter's method, are uniformly fast over all choices of the tilting and stable parameters. The key to the solution is a new distribution which we call Zolotarev's distribution. We also present a novel double rejection method that is useful whenever densities have an integral representation involving an auxiliary variable.

Universal Limit Laws for Depths in Random Trees

Luc Devroye

Signal Processing · SIAM Journal on Computing · · cited 114× · DOI ↗ · more like this ·

Random binary search trees, b-ary search trees, median-of-(2k+1) trees, quadtrees, simplex trees, tries, and digital search trees are special cases of random split trees. For these trees, we offer a universal law of large numbers and a limit law for the depth of the last inserted point, as well as a law of large numbers for the height.

On the Inequality of Cover and Hart in Nearest Neighbor Discrimination

Luc Devroye

Gender Studies · IEEE Transactions on Pattern Analysis and Machine Intelligence · · cited 96× · DOI ↗ · more like this ·

When (X1, ¿1),..., (Xn, ¿n) are independent identically distributed random vectors from IRd X {0, 1} distributed as (X, ¿), and when ¿ is estimated by its nearest neighbor estimate ¿(1), then Cover and Hart have shown that P{¿(1) ¿ ¿}n ¿ ¿ ¿ 2E {¿ (X) (1 - ¿(X))} ¿ 2R*(1 - R*) where R* is the Bayes probability of error and ¿(x) = P{¿ = 1 | X = x}. They have conditions on the distribution of (X, ¿). We give two proofs, one due to Stone and a short original one, of the same result for all distributions of (X, ¿). If ties are carefully taken care of, we also show that P{¿(1) ¿ ¿|X1, ¿1, ..., Xn, ¿n} converges in probability to a constant for all distributions of (X, ¿), thereby strengthening results of Wagner and Fritz.

A weighted k-nearest neighbor density estimate for geometric inference

Gérard Biau, Frédéric Chazal, David Cohen‐Steiner +2 more

Statistics and Probability · Electronic Journal of Statistics · · cited 93× · DOI ↗ · more like this ·

Motivated by a broad range of potential applications in topological and geometric inference, we introduce a weighted version of the k-nearest neighbor density estimate. Various pointwise consistency results of this estimate are established. We present a general central limit theorem under the lightest possible conditions. In addition, a strong approximation result is obtained and the choice of the optimal set of weights is discussed. In particular, the classical k-nearest neighbor estimate is not optimal in a sense described in the manuscript. The proposed method has been implemented to recover level sets in both simulated and real-life data.

On the Spanning Ratio of Gabriel Graphs and beta-Skeletons

Prosenjit Bose, Luc Devroye, William Evans +1 more

Computer Graphics and Computer-Aided Design · SIAM Journal on Discrete Mathematics · · cited 84× · DOI ↗ · more like this ·

The spanning ratio of a graph defined on n points in the Euclidean plane is the maximum ratio over all pairs of data points (u,v) of the minimum graph distance between u and v divided by the Euclidean distance between u and v. A connected graph is said to be an S-spanner if the spanning ratio does not exceed S. For example, for any S there exists a point set whose minimum spanning tree isnot an S-spanner. At the other end of the spectrum, a Delaunay triangulation is guaranteed to be a 2.42-spanner [J. M. Keil and C. A. Gutwin, Discrete Comput. Geom., 7 (1992), pp. 13-28]. For proximity graphs between these two extremes, such as Gabriel graphs [K. R. Gabriel and R. R. Sokal, Systematic Zoology, 18 (1969), pp. 259-278], relative neighborhood graphs [G. T. Toussaint, Pattern Recognition, 12 (1980), pp. 261-268], and $\beta$-skeletons [D. G. Kirkpatrick and J. D. Radke, Comput. Geom., G. T.…

A Note on the Usefulness of Superkernels in Density Estimation

Luc Devroye

Numerical Analysis · The Annals of Statistics · · cited 82× · DOI ↗ · more like this ·

We consider the Akaike-Parzen-Rosenblatt density estimate $f_{nh}$ based upon any superkernel $L$ (i.e., an absolutely integrable function with $\int L = 1$, whose characteristic function is 1 on $\lbrack -1, 1\rbrack)$, and compare it with a kernel estimate $g_{nh}$ based upon an arbitrary kernel $K$. We show that for a given subclass of analytic densities, $\inf_L \sup_K \lim \sup_{n\rightarrow \infty} \frac{\inf_h \mathbb{E} \int |f_{nh} - f|}{\inf_h \mathbb{E} \int |g_{nh} - f |} = 1,$ where $h > 0$ is the smoothing factor. Thus, asymptotically, the class of superkernels is as good as any other class of kernels when certain analytic densities are estimated. We also obtain exact asymptotic expressions for the expected $L_1$ error of the kernel estimate when superkernels are used.