arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Ruby C. Weng

20 papers · 1999–2025

1999: 12000: 12001: 12002: 02003: 22004: 32005: 02006: 12007: 22008: 22009: 02010: 12011: 02012: 12013: 02014: 12015: 02016: 12017: 02018: 12019: 02020: 12021: 02022: 02023: 02024: 02025: 1
1999peak 3 in 20042025

Frequent co-authors

Papers

showing 1–20 of 20 RSS

Probability Estimates for Multi-class Classification by Pairwise Coupling

Tingfan Wu, Chih‐Jen Lin, Ruby C. Weng

Statistics and Probability · Journal of Machine Learning Research · · cited 1,480× · DOI ↗ · more like this ·

Pairwise coupling is a popular multi-class classification method that combines all comparisons for each pair of classes. This paper presents two approaches for obtaining class probabilities. Both methods can be reduced to linear systems and are easy to implement. We show conceptually and experimentally that the proposed approaches are more stable than the two existing popular methods: voting and the method by Hastie and Tibshirani (1998)

Trust Region Newton Method for Logistic Regression

Chih‐Jen Lin, Ruby C. Weng, S. Sathiya Keerthi

Computer Vision and Pattern Recognition · Journal of Machine Learning Research · · cited 289× · DOI ↗ · more like this ·

Large-scale logistic regression arises in many applications such as document classification and natural language processing. In this paper, we apply a trust region Newton method to maximize the log-likelihood of the logistic regression model. The proposed method uses only approximate Newton steps in the beginning, but achieves fast convergence in the end. Experiments show that it is faster than the commonly used quasi Newton approach for logistic regression. We also extend the proposed method to large-scale L2-loss linear support vector machines (SVM).

Trust region Newton methods for large-scale logistic regression

Chih‐Jen Lin, Ruby C. Weng, S. Sathiya Keerthi

Artificial Intelligence · · cited 287× · DOI ↗ · more like this ·

Large-scale logistic regression arises in many applications such as document classification and natural language processing. In this paper, we apply a trust region Newton method to maximize the log-likelihood of the logistic regression model. The proposed method uses only approximate Newton steps in the beginning, but achieves fast convergence in the end. Experiments show that it is faster than the commonly used quasi Newton approach for logistic regression. We also compare it with linear SVM implementations.

Generalized Bradley-Terry Models and Multi-Class Probability Estimates

Tzu-Kuo Huang, Ruby C. Weng, Chih‐Jen Lin

Artificial Intelligence · Journal of Machine Learning Research · · cited 151× · DOI ↗ · more like this ·

Editor: The Bradley-Terry model for obtaining individual skill from paired comparisons has been popular in many areas. In machine learning, this model is related to multi-class probability estimates by coupling all pairwise classification results. Error correcting output codes (ECOC) are a general framework to decompose a multi-class problem to several binary problems. To obtain probability estimates under this framework, this paper introduces a generalized Bradley-Terry model in which paired individual comparisons are extended to paired team comparisons. We propose a simple algorithm with convergence proofs to solve the model and obtain individual skill. Experiments on synthetic and real data demonstrate that the algorithm is useful for obtaining multi-class probability estimates. Moreover, we discuss four extensions of the proposed model: 1) weighted individual skill, 2) home-field…

A Generalized Bradley-Terry Model: From Group Competition to Individual Skill

Tzu-Kuo Huang, Chih‐Jen Lin, Ruby C. Weng

Artificial Intelligence · · cited 19× · OpenAlex ↗ · more like this ·

The Bradley-Terry model for paired comparison has been popular in many areas. We propose a generalized version in which paired individual comparisons are extended to paired team comparisons. We introduce a simple algorithm with convergence proofs to solve the model and obtain individual skill. A useful application to multi-class probability estimates using error-correcting codes is demonstrated.

Analysis of switching dynamics with competing support vector machines

Ming‐Wei Chang, Chih‐Jen Lin, Ruby C. Weng

Artificial Intelligence · NTUR (臺灣機構典藏) · · cited 19× · DOI ↗ · more like this ·

We present a framework for the unsupervised segmentation of time series using support vector regression. It is applied to non-stationary time series which alter in time. We follow the architecture of Pawelzik et al. (1996) which consists of competing predictors. In the above paper competing neural networks were used while here we exploit the use of support vector machines, a learning technique. Results indicate that the proposed approach is as good as that of that Pawelzik et al. Differences between the two approaches are also discussed.

A Bayesian Edgeworth expansion by Stein's identity

Ruby C. Weng

Statistics and Probability · Bayesian Analysis · · cited 15× · DOI ↗ · more like this ·

The Edgeworth expansion is a series that approximates a probability distribution in terms of its cumulants. One can derive it by first expanding the probability distribution in Hermite orthogonal functions and then collecting terms in powers of the sample size. This paper derives an expansion for posterior distributions which possesses these features of an Edgeworth series. The techniques used are a version of Stein's Identity and properties of Hermite polynomials. Two examples are provided to illustrate the accuracy of our series.

INTEGRABLE EXPANSIONS FOR POSTERIOR DISTRIBUTIONS FOR MULTIPARAMETER EXPONENTIAL FAMILIES WITH APPLICATIONS TO SEQUENTIAL CONFIDENCE LEVELS

Ruby C. Weng, Michael Woodroofe

Artificial Intelligence · · cited 13× · OpenAlex ↗ · more like this ·

Integrable expansions for posterior distributions are obtained for sequen- tial samples from a multiparameter exponential family. A data dependent trans- formation is used to convert the likelihood function to the form of a standard multivariate normal density. Then a version of Stein's Identity is applied. This leaves an expression from which an asymptotic expansion is easily obtained. The results are applied to find confidence intervals for the ratio of two Poisson means after a sequential test and compare well with simulations.

Real-Time Bayesian Parameter Estimation for Item Response Models

Ruby C. Weng, D. Stephen Coad

Information Systems · Bayesian Analysis · · cited 12× · DOI ↗ · more like this ·

Bayesian item response models have been used in modeling educational testing and Internet ratings data. Typically, the statistical analysis is carried out using Markov Chain Monte Carlo methods. However, these may not be computationally feasible when real-time data continuously arrive and online parameter estimation is needed. We develop an efficient algorithm based on a deterministic moment-matching method to adjust the parameters in real-time. The proposed online algorithm works well for two real datasets, achieving good accuracy but with considerably less computational time.

ON STEIN'S IDENTITY FOR POSTERIOR NORMALITY

Ruby C. Weng

Statistics and Probability · · cited 5× · OpenAlex ↗ · more like this ·

Abstract: We propose a new method to derive posterior normality of stochastic processes. For a suitable parameter transformation Zt, the likelihood function is converted to a form close to a standard normal density. Then we apply a version of Stein’s Identity to obtain an expression for the posterior expectation. From this, posterior normality of Zt can be established. Applications of this method are illustrated by the conditional exponential family and a nonhomogeneous Poisson process. Key words and phrases: Maximum likelihood estimator, posterior distributions, posterior normality, Stein’s identity, stochastic processes. 1.

A Study of Expansions of Posterior Distributions

Ruby C. Weng, Cheng‐Hung Hsu

Statistics and Probability · Communication in Statistics- Theory and Methods · · cited 4× · DOI ↗ · more like this ·

Johnson (1970 Johnson , R. ( 1970 ). Asymptotic expansions associated with posterior distributions . Ann. Math. Statist. 41 : 851 – 864 .[Crossref] , [Google Scholar]) obtained expansions for marginal posterior distributions through Taylor expansions. Here, the posterior expansion is expressed in terms of the likelihood and the prior together with their derivatives. Recently, Weng (2010 Weng , R. C. ( 2010 ). A Bayesian Edgeworth expansion by Stein's Identity . Bayesian Anal. 5 ( 4 ): 741 – 764 .[Crossref], [Web of Science ®] , [Google Scholar]) used a version of Stein's identity to derive a Bayesian Edgeworth expansion, expressed by posterior moments. Since the pivots used in these two articles are the same, it is of interest to compare these two expansions. We found that our O(t −1/2) term agrees with Johnson's arithmetically, but the O(t −1) term does not. The simulations confirmed…

Very weak expansions for sequentially designed experiments.

Ruby C. Weng

Computational Theory and Mathematics · Deep Blue (University of Michigan) · · cited 4× · OpenAlex ↗ · more like this ·

Sequential methods were used to solve testing problems more efficiently. But at the same time, they make statistical inference more difficult, because they affect the sampling distributions of estimators. Typically, the effect on the sampling distributions is largest for small and moderate expected sample sizes and decreases as the expected sample sizes increases; but the rate of decrease may be slow. In this dissertation, we study the problem of setting confidence intervals following a sequential test. One main result provides approximate confidence intervals that are valid to a high order of magnitude for sequential samples from a multiparameter exponential family. This result is illustrated by an application to group sequential testing of two Poisson means. Simulations for this special case are conducted and they compare well with the theoretical results. The construction of these…

Adaptive deterministic annealing for two applications: competing SVR of switching dynamics and travelling salesman problems

Ming‐Wei Chang, Chih‐Jen Lin, Ruby C. Weng

Artificial Intelligence · NTUR (臺灣機構典藏) · · cited 2× · DOI ↗ · more like this ·

A deterministic annealing approach has been proposed to clustering by Rose et al. (1990; 1992) based on the maximum entropy principle. They obtain the association probabilities at a given average variance. The corresponding Lagrange multiplier is inversely related to the "temperature" and is used to control the annealing process. We propose an adaptive annealing schedule where the Lagrange multiplier is treated as an unknown parameter and is estimated by an expectation-maximization step. This technique is applied to using support vector regression (SVR) or switching dynamics. We also give some preliminary results on traveling salesman problems (TSP).

Bias approximations for likelihood‐based estimators

Ruby C. Weng, D. Stephen Coad

Statistics and Probability · Scandinavian Journal of Statistics · · cited 1× · DOI ↗ · more like this ·

Abstract Bias approximation has played an important rôle in statistical inference, and numerous bias calculation techniques have been proposed under different contexts. We provide a unified approach to approximating the bias of the maximum likelihood estimator and the l 2 penalized likelihood estimator for both linear and nonlinear models, where the design variables are allowed to be random and the sample size can be a stopping time. The proposed method is based on the Woodroofe–Stein identity and is justified by very weak approximations. The accuracy of the derived bias formulas is assessed by simulation for several examples. The bias of the ridge estimator in high‐dimensional settings is also discussed.

A Note on Item Response Theory Modeling for Online Customer Ratings

Chien-Lang Su, Sun-Hao Chang, Ruby C. Weng

Sociology and Political Science · The American Statistician · · cited 1× · DOI ↗ · more like this ·

Online consumer product ratings data are increasing rapidly. While most of the current graphical displays mainly represent the average ratings, Ho and Quinn proposed an easily interpretable graphical display based on an ordinal item response theory (IRT) model, which successfully accounts for systematic interrater differences. Conventionally, the discrimination parameters in IRT models are constrained to be positive, particularly in the modeling of scored data from educational tests. In this article, we use real-world ratings data to demonstrate that such a constraint can have a great impact on the parameter estimation. This impact on estimation was explained through rater behavior. We also discuss correlation among raters and assess the prediction accuracy for both the constrained and the unconstrained models. The results show that the unconstrained model performs better when a larger…

Improving unbiasedness of the proportional hazards model estimator with Cox and Snell's bias approximation and jackknife resampling

Chia‐Hui Huang, Ruby C. Weng

Statistics and Probability · Journal of Statistical Computation and Simulation · · cited 0× · DOI ↗ · more like this ·

Bias approximation has played an important role in the maximum likelihood estimation method, and numerous bias calculation techniques have been proposed under different contexts. For the semiparametric proportional hazards model, which is the standard regression method to study the time-to-event data, the existing work applied the bias formula and derived the approximate bias of Cox's estimator based on the partial likelihood function. In this work, we instead use the joint likelihood function and utilize the counting process approach to develop an approximate bias of Cox's estimator. Explicit expressions for the higher-order partial derivatives are derived, which facilitate the bias calculation techniques. We also incorporate the jackknife resampling method and propose a Jackknife-Cox-Snell method that processes the bias of Cox's estimator through two steps. The first step aims to…

C.J.: A Bayesian Approximation Method for Online Ranking

Ruby C. Weng, Chih-jen Lin, Thore Graepel

Artificial Intelligence · · cited 0× · OpenAlex ↗ · more like this ·

This paper describes a Bayesian approximation method to obtain online ranking algorithms for games with multiple teams and multiple players. Recently for Internet games large online ranking systems are much needed. We consider game models in which a k-team game is treated as several two-team games. By approximating the expectation of teams ’ (or players’) performances, we derive simple analytic update rules. These update rules, without numerical integrations, are very easy to interpret and implement. Experiments on game data show that the accuracy of our approach is competitive with state of the art systems such as TrueSkill, but the running time as well as the code is much shorter.

Abstract

Ming-wei Chang, Chih-jen Lin, Ruby C. Weng

Artificial Intelligence · · cited 0× · OpenAlex ↗ · more like this ·

This paper presents a simple and general approach for multi-label problems. In contrast with the commonly used binary approach, which roots from the 1-vs-rest strategy for multi-class classification, the new method extends the 1-vs-1 strategy and conducts pairwise training. While the binary approach assumes labels are assigned independently, the proposed method better incorporates information between any two labels. Behind this new framework any classification technique can be used as the base for training/predicting two-class sub-problems, but here we focus on using support vector machines (SVM). Experiments on real-world sets show that the new approach is usually better than the binary approach. 1

Trust region Newton method for large-scale logistic regression

Chih-jen Lin, Ruby C. Weng, S. Sathiya Keerthi +1 more

Artificial Intelligence · · cited 0× · OpenAlex ↗ · more like this ·

Large-scale logistic regression arises in many applications such as document classification and natural language processing. In this paper, we apply a trust region Newton method to maximize the log-likelihood of the logistic regression model. The proposed method uses only approximate Newton steps in the beginning, but achieves fast convergence in the end. Experiments show that it is faster than the commonly used quasi Newton approach for logistic regression. We also extend the proposed method to large-scale L2-loss linear support vector machines (SVM).

SEQUENTIAL CONFIDENCE INTERVALS FOR A POPULATION SIZE WITH FIXED PROPORTIONAL ACCURACY

Ki Heon Choi, Ruby C. Weng, Michael Woodroofe

Statistics and Probability · Sequential Analysis · · cited 0× · DOI ↗ · more like this ·

Consider a population of hidden objects of which the total number N is unknown. A search for the objects may be conducted in which the times at which the objects were found is recorded, along with the total number of objects found. From such data, a confidence interval for N is desired with specified proportional accuracy. Two sequential sampling plans are proposed for this problem, for two sets of underlying assumptions, and their properties are studied through asymptotic analysis and simulation.