arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Fu Yu

383 papers · 1998–2026

1998: 11999: 12000: 22001: 22002: 52003: 102004: 122005: 62006: 152007: 132008: 172009: 182010: 222011: 282012: 192013: 232014: 162015: 172016: 182017: 52018: 92019: 222020: 162021: 182022: 122023: 192024: 112025: 132026: 12
1998peak 28 in 20112026

Frequent co-authors

Papers

showing 1–25 of 383 RSS

Large-scale Multi-label Learning with Missing Labels

Hsiang-Fu Yu, Prateek Jain, Purushottam Kar +1 more

cs.LG · · cited 517× · arXiv ↗ · more like this ·

The multi-label classification problem has generated significant interest in recent years. However, existing approaches do not adequately address two key challenges: (a) the ability to tackle problems with a large number (say millions) of labels, and (b) the ability to handle data with missing labels. In this paper, we directly address both these problems by studying the multi-label problem in a generic empirical risk minimization (ERM) framework. Our framework, despite being simple, is surprisingly able to encompass several recent label-compression based methods which can be derived as special cases of our method. To optimize the ERM problem, we develop techniques that exploit the structure of specific loss functions - such as the squared loss function - to offer efficient algorithms. We further show that our learning framework admits formal excess risk bounds even in the presence of…

Temporal regularized matrix factorization for high-dimensional time series prediction

Hsiang‐Fu Yu, Nikhil Rao, Inderjit S. Dhillon

Signal Processing · Neural Information Processing Systems · · cited 324× · OpenAlex ↗ · more like this ·

Time series prediction problems are becoming increasingly high-dimensional in modern applications, such as climatology and demand forecasting. For example, in the latter problem, the number of items for which demand needs to be forecast might be as large as 50,000. In addition, the data is generally noisy and full of missing values. Thus, modern applications require methods that are highly scalable, and can deal with noisy data in terms of corruptions or missing values. However, classical time series methods usually fall short of handling these issues. In this paper, we present a temporal regularized matrix factorization (TRMF) framework which supports data-driven temporal learning and forecasting. We develop novel regularization schemes and use scalable matrix factorization methods that are eminently suited for high-dimensional time series data that has many missing values. Our…

Scalable Coordinate Descent Approaches to Parallel Matrix Factorization for Recommender Systems

Hsiang‐Fu Yu, Cho‐Jui Hsieh, Si Si +1 more

Information Systems · · cited 266× · DOI ↗ · more like this ·

Matrix factorization, when the matrix has missing values, has become one of the leading techniques for recommender systems. To handle web-scale datasets with millions of users and billions of ratings, scalability becomes an important issue. Alternating Least Squares (ALS) and Stochastic Gradient Descent (SGD) are two popular approaches to compute matrix factorization. There has been a recent flurry of activity to parallelize these algorithms. However, due to the cubic time complexity in the target rank, ALS is not scalable to large-scale datasets. On the other hand, SGD conducts efficient updates but usually suffers from slow convergence that is sensitive to the parameters. Coordinate descent, a classical optimization approach, has been used for many other large-scale problems, but its application to matrix factorization for recommender systems has not been explored thoroughly. In this…

Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction

Eli Chien, Wei-Cheng Chang, Cho-Jui Hsieh +4 more

cs.LG · · cited 207× · arXiv ↗ · more like this ·

Learning on graphs has attracted significant attention in the learning community due to numerous real-world applications. In particular, graph neural networks (GNNs), which take numerical node features and graph structure as inputs, have been shown to achieve state-of-the-art performance on various graph-related learning tasks. Recent works exploring the correlation between numerical node features and graph structure via self-supervised learning have paved the way for further performance improvements of GNNs. However, methods used for extracting numerical node features from raw data are still graph-agnostic within standard GNN pipelines. This practice is sub-optimal as it prevents one from fully utilizing potential correlations between graph topology and node attributes. To mitigate this issue, we propose a new self-supervised learning framework, Graph Information Aided Node feature…

Collaborative filtering with graph information: consistency and scalable methods

Nikhil Rao, Hsiang‐Fu Yu, Pradeep Ravikumar +1 more

Information Systems · Neural Information Processing Systems · · cited 193× · OpenAlex ↗ · more like this ·

Low rank matrix completion plays a fundamental role in collaborative filtering applications, the key idea being that the variables lie in a smaller subspace than the ambient space. Often, additional information about the variables is known, and it is reasonable to assume that incorporating this information will lead to better predictions. We tackle the problem of matrix completion when pairwise relationships among variables are known, via a graph. We formulate and derive a highly efficient, conjugate gradient based alternating minimization scheme that solves optimizations with over 55 million observations up to 2 orders of magni-tude faster than state-of-the-art (stochastic) gradient-descent based methods. On the theoretical front, we show that such methods generalize weighted nuclear norm formulations, and derive statistical consistency guarantees. We validate our results on both real…

Fast Multi-Resolution Transformer Fine-tuning for Extreme Multi-label Text Classification

Jiong Zhang, Wei-cheng Chang, Hsiang-fu Yu +1 more

cs.LG · · cited 146× · arXiv ↗ · more like this ·

Extreme multi-label text classification (XMC) seeks to find relevant labels from an extreme large label collection for a given text input. Many real-world applications can be formulated as XMC problems, such as recommendation systems, document tagging and semantic search. Recently, transformer based XMC methods, such as X-Transformer and LightXML, have shown significant improvement over other XMC methods. Despite leveraging pre-trained transformer models for text representation, the fine-tuning procedure of transformer models on large label space still has lengthy computational time even with powerful GPUs. In this paper, we propose a novel recursive approach, XR-Transformer to accelerate the procedure through recursively fine-tuning transformer models on a series of multi-resolution objectives related to the original XMC objective function. Empirical results show that XR-Transformer…

Learning to Encode Position for Transformer with Continuous Dynamical Model

Xuanqing Liu, Hsiang-Fu Yu, Inderjit Dhillon +1 more

cs.LG · · cited 145× · arXiv ↗ · more like this ·

We introduce a new way of learning to encode position information for non-recurrent models, such as Transformer models. Unlike RNN and LSTM, which contain inductive bias by loading the input tokens sequentially, non-recurrent models are less sensitive to position. The main reason is that position information among input units is not inherently encoded, i.e., the models are permutation equivalent; this problem justifies why all of the existing models are accompanied by a sinusoidal encoding/embedding layer at the input. However, this solution has clear limitations: the sinusoidal encoding is not flexible enough as it is manually designed and does not contain any learnable parameters, whereas the position embedding restricts the maximum length of input sequences. It is thus desirable to design a new position layer that contains learnable parameters to adjust to different datasets and…

NOMAD

Hyokun Yun, Hsiang‐Fu Yu, Cho‐Jui Hsieh +2 more

Artificial Intelligence · Proceedings of the VLDB Endowment · · cited 141× · DOI ↗ · more like this ·

We develop an efficient parallel distributed algorithm for matrix completion, named NOMAD (Non-locking, stOchastic Multi-machine algorithm for Asynchronous and Decentralized matrix completion). NOMAD is a decentralized algorithm with non-blocking communication between processors. One of the key features of NOMAD is that the ownership of a variable is asynchronously transferred between processors in a decentralized fashion. As a consequence it is a lock-free parallel algorithm. In spite of being asynchronous, the variable updates of NOMAD are serializable, that is, there is an equivalent update ordering in a serial implementation. NOMAD outperforms synchronous algorithms which require explicit bulk synchronization after every iteration: our extensive empirical evaluation shows that not only does our algorithm perform well in distributed setting on commodity hardware, but also outperforms…

Feature Engineering and Classifier Ensemble for KDD Cup 2010

Hsiang‐Fu Yu, Hung-Yi Lo, Hsun-Ping Hsieh +9 more

Artificial Intelligence · Knowledge Discovery and Data Mining · · cited 128× · OpenAlex ↗ · more like this ·

KDD Cup 2010 is an educational data mining competition. Participants are asked to learn a model from students ’ past behavior and then predict their future performance. At National Taiwan University, we organized a course for this competition. Most student sub-teams expanded features by various binarization and discretization techniques. The resulting sparse feature sets were trained by logistic regression (using LIBLINEAR). One sub-team considered condensed features using simple statistical techniques and applied Random Forest (through Weka) for training. Initial development was conducted on an internal split of training data for training and validation. We identified some useful feature combinations to improve performance. For the final submission, we combined results of student sub-teams by regularized linear regression. Our team is the first prize winner of both tracks (all teams…

NOMAD: Non-locking, stOchastic Multi-machine algorithm for Asynchronous and Decentralized matrix completion

Hyokun Yun, Hsiang-Fu Yu, Cho-Jui Hsieh +2 more

cs.DC · · cited 123× · arXiv ↗ · more like this ·

We develop an efficient parallel distributed algorithm for matrix completion, named NOMAD (Non-locking, stOchastic Multi-machine algorithm for Asynchronous and Decentralized matrix completion). NOMAD is a decentralized algorithm with non-blocking communication between processors. One of the key features of NOMAD is that the ownership of a variable is asynchronously transferred between processors in a decentralized fashion. As a consequence it is a lock-free parallel algorithm. In spite of being an asynchronous algorithm, the variable updates of NOMAD are serializable, that is, there is an equivalent update ordering in a serial implementation. NOMAD outperforms synchronous algorithms which require explicit bulk synchronization after every iteration: our extensive empirical evaluation shows that not only does our algorithm perform well in distributed setting on commodity hardware, but…

PASSCoDe: Parallel ASynchronous Stochastic dual Co-ordinate Descent

Cho-Jui Hsieh, Hsiang-Fu Yu, Inderjit S. Dhillon

cs.LG · · cited 109× · arXiv ↗ · more like this ·

Stochastic Dual Coordinate Descent (SDCD) has become one of the most efficient ways to solve the family of $\ell_2$-regularized empirical risk minimization problems, including linear SVM, logistic regression, and many others. The vanilla implementation of DCD is quite slow; however, by maintaining primal variables while updating dual variables, the time complexity of SDCD can be significantly reduced. Such a strategy forms the core algorithm in the widely-used LIBLINEAR package. In this paper, we parallelize the SDCD algorithms in LIBLINEAR. In recent research, several synchronized parallel SDCD algorithms have been proposed, however, they fail to achieve good speedup in the shared memory multi-core setting. In this paper, we propose a family of asynchronous stochastic dual coordinate descent algorithms (ASDCD). Each thread repeatedly selects a random dual variable and conducts…

PECOS: Prediction for Enormous and Correlated Output Spaces

Hsiang-Fu Yu, Kai Zhong, Jiong Zhang +2 more

cs.LG · · cited 100× · arXiv ↗ · more like this ·

Many large-scale applications amount to finding relevant results from an enormous output space of potential candidates. For example, finding the best matching product from a large catalog or suggesting related search phrases on a search engine. The size of the output space for these problems can range from millions to billions, and can even be infinite in some applications. Moreover, training data is often limited for the long-tail items in the output space. Fortunately, items in the output space are often correlated thereby presenting an opportunity to alleviate the data sparsity issue. In this paper, we propose the Prediction for Enormous and Correlated Output Spaces (PECOS) framework, a versatile and modular machine learning framework for solving prediction problems for very large output spaces, and apply it to the eXtreme Multilabel Ranking (XMR) problem: given an input instance…

Data Gathering in Wireless Sensor Networks: A Combine–TSP–Reduce Approach

Chien‐Fu Cheng, Chao-Fu Yu

Computer Networks and Communications · IEEE Transactions on Vehicular Technology · · cited 76× · DOI ↗ · more like this ·

Mobile sinks are extensively used for data gathering in wireless sensor networks (WSNs). This method avoids imbalances in energy consumption caused by multihop transmission but may cause an extended delay time. In this paper, we focus on how to shorten the length of the traveling path to reduce the delay time of data gathering. We propose that the mobile sink visits the overlapping areas of communication ranges of sensors instead of sensors one by one. Next, we determine the visiting point of each overlapping area and use the traveling salesperson problem (TSP) algorithm to plan a traveling path. Because the visiting point is a point within the overlapping area of communication ranges of sensors, it is possible that the length of the traveling path can be reduced further. Hence, we attempt to shorten the traveling path obtained by the TSP algorithm. The benefit of the proposed method is…

Extreme Multi-label Learning for Semantic Matching in Product Search

Wei-Cheng Chang, Daniel Jiang, Hsiang-Fu Yu +9 more

cs.IR · · cited 72× · arXiv ↗ · more like this ·

We consider the problem of semantic matching in product search: given a customer query, retrieve all semantically related products from a huge catalog of size 100 million, or more. Because of large catalog spaces and real-time latency constraints, semantic matching algorithms not only desire high recall but also need to have low latency. Conventional lexical matching approaches (e.g., Okapi-BM25) exploit inverted indices to achieve fast inference time, but fail to capture behavioral signals between queries and products. In contrast, embedding-based models learn semantic representations from customer behavior data, but the performance is often limited by shallow neural encoders due to latency constraints. Semantic product search can be viewed as an eXtreme Multi-label Classification (XMC) problem, where customer queries are input instances and products are output labels. In this paper…

Large linear classification when data cannot fit in memory

Hsiang‐Fu Yu, Cho‐Jui Hsieh, Kai‐Wei Chang +1 more

Computer Vision and Pattern Recognition · · cited 71× · DOI ↗ · more like this ·

Recent advances in linear classification have shown that for applications such as document classification, the training can be extremely efficient. However, most of the existing training methods are designed by assuming that data can be stored in the computer memory. These methods cannot be easily applied to data larger than the memory capacity due to the random access to the disk. We propose and analyze a block minimization framework for data larger than the memory size. At each step a block of data is loaded from the disk and handled by certain learning methods. We investigate two implementations of the proposed framework for primal and dual SVMs, respectively. As data cannot fit in memory, many design considerations are very different from those for traditional algorithms. Experiments using data sets 20 times larger than the memory demonstrate the effectiveness of the proposed method.

A Scalable Asynchronous Distributed Algorithm for Topic Modeling

Hsiang-Fu Yu, Cho-Jui Hsieh, Hyokun Yun +2 more

cs.DC · · cited 70× · arXiv ↗ · more like this ·

Learning meaningful topic models with massive document collections which contain millions of documents and billions of tokens is challenging because of two reasons: First, one needs to deal with a large number of topics (typically in the order of thousands). Second, one needs a scalable and efficient way of distributing the computation across multiple machines. In this paper we present a novel algorithm F+Nomad LDA which simultaneously tackles both these problems. In order to handle large number of topics we use an appropriately modified Fenwick tree. This data structure allows us to sample from a multinomial distribution over $T$ items in $O(\log T)$ time. Moreover, when topic counts change the data structure can be updated in $O(\log T)$ time. In order to distribute the computation across multiple processor we present a novel asynchronous framework inspired by the Nomad algorithm of…

Selection of Negative Samples for One-class Matrix Factorization

Hsiang‐Fu Yu, Mikhail Bilenko, Chih‐Jen Lin

Information Systems · Society for Industrial and Applied Mathematics eBooks · · cited 64× · DOI ↗ · more like this ·

Many recommender systems have only implicit user feedback. The two possible ratings are positive and negative, but only part of positive entries are observed. One-class matrix factorization (MF) is a popular approach for such scenarios by treating some missing entries as negative. Two major ways to select negative entries are by sub-sampling a set with similar size to that of observed positive entries or by including all missing entries as negative. They are referred to as “subsampled” and “full” approaches in this work, respectively. Currently detailed comparisons between these two selection schemes on large-scale data are still lacking. One important reason is that the “full” approach leads to a hard optimization problem after treating all missing entries as negative. In this paper, we successfully develop efficient optimization techniques to solve this challenging problem so that the…

A Greedy Approach for Budgeted Maximum Inner Product Search

Hsiang-Fu Yu, Cho-Jui Hsieh, Qi Lei +1 more

cs.DS · · cited 48× · arXiv ↗ · more like this ·

Maximum Inner Product Search (MIPS) is an important task in many machine learning applications such as the prediction phase of a low-rank matrix factorization model for a recommender system. There have been some works on how to perform MIPS in sub-linear time recently. However, most of them do not have the flexibility to control the trade-off between search efficient and search quality. In this paper, we study the MIPS problem with a computational budget. By carefully studying the problem structure of MIPS, we develop a novel Greedy-MIPS algorithm, which can handle budgeted MIPS by design. While simple and intuitive, Greedy-MIPS yields surprisingly superior performance compared to state-of-the-art approaches. As a specific example, on a candidate set containing half a million vectors of dimension 200, Greedy-MIPS runs 200x faster than the naive approach while yielding search results…

Reverse Fast Broadcasting (RFB) for Video-on-Demand Applications

Hsiang‐Fu Yu, Hung-Chang Yang, Li‐Ming Tseng

Sociology and Political Science · IEEE Transactions on Broadcasting · · cited 39× · DOI ↗ · more like this ·

A popular video can be broadcast by partitioning the video into segments, which are broadcast on several channels simultaneously and periodically. This method allows multiple users to share channels, leading to higher bandwidth utilization. Previous studies mainly focus on reducing viewers' waiting time. This work studies another important issue, namely client buffer. When the term "client buffer" is used, it means the worst-case buffer requirements (i.e., the maximum value). savings. A reverse fast broadcasting (RFB) scheme is proposed to alleviate the buffer problem. There are two specific properties for RFB. First, this scheme arranges the segments in descending order of their numeric indices on each channel at the server end. Second, RFB requires a client to receive segments as late as possible, such that the client buffers the smallest number of segments. RFB has the same waiting…

On the supersingular locus in Hilbert-Blumenthal 4-folds

Chia‐Fu Yu

Mathematical Physics · Journal of Algebraic Geometry · · cited 34× · DOI ↗ · more like this ·

We study the supersingular locus of Hilbert-Blumenthal four-folds modulo <inline-formula content-type="math/mathml"> <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" alttext="p"> <mml:semantics> <mml:mi>p</mml:mi> <mml:annotation encoding="application/x-tex">p</mml:annotation> </mml:semantics> </mml:math> </inline-formula> when <inline-formula content-type="math/mathml"> <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" alttext="p"> <mml:semantics> <mml:mi>p</mml:mi> <mml:annotation encoding="application/x-tex">p</mml:annotation> </mml:semantics> </mml:math> </inline-formula> is inert in the totally real field. The dimension, local moduli spaces, number of the irreducible components, and a description of intersections of these components are given. We also show that each irreducible component is a smooth algebraic stack which is a quotient of a ruled surface over…

Extreme Zero-Shot Learning for Extreme Text Classification

Yuanhao Xiong, Wei-Cheng Chang, Cho-Jui Hsieh +2 more

cs.LG · · cited 33× · arXiv ↗ · more like this ·

The eXtreme Multi-label text Classification (XMC) problem concerns finding most relevant labels for an input text instance from a large label set. However, the XMC setup faces two challenges: (1) it is not generalizable to predict unseen labels in dynamic environments, and (2) it requires a large amount of supervised (instance, label) pairs, which can be difficult to obtain for emerging domains. Recently, the generalized zero-shot XMC (GZ-XMC) setup has been studied and ZestXML is proposed accordingly to handle the unseen labels, which still requires a large number of annotated (instance, label) pairs. In this paper, we consider a more practical scenario called Extreme Zero-Shot XMC (EZ-XMC), in which no supervision is needed and merely raw text of instances and labels are accessible. Few-Shot XMC (FS-XMC), an extension to EZ-XMC with limited supervision is also investigated. To learn…

Supersingular Kottwitz-Rapoport strata and Deligne-Lusztig varieties

Ulrich Goertz, Chia-Fu Yu

math.AG · · cited 30× · arXiv ↗ · more like this ·

We investigate Siegel modular varieties in positive characteristic with Iwahori level structure. On these spaces, we have the Newton stratification, and the Kottwitz-Rapoport stratification; one would like to understand how these stratifications are related to each other. We give a simple description of all KR strata which are entirely contained in the supersingular locus as disjoint unions of Deligne-Lusztig varieties. We also give an explicit numerical description of the KR stratification in terms of abelian varieties.

X-BERT: eXtreme Multi-label Text Classification with using Bidirectional Encoder Representations from Transformers

Wei-Cheng Chang, Hsiang‐Fu Yu, Kai Zhong +2 more

Artificial Intelligence · · cited 29× · OpenAlex ↗ · more like this ·

We consider the extreme multi-label text classification (XMC) problem: given an input text, return the most relevant labels from a large label collection. For example, the input text could be a product description on this http URL and the labels could be product categories. XMC is an important yet challenging problem in the NLP community. Recently, deep pretrained transformer models have achieved state-of-the-art performance on many NLP tasks including sentence classification, albeit with small label sets. However, naively applying deep transformer models to the XMC problem leads to sub-optimal performance due to the large output space and the label sparsity issue. In this paper, we propose X-Transformer, the first scalable approach to fine-tuning deep transformer models for the XMC problem. The proposed method achieves new state-of-the-art results on four XMC benchmark datasets. In…

Simple mass formulas on Shimura varieties of PEL-type

Chia-Fu Yu

math.NT · · cited 29× · arXiv ↗ · more like this ·

We give a unified formulation of a mass for arbitrary abelian varieties with PEL-structures and show that it equals a weighted class number of a reductive $\Q$-group $G$ relative to an open compact subgroup $U$ of $G(\A_f)$, or simply called an {\it arithmetic mass}. We classify the special objects for which our formulation remains valid over algebraic closed fields. As a result, we show that the set of basic points in a mod $p$ moduli space of PEL-type with a local condition (and a mild condition subject to the Hasse principle) can be expressed as a double coset space and its mass equals an arithmetic mass. The moduli space does not need to have good reduction at $p$. This generalizes a well-known result for superspecial abelian varieties.