arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Charu

1,126 papers · 1966–2026

1967: 01968: 01969: 01970: 01971: 01972: 11973: 01974: 01975: 01976: 01977: 01978: 01979: 01980: 01981: 01982: 01983: 01984: 01985: 01986: 01987: 01988: 01989: 01990: 01991: 01992: 21993: 01994: 01995: 01996: 41997: 21998: 91999: 42000: 62001: 222002: 202003: 142004: 72005: 102006: 102007: 202008: 222009: 112010: 222011: 272012: 382013: 382014: 462015: 572016: 452017: 602018: 562019: 572020: 522021: 542022: 552023: 942024: 1022025: 962026: 61
1967peak 102 in 20242026

Frequent co-authors

Papers

showing 1–25 of 1,126 RSS

Outlier detection for high dimensional data

Charų C. Aggarwal, Philip S. Yu

Artificial Intelligence · · cited 1,034× · DOI ↗ · more like this ·

The outlier detection problem has important applications in the field of fraud detection, network robustness analysis, and intrusion detection. Most such applications are high dimensional domains in which the data can contain hundreds of dimensions. Many recent algorithms use concepts of proximity in order to find outliers based on their relationship to the rest of the data. However, in high dimensional space, the data is sparse and the notion of proximity fails to retain its meaningfulness. In fact, the sparsity of high dimensional data implies that every point is an almost equally good outlier from the perspective of proximity-based definitions. Consequently, for high dimensional data, the notion of finding meaningful outliers becomes substantially more complex and non-obvious. In this paper, we discuss new techniques for outlier detection which find the outliers by studying the…

On the design and quantification of privacy preserving data mining algorithms

Dakshi Agrawal, Charų C. Aggarwal

Artificial Intelligence · · cited 1,018× · DOI ↗ · more like this ·

The increasing ability to track and collect large amounts of data with the use of current hardware technology has lead to an interest in the development of data mining algorithms which preserve user privacy. A recently proposed technique addresses the issue of privacy preservation by perturbing the data and reconstructing distributions at an aggregate level in order to perform the mining. This method is able to retain privacy while accessing the information implicit in the original attributes. The distribution reconstruction process naturally leads to some loss of information which is acceptable in many practical situations. This paper discusses an Expectation Maximization (EM) algorithm for distribution reconstruction which is more effective than the currently available method in terms of the level of information loss. Specifically, we prove that the EM algorithm converges to the…

Outlier Detection for Temporal Data: A Survey

Manish Gupta, Jing Gao, Charų C. Aggarwal +1 more

Artificial Intelligence · IEEE Transactions on Knowledge and Data Engineering · · cited 983× · DOI ↗ · more like this ·

In the statistics community, outlier detection for time series data has been studied for decades. Recently, with advances in hardware and software technology, there has been a large body of work on temporal outlier detection from a computational perspective within the computer science community. In particular, advances in hardware technology have enabled the availability of various forms of temporal data collection mechanisms, and advances in software technology have enabled a variety of data management mechanisms. This has fueled the growth of different kinds of data sets such as data streams, spatio-temporal data, distributed streams, temporal networks, and time series data, generated by a multitude of applications. There arises a need for an organized and detailed study of the work done in the area of outlier detection with respect to such temporal datasets. In this survey, we…

Fast algorithms for projected clustering

Charų C. Aggarwal, Joel L. Wolf, Philip S. Yu +2 more

Artificial Intelligence · ACM SIGMOD Record · · cited 903× · DOI ↗ · more like this ·

The clustering problem is well known in the database literature for its numerous applications in problems such as customer segmentation, classification and trend analysis. Unfortunately, all known algorithms tend to break down in high dimensional spaces because of the inherent sparsity of the points. In such high dimensional spaces not all dimensions may be relevant to a given cluster. One way of handling this is to pick the closely correlated dimensions and find clusters in the corresponding subspace. Traditional feature selection algorithms attempt to achieve this. The weakness of this approach is that in typical high dimensional data mining applications different sets of points may cluster better for different subsets of dimensions. The number of dimensions in each such cluster-specific subspace may also vary. Hence, it may be impossible to find a single small subset of dimensions…

Deep Learning for Time Series Anomaly Detection: A Survey

Zahra Zamanzadeh Darban, Geoffrey I. Webb, Shirui Pan +2 more

cs.LG · · cited 775× · arXiv ↗ · more like this ·

Time series anomaly detection has applications in a wide range of research fields and applications, including manufacturing and healthcare. The presence of anomalies can indicate novel or unexpected events, such as production faults, system defects, or heart fluttering, and is therefore of particular interest. The large size and complex patterns of time series have led researchers to develop specialised deep learning models for detecting anomalous patterns. This survey focuses on providing structured and comprehensive state-of-the-art time series anomaly detection models through the use of deep learning. It providing a taxonomy based on the factors that divide anomaly detection models into different categories. Aside from describing the basic anomaly detection technique for each category, the advantages and limitations are also discussed. Furthermore, this study includes examples of…

Learning Attention-based Embeddings for Relation Prediction in
\nKnowledge Graphs

Deepak Nathani, Jatin Chauhan, Charu Sharma +1 more

Artificial Intelligence · Research Archive of Indian Institute of Technology Hyderabad (Indian Institute of Technology Hyderabad) · · cited 625× · OpenAlex ↗ · more like this ·

The recent proliferation of knowledge graphs
\n(KGs) coupled with incomplete or partial information, in the form of missing relations
\n(links) between entities, has fueled a lot of
\nresearch on knowledge base completion (also
\nknown as relation prediction). Several recent works suggest that convolutional neural
\nnetwork (CNN) based models generate richer
\nand more expressive feature embeddings and
\nhence also perform well on relation prediction.
\nHowever, we observe that these KG embeddings treat triples independently and thus fail
\nto cover the complex and hidden information
\nthat is inherently implicit in the local neighborhood surrounding a triple. To this effect, our
\npaper proposes a novel attention-based feature
\nembedding that captures both entity and relation features in any given entity’s neighborhood. Additionally, we…

On k -anonymity and the curse of dimensionality

Charų C. Aggarwal

Artificial Intelligence · · cited 594× · OpenAlex ↗ · more like this ·

In recent years, the wide availability of personal data has made the problem of privacy preserving data mining an important one. A number of methods have recently been proposed for privacy preserving data mining of multidimensional data records. One of the methods for privacy preserving data mining is that of anonymization, in which a record is released only if it is indistinguishable from k other entities in the data. We note that methods such as k-anonymity are highly dependent upon spatial locality in order to e#ectively implement the technique in a statistically robust way. In high dimensional space the data becomes sparse, and the concept of spatial locality is no longer easy to define from an application point of view. In this paper, we view the k-anonymization problem from the perspective of inference attacks over all possible combinations of attributes. We show that when the…

Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs

Deepak Nathani, Jatin Chauhan, Charu Sharma +1 more

cs.LG · · cited 588× · arXiv ↗ · more like this ·

The recent proliferation of knowledge graphs (KGs) coupled with incomplete or partial information, in the form of missing relations (links) between entities, has fueled a lot of research on knowledge base completion (also known as relation prediction). Several recent works suggest that convolutional neural network (CNN) based models generate richer and more expressive feature embeddings and hence also perform well on relation prediction. However, we observe that these KG embeddings treat triples independently and thus fail to cover the complex and hidden information that is inherently implicit in the local neighborhood surrounding a triple. To this effect, our paper proposes a novel attention based feature embedding that captures both entity and relation features in any given entity's neighborhood. Additionally, we also encapsulate relation clusters and multihop relations in our model.…

Finding generalized projected clusters in high dimensional spaces

Charų C. Aggarwal, Philip S. Yu

Artificial Intelligence · ACM SIGMOD Record · · cited 532× · DOI ↗ · more like this ·

High dimensional data has always been a challenge for clustering algorithms because of the inherent sparsity of the points. Recent research results indicate that in high dimensional data, even the concept of proximity or clustering may not be meaningful. We discuss very general techniques for projected clustering which are able to construct clusters in arbitrarily aligned subspaces of lower dimensionality. The subspaces are specific to the clusters themselves. This definition is substantially more general and realistic than currently available techniques which limit the method to only projections from the original set of attributes. The generalized projected clustering technique may also be viewed as a way of trying to redefine clustering for high dimensional applications by searching for hidden subspaces with clusters which are created by inter-attribute correlations. We provide a new…

What Is the Nearest Neighbor in High Dimensional Spaces

Alexander Hinneburg, Charų C. Aggarwal, Daniel A. Keim

Signal Processing · KOPS (University of Konstanz) · · cited 497× · OpenAlex ↗ · more like this ·

Nearest neighbor search in high dimensional spaces is an interesting and important problem which is relevant for a wide varietyofnovel database applications. As recentresults show, however, the problem is a very difficult one, not only with regards to the performance issue but also to the quality issue. In this paper, we discuss the quality issue and identify a new generalized notion of nearest neighbor search as the relevant problem in high dimensional space. In contrast to previous approaches, our new notion of nearest neighbor search does not treat all dimensions equally but uses a quality criterion to select relevant dimensions (projections) with respect to the given query. As an example for a useful quality criterion, we rate howwell the data is clustered around the query point within the selected projection. We then propose an efficient and effectivealgorithm to solve the…

A Survey of Uncertain Data Algorithms and Applications

Charų C. Aggarwal, Philip S. Yu

Signal Processing · IEEE Transactions on Knowledge and Data Engineering · · cited 471× · DOI ↗ · more like this ·

In recent years, a number of indirect data collection methodologies have lead to the proliferation of uncertain data. Such data points are often represented in the form of a probabilistic function, since the corresponding deterministic value is not known. This increases the challenge of mining and managing uncertain data, since the precise behavior of the underlying data is no longer known. In this paper, we provide a survey of uncertain data mining and management applications. In the field of uncertain data management, we will examine traditional methods such as join processing, query processing, selectivity estimation, OLAP queries, and indexing. In the field of uncertain data mining, we will examine traditional mining problems such as classification and clustering. We will also examine a general transform based technique for mining uncertain data. We discuss the models for uncertain…

Outlier Detection with Autoencoder Ensembles

Jinghui Chen, Saket Sathe, Charų C. Aggarwal +1 more

Artificial Intelligence · Society for Industrial and Applied Mathematics eBooks · · cited 450× · DOI ↗ · more like this ·

In this paper, we introduce autoencoder ensembles for unsupervised outlier detection. One problem with neural networks is that they are sensitive to noise and often require large data sets to work robustly, while increasing data size makes them slow. As a result, there are only a few existing works in the literature on the use of neural networks in outlier detection. This paper shows that neural networks can be a very competitive technique to other existing methods. The basic idea is to randomly vary on the connectivity architecture of the autoencoder to obtain significantly better performance. Furthermore, we combine this technique with an adaptive sampling method to make our approach more efficient and effective. Experimental results comparing the proposed approach with state-of-the-art detectors are presented on several benchmark data sets showing the accuracy of our approach.

Horting hatches an egg

Charų C. Aggarwal, Joel L. Wolf, Kun‐Lung Wu +1 more

Information Systems · · cited 388× · DOI ↗ · more like this ·

Article Free Access Share on Horting hatches an egg: a new graph-theoretic approach to collaborative filtering Authors: Charu C. Aggarwal IBM T. J. Watson Research Center, Yorktown Heights, NY IBM T. J. Watson Research Center, Yorktown Heights, NYView Profile , Joel L. Wolf IBM T. J. Watson Research Center, Yorktown Heights, NY IBM T. J. Watson Research Center, Yorktown Heights, NYView Profile , Kun-Lung Wu IBM T. J. Watson Research Center, Yorktown Heights, NY IBM T. J. Watson Research Center, Yorktown Heights, NYView Profile , Philip S. Yu IBM T. J. Watson Research Center, Yorktown Heights, NY IBM T. J. Watson Research Center, Yorktown Heights, NYView Profile Authors Info & Claims KDD '99: Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data miningAugust 1999 Pages 201–212https://doi.org/10.1145/312129.312230Published:01 August 1999Publication…

Graph Convolutional Networks with EigenPooling

Yao Ma, Suhang Wang, Charu C. Aggarwal +1 more

cs.LG · · cited 371× · arXiv ↗ · more like this ·

Graph neural networks, which generalize deep neural network models to graph structured data, have attracted increasing attention in recent years. They usually learn node representations by transforming, propagating and aggregating node features and have been proven to improve the performance of many graph related tasks such as node classification and link prediction. To apply graph neural networks for the graph classification task, approaches to generate the \textit{graph representation} from node representations are demanded. A common way is to globally combine the node representations. However, rich structural information is overlooked. Thus a hierarchical pooling procedure is desired to preserve the graph structure during the graph representation learning. There are some recent works on hierarchically learning graph representation analogous to the pooling step in conventional…

Caching on the World Wide Web

Charų C. Aggarwal, Joel L. Wolf, Philip S. Yu

Computer Networks and Communications · IEEE Transactions on Knowledge and Data Engineering · · cited 352× · DOI ↗ · more like this ·

With the recent explosion in usage of the World Wide Web, the problem of caching Web objects has gained considerable importance. Caching on the Web differs from traditional caching in several ways. The nonhomogeneity of the object sizes is probably the most important such difference. In this paper, we give an overview of caching policies designed specifically for Web objects and provide a new algorithm of our own. This new algorithm can be regarded as a generalization of the standard LRU algorithm. We examine the performance of this and other Web caching algorithms via event- and trace-driven simulation.

Depth first generation of long patterns

Ramesh C. Agarwal, Charų C. Aggarwal, Vara Prasad

Information Systems · · cited 322× · DOI ↗ · more like this ·

In this paper we present an algorithm for mining long patterns in databases. The algorithm finds large itemsets by using depth first search on a lexicographic tree of itemsets. The focus of this paper is to develop CPU-efficient algorithms for finding frequent itemsets in the cases when the database contains patterns which are very wide. We refer to this algorithm as DepthProject, and it achieves more than one order of magnitude speedup over the recently proposed MaxMiner algorithm for finding long patterns. These techniques may be quite useful for applications in areas such as computational biology in which the number of records is relatively small, but the itemsets are very long. This necessitates the discovery of patterns using algorithms which are especially tailored to the nature of such domains.

Frequent pattern mining with uncertain data

Charų C. Aggarwal, Yan Li, Jianyong Wang +1 more

Information Systems · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · · cited 296× · DOI ↗ · more like this ·

This paper studies the problem of frequent pattern mining with uncertain data. We will show how broad classes of algorithms can be extended to the uncertain data setting. In particular, we will study candidate generate-and-test algorithms, hyper-structure algorithms and pattern growth based algorithms. One of our insightful observations is that the experimental behavior of different classes of algorithms is very different in the uncertain case as compared to the deterministic case. In particular, the hyper-structure and the candidate generate-and-test algorithms perform much better than tree-based algorithms. This counter-intuitive behavior is an important observation from the perspective of algorithm design of the uncertain variation of the problem. We will test the approach on a number of real and synthetic data sets, and show the effectiveness of two of our approaches over…

Theoretical Foundations and Algorithms for Outlier Ensembles

Charų C. Aggarwal, Saket Sathe

Artificial Intelligence · ACM SIGKDD Explorations Newsletter · · cited 284× · DOI ↗ · more like this ·

Ensemble analysis has recently been studied in the context of the outlier detection problem. In this paper, we investigate the theoretical underpinnings of outlier ensemble analysis. In spite of the significant differences between the classification and the outlier analysis problems, we show that the theoretical underpinnings between the two problems are actually quite similar in terms of the bias-variance trade-off. We explain the existing algorithms within this traditional framework, and clarify misconceptions about the reasoning underpinning these methods. We propose more effective variants of subsampling and feature bagging. We also discuss the impact of the combination function and discuss the specific trade-offs of the average and maximization functions. We use these insights to propose new combination functions that are robust in many settings.

Human mobility and the spatial transmission of influenza in the United States

Vivek Charu, Scott L. Zeger, Julia R. Gog +5 more

Modeling and Simulation · PLoS Computational Biology · · cited 263× · DOI ↗ · more like this ·

Seasonal influenza epidemics offer unique opportunities to study the invasion and re-invasion waves of a pathogen in a partially immune population. Detailed patterns of spread remain elusive, however, due to lack of granular disease data. Here we model high-volume city-level medical claims data and human mobility proxies to explore the drivers of influenza spread in the US during 2002-2010. Although the speed and pathways of spread varied across seasons, seven of eight epidemics likely originated in the Southern US. Each epidemic was associated with 1-5 early long-range transmission events, half of which sparked onward transmission. Gravity model estimates indicate a sharp decay in influenza transmission with the distance between infectious and susceptible cities, consistent with spread dominated by work commutes rather than air traffic. Two early-onset seasons associated with antigenic…

On demand classification of data streams

Charų C. Aggarwal, Jiawei Han, Jianyong Wang +1 more

Artificial Intelligence · · cited 260× · DOI ↗ · more like this ·

Current models of the classification problem do not effectively handle bursts of particular classes coming in at different times. In fact, the current model of the classification problem simply concentrates on methods for one-pass classification modeling of very large data sets. Our model for data stream classification views the data stream classification problem from the point of view of a dynamic approach in which simultaneous training and testing streams are used for dynamic classification of data sets. This model reflects real life situations effectively, since it is desirable to classify test streams in real time over an evolving training and test stream. The aim here is to create a classification system in which the training model can adapt quickly to the changes of the underlying data stream. In order to achieve this goal, we propose an on-demand classification process which can…

Intelligent crawling on the World Wide Web with arbitrary predicates

Charų C. Aggarwal, Fatima Al-Garawi, Philip S. Yu

Information Systems · · cited 256× · DOI ↗ · more like this ·

The enormous growth of the world wide web in recent years has made it important to perform resource discovery efficiently. Consequently, several new ideas have been proposed in recent years, among them a key technique is focused crawling which is able to crawl particular topical portions of the world wide web quickly without having to explore all web pages. In this paper, we propose the novel concept of intelligent crawling which actually learns characteristics of the linkage structure of the world wide web while performing the crawling. Specifically, the intelligent crawler uses the inlinking web page content, candidate URL structure, or other behaviors of the inlinking web pages or siblings in order to estimate the probability that a candidate is useful for a given crawl. This is amuch more general framework than the focused crawling technique which is based on a pre-defined…

Signed Network Embedding in Social Media

Suhang Wang, Jiliang Tang, Charų C. Aggarwal +2 more

Artificial Intelligence · Society for Industrial and Applied Mathematics eBooks · · cited 247× · DOI ↗ · more like this ·

Network embedding is to learn low-dimensional vector representations for nodes of a given social network, facilitating many tasks in social network analysis such as link prediction. The vast majority of existing embedding algorithms are designed for unsigned social networks or social networks with only positive links. However, networks in social media could have both positive and negative links, and little work exists for signed social networks. From recent findings of signed network analysis, it is evident that negative links have distinct properties and added value besides positive links, which brings about both challenges and opportunities for signed network embedding. In this paper, we propose a deep learning framework SiNE for signed network embedding. The framework optimizes an objective function guided by social theories that provide a fundamental understanding of signed social…

On optimal batching policies for video-on-demand storage servers

Charų C. Aggarwal, Joel L. Wolf, Philip S. Yu

Computer Networks and Communications · · cited 245× · DOI ↗ · more like this ·

In a video-on-demand environment, batching of video requests is often used to reduce I/O demand and improve throughput. Since viewers may defect if they experience long waits, a good video scheduling policy needs to consider not only the batch size but also the viewer defection probabilities and wait times. Two conventional scheduling policies for batching are first-come-first-served (FCFS) and maximum queue length (MOL). Neither of these policies lead to entirely satisfactory results. MQL tends to be too aggressive in scheduling popular videos by only considering the queue length to maximize batch size, while FCFS has the opposite effect. We introduce the notion of factored queue length and propose a batching policy that schedules the video with the maximum factored queue length. We refer to this as the MFQ policy. The factored queue length is obtained by weighting each video queue…

A permutation-based pyramid broadcasting scheme for video-on-demand systems

Charų C. Aggarwal, Joel L. Wolf, Philip S. Yu

Sociology and Political Science · · cited 244× · DOI ↗ · more like this ·

Periodic broadcasting can be used to support near video on demand for popular videos. For a given bandwidth allocation, pyramid broadcasting schemes substantially reduce the viewer latency (waiting) time as compared with conventional broadcasting schemes. Nevertheless, such pyramid schemes typically have substantial storage requirements at the client end, and this results in set top boxes needing disks with high transfer rate capabilities. We present a permutation based pyramid scheme in which the storage requirements and disk transfer rates are greatly reduced, and yet the viewer latency is smaller as well. Under the proposed approach, each video is partitioned into contiguous segments of geometrically increasing sizes and each segment is further divided into blocks, where a block is the basic unit of transmission. As in the original pyramid scheme, frequencies of transmission for the…

A new framework for itemset generation

Charų C. Aggarwal, Philip S. Yu

Information Systems · · cited 244× · DOI ↗ · more like this ·

The problem of finding association rules in a large database of sales transactions has been widely studied in the literature, We discuss some of the weaknesses of the large itemset method for association rule generation. A different method for evaluating and finding itemsets referred to as otrongIyl collective itemsets is proposed. The concepts of "support" of an itemset and correlation of the items within an itemset are related, though not quite the same. This criterion stresses the importance of the actual correlation of the items with one another rather than the absolute support, Previously proposed methods to provide correlated itemsets are not necessarily applicable to very large databases, We provide an algorithm which provides very good computational efficiency, while maintaining statistical robuetneas, The fact that this algorithm relies on relative measures rather than absolute…