arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Yongyang Yu

14 papers · 2010–2021

2010: 12011: 02012: 12013: 12014: 12015: 12016: 12017: 12018: 32019: 22020: 02021: 2
2010peak 3 in 20182021

Frequent co-authors

Papers

showing 1–14 of 14 RSS

LocationSpark

Mingjie Tang, Yongyang Yu, Qutaibah Malluhi +2 more

Signal Processing · Proceedings of the VLDB Endowment · · cited 155× · DOI ↗ · more like this ·

We present LocationSpark, a spatial data processing system built on top of Apache Spark, a widely used distributed data processing system. LocationSpark offers a rich set of spatial query operators, e.g., range search, k NN, spatio-textual operation, spatial-join, and k NN-join. To achieve high performance, LocationSpark employs various spatial indexes for in-memory data, and guarantees that immutable spatial indexes have low overhead with fault tolerance. In addition, we build two new layers over Spark, namely a query scheduler and a query executor. The query scheduler is responsible for mitigating skew in spatial queries, while the query executor selects the best plan based on the indexes and the nature of the spatial queries. Furthermore, to avoid unnecessary network communication overhead when processing overlapped spatial data, We embed an efficient spatial Bloom filter into…

Multilinear PageRank

David F. Gleich, Lek-Heng Lim, Yongyang Yu

math.NA · · cited 148× · arXiv ↗ · more like this ·

In this paper, we first extend the celebrated PageRank modification to a higher-order Markov chain. Although this system has attractive theoretical properties, it is computationally intractable for many interesting problems. We next study a computationally tractable approximation to the higher-order PageRank vector that involves a system of polynomial equations called multilinear PageRank, which is a type of tensor PageRank vector. It is motivated by a novel "spacey random surfer" model, where the surfer remembers bits and pieces of history and is influenced by this information. The underlying stochastic process is an instance of a vertex-reinforced random walk. We develop convergence theory for a simple fixed-point method, a shifted fixed-point method, and a Newton iteration in a particular parameter regime. In marked contrast to the case of the PageRank vector of a Markov chain where…

LocationSpark: In-memory Distributed Spatial Query Processing and Optimization

Mingjie Tang, Yongyang Yu, Walid G. Aref +3 more

cs.DB · · cited 36× · arXiv ↗ · more like this ·

Due to the ubiquity of spatial data applications and the large amounts of spatial data that these applications generate and process, there is a pressing need for scalable spatial query processing. In this paper, we present new techniques for spatial query processing and optimization in an in-memory and distributed setup to address scalability. More specifically, we introduce new techniques for handling query skew, which is common in practice, and optimize communication costs accordingly. We propose a distributed query scheduler that use a new cost model to optimize the cost of spatial query processing. The scheduler generates query execution plans that minimize the effect of query skew. The query scheduler employs new spatial indexing techniques based on bitmap filters to forward queries to the appropriate local nodes. Each local computation node is responsible for optimizing and…

In-Memory Distributed Matrix Computation Processing and Optimization

Yongyang Yu, Mingjie Tang, Walid G. Aref +3 more

Computer Vision and Pattern Recognition · Qatar University QSpace (Qatar University) · · cited 21× · DOI ↗ · more like this ·

The use of large-scale machine learning and data mining methods is becoming ubiquitous in many application domains ranging from business intelligence and bioinformatics to self-driving cars. These methods heavily rely on matrix computations, and it is hence critical to make these computations scalable and efficient. These matrix computations are often complex and involve multiple steps that need to be optimized and sequenced properly for efficient execution. This paper presents new efficient and scalable matrix processing and optimization techniques for in-memory distributed clusters. The proposed techniques estimate the sparsity of intermediate matrix-computation results and optimize communication costs. An evaluation plan generator for complex matrix computations is introduced as well as a distributed plan optimizer that exploits dynamic cost-based analysis and rule-based heuristics…

Efficient Parallel Skyline Query Processing for High-Dimensional Data

Mingjie Tang, Yongyang Yu, Walid G. Aref +2 more

Signal Processing · IEEE Transactions on Knowledge and Data Engineering · · cited 19× · DOI ↗ · more like this ·

Given a set of multidimensional data points, skyline queries retrieve those points that are not dominated by any other points in the set. Due to the ubiquitous se of skyline queries, such as in preference-based query answering and decision making, and the large amount of data that these queries have to deal with, enabling their scalable processing is of critical importance. However, there are several outstanding challenges that have not been well addressed. More specifically, in this paper, we are tackling the data straggler and data skew challenges introduced by distributed skyline query processing, as well as the ensuing high computation cost of merging skyline candidates. We thus introduce a new efficient three-phase approach for large scale processing of skyline queries. In the first preprocessing phase, the data is partitioned along the Z-order curve. We utilize a novel data…

Efficient Processing of Hamming-Distance-Based Similarity-Search Queries Over MapReduce

Mingjie Tang, Yongyang Yu, Walid G. Aref +2 more

Computer Vision and Pattern Recognition · Qatar University QSpace (Qatar University) · · cited 14× · DOI ↗ · more like this ·

Similarity search is crucial to many applications. Of particular interest are two flavors of the Hamming distance range query, namely, the Hamming select and the Hamming join (Hamming-select and Hamming-join, respectively). Hamming distance is widely used in approximate near neighbor search for high dimensional data, such as images and document collections. For example, using predefined similarity hash functions, high-dimensional data is mapped into one-dimensional binary codes that are, then linearly scanned to perform Hamming-distance comparisons. These distance comparisons on the binary codes are usually costly and, often involves excessive redundancies. This paper introduces a new index, termed the HA-Index, that speeds up distance comparisons and eliminates redundancies when performing the two flavors of Hamming distance range queries. An efficient search algorithm based on the…

In-memory Distributed Spatial Query Processing and Optimization

Mingjie Tang, Yongyang Yu, Walid G. Aref +3 more

Signal Processing · · cited 4× · OpenAlex ↗ · more like this ·

Due to the ubiquity of spatial data applications and the large amounts of spatial data that these applications generate and process, there is a pressing need for scalable spatial query processing. In this paper, we present new techniques for spatial query processing and optimization in an in-memory and distributed setup to address scalability. More specifically, we introduce new techniques for handling query skew, which is common in practice, and optimize communication costs accordingly. We propose a distributed query scheduler that use a new cost model to optimize the cost of spatial query processing. The scheduler generates query execution plans that minimize the effect of query skew. The query scheduler employs new spatial indexing techniques based on bitmap filters to forward queries to the appropriate local nodes. Each local computation node is responsible for optimizing and…

Refactoring HUBzero for linked data

Michael Witt, Yongyang Yu

Artificial Intelligence · Purdue e-Pubs (Purdue University System) · · cited 3× · DOI ↗ · more like this ·

The HUBzero cyberinfrastructure provides a virtual research environment that includes a set of tools for web-based, scientific collaboration and a platform for publishing and using resources such as executable software, source code, images, learning modules, videos, documents, and datasets. Released as open source in 2010, HUBzero has been implemented on a typical LAMP stack (Linux, Apache, MySQL, and PHP) and utilizes the Joomla! content management system. This paper describes the subsequent refactoring of HUBzero to produce and expose Linked Data from its backend, relational database, altering the external expression of the data without changing its internal structure. The Open Archives Initiative Object Reuse and Exchange (OAI-ORE) specification is applied to model the basic structural semantics of HUBzero resources as Nested Aggregations, and data and metadata are mapped to…

HIBOR: an efficient approach to sequence pattern query processing in wireless sensor networks

Yongyang Yu, Shengfei Shi, Jianzhong Li +1 more

Signal Processing · · cited 2× · DOI ↗ · more like this ·

Sequence-based query processing has not attracted much attention in wireless sensor networks though its counterpart has been studied extensively in time series databases. So as to answer such queries of interest, data distribution collected by sensor nodes and moving trends of sequences can be captured by HIBOR (Histogram with Bit vectOR). We consider the problem of distributed clustering and querying over histograms with bit vectors of moving trends of sensor data sequences. Especially, we are interested in efficiently answering the following query, namely query by example: return all the sensor nodes that have observed a particular sequence pattern issued by the user with specified thresholds. In this paper, we present a novel approach to addressing the query mentioned above efficiently. First, the whole sensor network is partitioned into several clusters. Second, a distributed index…

Scalable Relational Query Processing on Big Matrix Data

Yongyang Yu, Mingjie Tang, Walid G. Aref

cs.DB · · cited 1× · arXiv ↗ · more like this ·

The use of large-scale machine learning methods is becoming ubiquitous in many applications ranging from business intelligence to self-driving cars. These methods require a complex computation pipeline consisting of various types of operations, e.g., relational operations for pre-processing or post-processing the dataset, and matrix operations for core model computations. Many existing systems focus on efficiently processing matrix-only operations, and assume that the inputs to the relational operators are already pre-computed and are materialized as intermediate matrices. However, the input to a relational operator may be complex in machine learning pipelines, and may involve various combinations of matrix operators. Hence, it is critical to realize scalable and efficient relational query processors that directly operate on big matrix data. This paper presents new efficient and…

Optimizing Generalized Linear Models with Billions of Variables

Yanbo Liang, Yongyang Yu, Mingjie Tang +4 more

Computer Vision and Pattern Recognition · · cited 1× · DOI ↗ · more like this ·

The use of large-scale machine learning~(ML) is becoming ubiquitous in various domains ranging from business intelligence to self-driving cars. Many companies are building ML pipelines in a unified data processing environment, and leveraging well-tuned numerical optimization packages for obtaining model parameters. However, most existing optimization tools are specifically designed for a single machine setup, and cannot handle vast volume of data. In this work, we build a distributed computing framework towards optimizing generalized linear models with billions of variables. We at first design a new distributed vector to represent data points from extremely large feature space. Then, we introduce an efficient and scalable approach to compute the second order derivatives of loss function, and optimizes model parameters with limited memory requirement. Experiments on real-world datasets…

SHC: Distributed Query Processing for Non-Relational Data Store

Wenqing Yang, Mingjie Tang, Yongyang Yu +2 more

Computer Networks and Communications · · cited 1× · DOI ↗ · more like this ·

We introduce a simple data model to process non-relational data for relational operations, and SHC (Apache Spark - Apache HBase Connector), an implementation of this model in the cluster computing framework, Spark. SHC leverages optimization techniques of relational data processing over the distributed and column-oriented key-value store (i.e., HBase). Compared to existing systems, SHC makes two major contributions. At first, SHC offers a much tighter integration between optimizations of relational data processing and non-relational data store, through a plug-in implementation that integrates with Spark SQL, a distributed in-memory computing engine for relational data. The design makes the system maintenance relatively easy, and enables users to perform complex data analytics on top of key-value store. Second, SHC leverages the Spark SQL Catalyst engine for high performance query…

Towards Dual Governance Structure: A Philosophical Reflection on the Disruptive Equipment Innovation

Dazhou Wang, Yongyang Yu

Political Science and International Relations · JOURNAL OF ENGINEERING STUDIES · · cited 0× · DOI ↗ · more like this ·

This paper discusses the governance of disruptive equipment innovation from a philosophical perspective and advocates the establishment of a dual structure of equipment innovation governance based on ecological thinking. With this dual structure, planning and market mechanisms can complement each other,procedural procedures and the green channel can go hand in hand, strict control and lax governance can balance flexibility and flexibility, and the respective strengths of centralized and decentralized efforts can be fully brought into play. Therefore, the demands of non-disruptive equipment innovation and disruptive equipment innovation can be taken into account. Only in this way can all kinds of disruptive equipment innovation emerge, and both civilian and military equipment can develop interactively and sustainably.

The open parks network

Yongyang Yu, Michael Witt, Mohamed Saber Abdelfattah +2 more

Artificial Intelligence · · cited 0× · DOI ↗ · more like this ·

The goal of the Open Parks Network (OPN) is to create a portal that connects park managers, researchers, policy makers, and citizens to each other and to valuable cultural resources related. Led by Clemson University in collaboration with the National Parks Service and Purdue University, OPN is designed to provide a virtual community of professionals in parks and protected areas with the tools, resources, and knowledge base they need to conduct intensive research, perform their jobs duties effectively, and share information with colleagues and users on an international scale. To date, 80,000 of 200,000 archival images and 500,000 out of 2 million bound pages have been digitized from various parks, and a beta version of the platform will be available for demonstration in March 2013. The project includes the integration of a Fedora repository with the Joomla! content management system…