Author

Sai Wu

141 papers · 1994–2026

1994peak 17 in 20252026

Frequent co-authors

Papers

showing 1–25 of 141 RSS

The performance of MapReduce

Dawei Jiang, Beng Chin Ooi, Lei Shi +1 more

Information Systems · Proceedings of the VLDB Endowment · September 2010 · cited 394× · DOI ↗ · more like this ·

MapReduce has been widely used for large-scale data analysis in the Cloud. The system is well recognized for its elastic scalability and fine-grained fault tolerance although its performance has been noted to be suboptimal in the database context. According to a recent study [19], Hadoop, an open source implementation of MapReduce, is slower than two state-of-the-art parallel database systems in performing a variety of analytical tasks by a factor of 3.1 to 6.5. MapReduce can achieve better performance with the allocation of more compute nodes from the cloud to speed up computation; however, this approach of "renting more nodes" is not cost effective in a pay-as-you-go environment. Users desire an economical elastically scalable data processing system, and therefore, are interested in whether MapReduce can offer both elastic scalability and efficiency. In this paper, we conduct a…

CDAS: A Crowdsourcing Data Analytics System

Xuan Liu, Meiyu Lu, Beng Chin Ooi +3 more

cs.DB · 30 Jun 2012 · cited 300× · arXiv ↗ · more like this ·

Some complex problems, such as image tagging and natural language processing, are very challenging for computers, where even state-of-the-art technology is yet able to provide satisfactory accuracy. Therefore, rather than relying solely on developing new and better algorithms to handle such tasks, we look to the crowdsourcing solution -- employing human participation -- to make good the shortfall in current technology. Crowdsourcing is a good supplement to many computer tasks. A complex job may be divided into computer-oriented tasks and human-oriented tasks, which are then assigned to machines and humans respectively. To leverage the power of crowdsourcing, we design and implement a Crowdsourcing Data Analytics System, CDAS. CDAS is a framework designed to support the deployment of various crowdsourcing applications. The core part of CDAS is a quality-sensitive answering model, which…

CDAS

Xuan Liu, Meiyu Lu, Beng Chin Ooi +3 more

Computer Science Applications · Proceedings of the VLDB Endowment · June 2012 · cited 255× · DOI ↗ · more like this ·

Indexing multi-dimensional data in a cloud system

Jinbao Wang, Sai Wu, Hong Gao +2 more

Signal Processing · National University of Singapore · 6 Jun 2010 · cited 188× · DOI ↗ · more like this ·

Providing scalable database services is an essential requirement for extending many existing applications of the Cloud platform. Due to the diversity of applications, database services on the Cloud must support large-scale data analytical jobs and high concurrent OLTP queries. Most existing work focuses on some specific type of applications. To provide an integrated framework, we are designing a new system, epiC, as our solution to next-generation database systems. In epiC, indexes play an important role in improving overall performance. Different types of indexes are built to provide efficient query processing for different applications.

Query optimization for massively parallel data processing

Sai Wu, Li Feng, Sharad Mehrotra +1 more

Information Systems · National University of Singapore · 26 Oct 2011 · cited 159× · DOI ↗ · more like this ·

MapReduce has been widely recognized as an efficient tool for large-scale data analysis. It achieves high performance by exploiting parallelism among processing nodes while providing a simple interface for upper-layer applications. Some vendors have enhanced their data warehouse systems by integrating MapReduce into the systems. However, existing MapReduce-based query processing systems, such as Hive, fall short of the query optimization and competency of conventional database systems. Given an SQL query, Hive translates the query into a set of MapReduce jobs sentence by sentence. This design assumes that the user can optimize his query before submitting it to the system. Unfortunately, manual query optimization is time consuming and difficult, even to an experienced database user or administrator. In this paper, we propose a query optimization scheme for MapReduce-based processing…

Efficient B-tree based indexing for cloud data processing

Sai Wu, Dawei Jiang, Beng Chin Ooi +1 more

Computer Networks and Communications · Proceedings of the VLDB Endowment · September 2010 · cited 148× · DOI ↗ · more like this ·

A Cloud may be seen as a type of flexible computing infrastructure consisting of many compute nodes, where resizable computing capacities can be provided to different customers. To fully harness the power of the Cloud, efficient data management is needed to handle huge volumes of data and support a large number of concurrent end users. To achieve that, a scalable and high-throughput indexing scheme is generally required. Such an indexing scheme must not only incur a low maintenance cost but also support parallel search to improve scalability. In this paper, we present a novel, scalable B + -tree based indexing scheme for efficient data processing in the Cloud. Our approach can be summarized as follows. First, we build a local B + -tree index for each compute node which only indexes data residing on the node. Second, we organize the compute nodes as a structured overlay and publish a…

Personal recommendation using deep recurrent neural networks in NetEase

Sai Wu, Weichao Ren, Chengchao Yu +3 more

Information Systems · May 2016 · cited 141× · DOI ↗ · more like this ·

Each user session in an e-commerce system can be modeled as a sequence of web pages, indicating how the user interacts with the system and makes his/her purchase. A typical recommendation approach, e.g., Collaborative Filtering, generates its results at the beginning of each session, listing the most likely purchased items. However, such approach fails to exploit current viewing history of the user and hence, is unable to provide a real-time customized recommendation service. In this paper, we build a deep recurrent neural network to address the problem. The network tracks how users browse the website using multiple hidden layers. Each hidden layer models how the combinations of webpages are accessed and in what order. To reduce the processing cost, the network only records a finite number of states, while the old states collapse into a single history state. Our model refreshes the…

Llama

Yu-Ting Lin, Divyakant Agrawal, Chun Chen +2 more

Information Systems · National University of Singapore · 12 Jun 2011 · cited 126× · DOI ↗ · more like this ·

To achieve high reliability and scalability, most large-scale data warehouse systems have adopted the cluster-based architecture. In this paper, we propose the design of a new cluster-based data warehouse system, LLama, a hybrid data management system which combines the features of row-wise and column-wise database systems. In Llama, columns are formed into correlation groups to provide the basis for the vertical partitioning of tables. Llama employs a distributed file system (DFS) to disseminate data among cluster nodes. Above the DFS, a MapReduce-based query engine is supported. We design a new join algorithm to facilitate fast join processing. We present a performance study on TPC-H dataset and compare Llama with Hive, a data warehouse infrastructure built on top of Hadoop. The experiment is conducted on EC2. The results show that Llama has an excellent load performance and its query…

TI

Chun Chen, Feng Li, Beng Chin Ooi +1 more

Computer Networks and Communications · National University of Singapore · 12 Jun 2011 · cited 112× · DOI ↗ · more like this ·

Real-time search dictates that new contents be made available for search immediately following their creation. From the database perspective, this requirement may be quite easily met by creating an up-to-date index for the contents and measuring search quality by the time gap between insertion time and availability of the index. This approach, however, poses new challenges for micro-blogging systems where thousands of concurrent users may upload their micro-blogs or tweets simultaneously. Due to the high update and query loads, conventional approaches would either fail to index the huge amount of newly created contents in real time or fall short of providing a scalable indexing service.

A Comprehensive Survey of Dynamic Graph Neural Networks: Models, Frameworks, Benchmarks, Experiments and Challenges

ZhengZhao Feng, Rui Wang, TianXing Wang +3 more

cs.LG · May 2024 · cited 91× · arXiv ↗ · more like this ·

Dynamic Graph Neural Networks (GNNs) combine temporal information with GNNs to capture structural, temporal, and contextual relationships in dynamic graphs simultaneously, leading to enhanced performance in various applications. As the demand for dynamic GNNs continues to grow, numerous models and frameworks have emerged to cater to different application needs. There is a pressing need for a comprehensive survey that evaluates the performance, strengths, and limitations of various approaches in this domain. This paper aims to fill this gap by offering a thorough comparative analysis and experimental evaluation of dynamic GNNs. It covers 81 dynamic GNN models with a novel taxonomy, 12 dynamic GNN training frameworks, and commonly used benchmarks. We also conduct experimental results from testing representative nine dynamic GNN models and three frameworks on six standard graph datasets.…

ES<sup>2</sup>: A cloud data storage system for supporting both OLTP and OLAP

Yu Cao, Chun Chen, Fei Guo +6 more

Information Systems · National University of Singapore · April 2011 · cited 87× · DOI ↗ · more like this ·

Cloud computing represents a paradigm shift driven by the increasing demand of Web based applications for elastic, scalable and efficient system architectures that can efficiently support their ever-growing data volume and large-scale data analysis. A typical data management system has to deal with real-time updates by individual users, and as well as periodical large scale analytical processing, indexing, and data extraction. While such operations may take place in the same domain, the design and development of the systems have somehow evolved independently for transactional and periodical analytical processing. Such a system-level separation has resulted in problems such as data freshness as well as serious data storage redundancy. Ideally, it would be more efficient to apply ad-hoc analytical processing on the same data directly. However, to the best of our knowledge, such an…

Continuous sampling for online aggregation over multiple queries

Sai Wu, Beng Chin Ooi, Kian‐Lee Tan

Signal Processing · National University of Singapore · 6 Jun 2010 · cited 78× · DOI ↗ · more like this ·

In this paper, we propose an online aggregation system called COSMOS (Continuous Sampling for Multiple queries in an Online aggregation System), to process multiple aggregate queries efficiently. In COSMOS, a dataset is first scrambled so that sequentially scanning the dataset gives rise to a stream of random samples for all queries. Moreover, COS-MOS organizes queries into a dissemination graph to exploit the dependencies across queries. In this way, aggregates of queries closer to the root (source of data flow) can potentially be used to compute the aggregates of descendent/dependent queries. COSMOS applies some statistical approach to combine answers from ancestor nodes to generate the online aggregates for a node. COSMOS also offers a partitioning strategy to further salvage intermediate answers. We have implemented COSMOS and conducted an extensive experimental study in PostgreSQL.…

A critical look at the current train/test split in machine learning

Jimin Tan, Jianan Yang, Sai Wu +2 more

cs.LG · 8 Jun 2021 · cited 68× · arXiv ↗ · more like this ·

The randomized or cross-validated split of training and testing sets has been adopted as the gold standard of machine learning for decades. The establishment of these split protocols are based on two assumptions: (i)-fixing the dataset to be eternally static so we could evaluate different machine learning algorithms or models; (ii)-there is a complete set of annotated data available to researchers or industrial practitioners. However, in this article, we intend to take a closer and critical look at the split protocol itself and point out its weakness and limitation, especially for industrial applications. In many real-world problems, we must acknowledge that there are numerous situations where assumption (ii) does not hold. For instance, for interdisciplinary applications like drug discovery, it often requires real lab experiments to annotate data which poses huge costs in both time and…

LazyLSH

Yuxin Zheng, Qi Guo, Anthony K. H. Tung +1 more

Computer Vision and Pattern Recognition · 16 Jun 2016 · cited 67× · DOI ↗ · more like this ·

Due to the "curse of dimensionality" problem, it is very expensive to process the nearest neighbor (NN) query in high-dimensional spaces; and hence, approximate approaches, such as Locality-Sensitive Hashing (LSH), are widely used for their theoretical guarantees and empirical performance. Current LSH-based approaches target at the L1 and L2 spaces, while as shown in previous work, the fractional distance metrics (Lp metrics with 0 < p < 1) can provide more insightful results than the usual L1 and L2 metrics for data mining and multimedia applications. However, none of the existing work can support multiple fractional distance metrics using one index. In this paper, we propose LazyLSH that answers approximate nearest neighbor queries for multiple Lp metrics with theoretical guarantees. Different from previous LSH approaches which need to build one dedicated index for every query space…

epiC

Dawei Jiang, Gang Chen, Beng Chin Ooi +2 more

Information Systems · Proceedings of the VLDB Endowment · March 2014 · cited 66× · DOI ↗ · more like this ·

The Big Data problem is characterized by the so called 3V features: Volume - a huge amount of data, Velocity - a high data ingestion rate, and Variety - a mix of structured data, semi-structured data, and unstructured data. The state-of-the-art solutions to the Big Data problem are largely based on the MapReduce framework (aka its open source implementation Hadoop). Although Hadoop handles the data volume challenge successfully, it does not deal with the data variety well since the programming interfaces and its associated data processing model is inconvenient and inefficient for handling structured data and graph data. This paper presents epiC , an extensible system to tackle the Big Data's data variety challenge. epiC introduces a general Actor-like concurrent programming model, independent of the data processing models, for specifying parallel computations. Users process…

Distributed online aggregations

Sai Wu, Shouxu Jiang, Beng Chin Ooi +1 more

Computer Networks and Communications · Proceedings of the VLDB Endowment · August 2009 · cited 64× · DOI ↗ · more like this ·

In many decision making applications, users typically issue aggregate queries. To evaluate these computationally expensive queries, online aggregation has been developed to provide approximate answers (with their respective confidence intervals) quickly, and to continuously refine the answers. In this paper, we extend the online aggregation technique to a distributed context where sites are maintained in a DHT (Distributed Hash Table) network. Our Distributed Online Aggregation (DoA) scheme iteratively and progressively produces approximate aggregate answers as follows: in each iteration, a small set of random samples are retrieved from the data sites and distributed to the processing sites; at each processing site, a local aggregate is computed based on the allocated samples; at a coordinator site, these local aggregates are combined into a global aggregate. DoA adaptively grows the…

Assessing Hidden Risks of LLMs: An Empirical Study on Robustness, Consistency, and Credibility

Wentao Ye, Mingfeng Ou, Tianyi Li +8 more

cs.LG · May 2023 · cited 50× · arXiv ↗ · more like this ·

The recent popularity of large language models (LLMs) has brought a significant impact to boundless fields, particularly through their open-ended ecosystem such as the APIs, open-sourced models, and plugins. However, with their widespread deployment, there is a general lack of research that thoroughly discusses and analyzes the potential risks concealed. In that case, we intend to conduct a preliminary but pioneering study covering the robustness, consistency, and credibility of LLMs systems. With most of the related literature in the era of LLM uncharted, we propose an automated workflow that copes with an upscaled number of queries/responses. Overall, we conduct over a million queries to the mainstream LLMs including ChatGPT, LLaMA, and OPT. Core to our workflow consists of a data primitive, followed by an automated interpreter that evaluates these LLMs under different adversarial…

A Framework for supporting DBMS-like indexes in the cloud

Gang Chen, Hoang Tam Vo, Sai Wu +2 more

Computer Networks and Communications · Proceedings of the VLDB Endowment · August 2011 · cited 47× · DOI ↗ · more like this ·

To support "Database as a service" (DaaS) in the cloud, the database system is expected to provide similar functionalities as in centralized DBMS such as efficient processing of ad hoc queries. The system must therefore support DBMS-like indexes, possibly a few indexes for each table to provide fast location of data distributed over the network. In such a distributed environment, the indexes have to be distributed over the network to achieve scalability and reliability. Each cluster node maintains a subset of the index data. As in conventional DBMS, indexes incur maintenance overhead and the problem is more complex in the distributed environment since the data are typically partitioned and distributed based on a subset of attributes. Further, the distribution of indexes is not straight forward, and there is therefore always the question of scalability, in terms of data volume, network…

An Indexing Framework for Efficient Retrieval on the Cloud.

Sai Wu, Kun‐Lung Wu

Computer Networks and Communications · IEEE Data(base) Engineering Bulletin · 2009 · cited 47× · OpenAlex ↗ · more like this ·

The emergence of the Cloud system has simplified the deployment of large-scale distributed systems for software vendors. The Cloud system provides a simple and unified interface between vendor and user, allowing vendors to focus more on the software itself rather than the underlying framework. Existing Cloud systems seek to improve performance by increasing parallelism. In this paper, we explore an alternative solution, proposing an indexing framework for the Cloud system based on the structured overlay. Our indexing framework reduces the amount of data transferred inside the Cloud and facilitates the deployment of database back-end applications. 1

Detecting Logic Bugs of Join Optimizations in DBMS

Xiu Tang, Sai Wu, Dongxiang Zhang +2 more

cs.DB · 25 Feb 2026 · cited 44× · arXiv ↗ · more like this ·

Generation-based testing techniques have shown their effectiveness in detecting logic bugs of DBMS, which are often caused by improper implementation of query optimizers. Nonetheless, existing generation-based debug tools are limited to single-table queries and there is a substantial research gap regarding multi-table queries with join operators. In this paper, we propose TQS, a novel testing framework targeted at detecting logic bugs derived by queries involving multi-table joins. Given a target DBMS, TQS achieves the goal with two key components: Data-guided Schema and Query Generation (DSG) and Knowledge-guided Query Space Exploration (KQE). DSG addresses the key challenge of multi-table query debugging: how to generate ground-truth (query, result) pairs for verification. It adopts the database normalization technique to generate a testing schema and maintains a bitmap index for…

K-Anonymity for Crowdsourcing Database

Sai Wu, Xiaoli Wang, Sheng Wang +2 more

Artificial Intelligence · IEEE Transactions on Knowledge and Data Engineering · 11 Jun 2013 · cited 43× · DOI ↗ · more like this ·

In crowdsourcing database, human operators are embedded into the database engine and collaborate with other conventional database operators to process the queries. Each human operator publishes small HITs (Human Intelligent Task) to the crowdsourcing platform, which consists of a set of database records and corresponding questions for human workers. The human workers complete the HITs and return the results to the crowdsourcing database for further processing. In practice, published records in HITs may contain sensitive attributes, probably causing privacy leakage so that malicious workers could link them with other public databases to reveal individual private information. Conventional privacy protection techniques, such as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K-Anonymity</i> , can be applied to partially solve the problem.…

Skyline-join in distributed databases

Dalie Sun, Sai Wu, Jianzhong Li +1 more

Signal Processing · National University of Singapore · April 2008 · cited 43× · DOI ↗ · more like this ·

The database research community has recently recognized the usefulness of skyline query. As an extension of existing database operator, the skyline query is valuable for multi-criteria decision making. However, current research tends to assume that the skyline operator is applied to one table which is not true for many applications on Web databases. In Web databases, tables are distributed in different sites, and a skyline query may involve attributes of multiple tables. In this paper, we address the problem of processing skyline queries on multiple tables in a distributed environment. We call the new operator skyline-join, as it is a hybrid of skyline and join operations. We propose two efficient approaches to process skyline-join queries which can significantly reduce the communication cost and processing time. Experiments are conducted and results show that our approaches are…

ScalaGiST

Lu Peng, Gang Chen, Beng Chin Ooi +2 more

Information Systems · Proceedings of the VLDB Endowment · October 2014 · cited 36× · DOI ↗ · more like this ·

MapReduce has become the state-of-the-art for data parallel processing. Nevertheless, Hadoop, an open-source equivalent of MapReduce, has been noted to have sub-optimal performance in the database context since it is initially designed to operate on raw data without utilizing any type of indexes. To alleviate the problem, we present ScalaGiST - scalable generalized search tree that can be seamlessly integrated with Hadoop, together with a cost-based data access optimizer for efficient query processing at run-time. ScalaGiST provides extensibility in terms of data and query types, hence is able to support unconventional queries (e.g., multi-dimensional range and k -NN queries) in MapReduce systems, and can be dynamically deployed in large cluster environments for handling big users and data. We have built ScalaGiST and demonstrated that it can be easily instantiated to common B + -tree…

Adaptive Logging

Chang Yao, Divyakant Agrawal, Gang Chen +2 more

Computer Networks and Communications · 14 Jun 2016 · cited 33× · DOI ↗ · more like this ·

By maintaining the data in main memory, in-memory databases dramatically reduce the I/O cost of transaction processing. However, for recovery purposes, in-memory systems still need to flush the log to disk, which incurs a substantial number of I/Os. Recently, command logging has been proposed to replace the traditional data log (e.g., ARIES logging) in in-memory databases. Instead of recording how the tuples are updated, command logging only tracks the transactions that are being executed, thereby effectively reducing the size of the log and improving the performance. However, when a failure occurs, all the transactions in the log after the last checkpoint must be redone sequentially and this significantly increases the cost of recovery. In this paper, we first extend the command logging technique to a distributed system, where all the nodes can perform their recovery in parallel. We…

Continuous Trajectory Similarity Search for Online Outlier Detection

Dongxiang Zhang, Zhihao Chang, Sai Wu +3 more

Signal Processing · IEEE Transactions on Knowledge and Data Engineering · 24 Dec 2020 · cited 32× · DOI ↗ · more like this ·

In this paper, we study a new variant of trajectory similarity search from the context of continuous query processing. Given a moving object from <inline-formula><tex-math notation="LaTeX">$s$</tex-math></inline-formula> to <inline-formula><tex-math notation="LaTeX">$d$</tex-math></inline-formula> , following a reference route <inline-formula><tex-math notation="LaTeX">$T_r$</tex-math></inline-formula> , we monitor the trajectory similarity between the reference route and the current partial route at each timestamp for online detour detection. Since existing trajectory distance measures fail to adequately capture the deviation between a partial route and a complete route, we propose a partial trajectory similarity measure to bridge the gap. In particular, we enumerate all the possible routes extended from the partial route to reach the destination <inline-formula><tex-math…