arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Adam Silberstein

35 papers · 2006–2019

2006: 32007: 52008: 62009: 32010: 32011: 52012: 72013: 22014: 02015: 02016: 02017: 02018: 02019: 1
2006peak 7 in 20122019

Frequent co-authors

Papers

showing 1–25 of 35 RSS

Benchmarking cloud serving systems with YCSB

Brian F. Cooper, Adam Silberstein, Erwin Tam +2 more

Information Systems · · cited 3,632× · DOI ↗ · more like this ·

While the use of MapReduce systems (such as Hadoop) for large scale data analysis has been widely recognized and studied, we have recently seen an explosion in the number of systems developed for cloud data serving. These newer systems address "cloud OLTP" applications, though they typically do not support ACID transactions. Examples of systems proposed for cloud serving use include BigTable, PNUTS, Cassandra, HBase, Azure, CouchDB, SimpleDB, Voldemort, and many others. Further, they are being applied to a diverse range of applications that differ considerably from traditional (e.g., TPC-C like) serving workloads. The number of emerging cloud serving systems and the wide range of proposed applications, coupled with a lack of apples-to-apples performance comparisons, makes it difficult to understand the tradeoffs between systems and the workloads for which they are suited. We present the…

PNUTS

Brian F. Cooper, Raghu Ramakrishnan, Utkarsh Srivastava +6 more

Computer Networks and Communications · Proceedings of the VLDB Endowment · · cited 960× · DOI ↗ · more like this ·

We describe PNUTS, a massively parallel and geographically distributed database system for Yahoo!'s web applications. PNUTS provides data storage organized as hashed or ordered tables, low latency for large numbers of concurrent requests including updates and queries, and novel per-record consistency guarantees. It is a hosted, centrally managed, and geographically distributed service, and utilizes automated load-balancing and failover to reduce operational complexity. The first version of the system is currently serving in production. We describe the motivation for PNUTS and the design and implementation of its table storage and replication layers, and then present experimental results.

Constraint chaining

Adam Silberstein, Rebecca Braynard, Jun Yang

Computer Networks and Communications · · cited 159× · DOI ↗ · more like this ·

Wireless sensor networks have created new opportunities for data collection in a variety of scenarios, such as environmental and industrial, where we expect data to be temporally and spatially correlated. Researchers may want to continuously collect all sensor data from the network for later analysis. Suppression, both temporal and spatial, provides opportunities for reducing the energy cost of sensor data collection. We demonstrate how both types can be combined for maximal benefit. We frame the problem as one of monitoring node and edge constraints. A monitored node triggers a report if its value changes. A monitored edge triggers a report if the difference between its nodes' values changes. The set of reports collected at the base station is used to derive all node values. We fully exploit the potential of this global inference in our algorithm, CONCH, short for constraint chaining.…

Sailfish

Sriram Rao, Raghu Ramakrishnan, Adam Silberstein +2 more

Information Systems · · cited 95× · DOI ↗ · more like this ·

In this paper, we present Sailfish, a new Map-Reduce framework for large scale data processing. The Sailfish design is centered around aggregating intermediate data, specifically data produced by map tasks and consumed later by reduce tasks, to improve performance by batching disk I/O. We introduce an abstraction called I-files for supporting data aggregation, and describe how we implemented it as an extension of the distributed filesystem, to efficiently batch data written by multiple writers and read by multiple readers. Sailfish adapts the Map-Reduce layer in Hadoop to use I-files for transporting data from map tasks to reduce tasks. We present experimental results demonstrating that Sailfish improves performance of standard Hadoop; in particular, we show 20% to 5 times faster performance on a representative mix of real jobs and datasets at Yahoo!. We also demonstrate that the…

Energy-efficient monitoring of extreme values in sensor networks

Adam Silberstein, Kamesh Munagala, Jun Yang

Computer Networks and Communications · · cited 86× · DOI ↗ · more like this ·

Monitoring extreme values (MAX or MIN) is a fundamental problem in wireless sensor networks (and in general, complex dynamic systems). This problem presents very different algorithmic challenges from aggregate and selection queries, in the sense that an individual node cannot by itself determine its inclusion in the query result. We present novel query processing algorithms for this problem, with the goal of minimizing message traffic in the network. These algorithms employ a hierarchy of local constraints, or thresholds, to leverage network topology such that message-passing is localized. We evaluate all algorithms using simulated and real-world data to study various trade-offs.

Feeding frenzy

Adam Silberstein, Jeff Terrace, Brian F. Cooper +1 more

Computer Networks and Communications · · cited 78× · DOI ↗ · more like this ·

Near real-time event streams are becoming a key feature of many popular web applications. Many web sites allow users to create a personalized feed by selecting one or more event streams they wish to follow. Examples include Twitter and Facebook, which a user to follow other users' activity, and iGoogle and My Yahoo, which allow users to follow selected RSS streams. How can we efficiently construct a web page showing the latest events from a user's feed? Constructing such a feed must be fast so the page loads quickly, yet reflects recent updates to the underlying event streams. The wide fanout of popular streams (those with many followers) and high skew (fanout and update rates vary widely) make it difficult to scale such applications.

Asynchronous view maintenance for VLSD databases

Parag Agrawal, Adam Silberstein, Brian F. Cooper +2 more

Computer Networks and Communications · · cited 68× · DOI ↗ · more like this ·

The query models of the recent generation of very large scale distributed (VLSD) shared-nothing data storage systems, including our own PNUTS and others (e.g. BigTable, Dynamo, Cassandra, etc.) are intentionally simple, focusing on simple lookups and scans and trading query expressiveness for massive scale. Indexes and views can expand the query expressiveness of such systems by materializing more complex access paths and query results. In this paper, we examine mechanisms to implement indexes and views in a massive scale distributed database. For web applications, minimizing update latencies is critical, so we advocate deferring the work of maintaining views and indexes as much as possible. We examine the design space, and conclude that two types of view implementations, called remote view tables (RVTs) and local view tables (LVTs), provide good tradeoff between system throughput and…

Automatic optimization of parallel dataflow programs

Christopher Olston, Benjamin Reed, Adam Silberstein +1 more

Computer Networks and Communications · USENIX Annual Technical Conference · · cited 66× · OpenAlex ↗ · more like this ·

Large-scale parallel dataflow systems, e.g., Dryad and Map-Reduce, have attracted significant attention recently. High-level dataflow languages such as Pig Latin and Sawzall are being layered on top of these systems, to enable faster program development and more maintainable code. These languages engender greater transparency in program structure, and open up opportunities for automatic optimization. This paper proposes a set of optimization strategies for this context, drawing on and extending techniques from the database community. 1

Suppression and failures in sensor networks: a Bayesian approach

Adam Silberstein, Gavino Puggioni, Alan E. Gelfand +2 more

Computer Networks and Communications · Journal of Media Literacy Education · · cited 53× · OpenAlex ↗ · more like this ·

Sensor networks allow continuous data collection on unprecedented scales. The primary limiting factor of such networks is energy, of which communication is the dominant consumer. The default strategy of nodes continually reporting their data to the root results in too much messaging. Suppression stands to greatly alleviate this problem. The simplest such scheme is temporal suppression, in which a node transmits its reading only when it has changed beyond some ɛ since last transmitted. In the absence of a report, the root can infer that the value remains within ±ɛ; hence, it is still able to derive the history of readings produced at the node. The critical weakness of suppression is message failure, to which sensor networks are particularly vulnerable. Failure creates ambiguity: a non-report may either be a suppression or a failure. Inferring the correct values for missing data and…

Many-to-Many Aggregation for Sensor Networks

Adam Silberstein, Jun Yang

Computer Networks and Communications · · cited 49× · DOI ↗ · more like this ·

Wireless sensor networks have enormous potential to aid data collection in a number of areas, such as environmental and wildlife research. In this paper, we address the challenges of supporting many-to-many aggregation in a sensor network. An application of many-to-many aggregation is in-network control of sensors. For expensive sensing tasks such as sap flux measurements and camera repositioning, we use low-cost information obtained at multiple other nodes in the network to control such tasks, e.g., decreasing sampling rates when readings are predictable or unimportant, while increasing sampling rates when there are interesting activities. In general, there is a many-to-many relationship between sources (nodes providing control inputs) and destinations (nodes requiring control outputs). We present a method for implementing many-to-many aggregation in a sensor network that minimizes the…

Challenges, Techniques and Directions in Building XSeek: an XML Search Engine.

Brian F. Cooper, Eric Baldeschwieler, Rodrigo Fonseca +8 more

Computer Networks and Communications · IEEE Data(base) Engineering Bulletin · · cited 45× · OpenAlex ↗ · more like this ·

Yahoo! is building a set of scalable, highly-available data storage and processing services, and deploying them in a cloud model to make application development and ongoing maintenance significantly easier. In this paper we discuss the vision and requirements, as well as the components that will go into the cloud. We highlight the challenges and research questions that arise from trying to build a comprehensive web-scale cloud infrastructure, emphasizing data storage and processing capabilities. (The Yahoo! cloud infrastructure also includes components for provisioning, virtualization, and edge content delivery, but these aspects are only briefly touched on.)

Data›Driven Processing in Sensor Networks

Adam Silberstein, Gregory Filpus, Kamesh Munagala +1 more

Computer Networks and Communications · Journal of Media Literacy Education · · cited 43× · OpenAlex ↗ · more like this ·

Wireless sensor networks are poised to enable continuous data collection on unprecedented scales, in terms of area location and size, and frequency. This is a great boon to fields such as ecological modeling. We are collaborating with researchers to build sophisticated temporal and spatial models of forest growth, utilizing a variety of measurements. There exists a crucial challenge in supporting this activity: network nodes have limited battery life, and radio communication is the dominant energy consumer. The straightforward solution of instructing all nodes to report their measurements as they are taken to a base station will quickly consume the network's energy. On the other hand, the solution of building models for node behavior and substituting these in place of the actual measurements is in conflict with the end goal of constructing models. To address this dilemma, we propose…

Efficient bulk insertion into a distributed ordered table

Adam Silberstein, Brian F. Cooper, Utkarsh Srivastava +3 more

Computer Networks and Communications · · cited 40× · DOI ↗ · more like this ·

We study the problem of bulk-inserting records into tables in a system that horizontally range-partitions data over a large cluster of shared-nothing machines. Each table partition contains a contiguous portion of the table's key range, and must accept all records inserted into that range. Examples of such systems include BigTable[8] at Google, and PNUTS [15] at Yahoo! During bulk inserts into an existing table, if most of the inserted records end up going into a small number of data partitions, the obtained throughput may be very poor due to ineffective use of cluster parallelism. We propose a novel approach in which a planning phase is invoked before the actual insertions. By creating new partitions and intelligently distributing partitions across machines, the planning phase ensures that the insertion load will be well-balanced. Since there is a tradeoff between the cost of moving…

Where in the world is my data?

Sudarshan Kadambi, Jianjun Chen, Brian F. Cooper +5 more

Computer Networks and Communications · Proceedings of the VLDB Endowment · · cited 32× · DOI ↗ · more like this ·

Users of websites such as Facebook, Ebay and Yahoo! demand fast response times, and these sites replicate data across globally distributed datacenters to achieve this. However, it is not necessary to replicate all data to all locations: if a European user's record is never accessed in Asia, it does not make sense to pay the bandwidth and disk costs to maintain an Asian replica. In this paper, we describe mechanisms for selectively replicating large-scale web databases on a record-by-record basis. We introduce a flexible constraint language to specify replication policy constraints. We then present an adaptive scheme for replicating data to where it is most frequently accessed, while respecting policy constraints and using minimal bookkeeping. Experiments using a modified version of our PNUTS system demonstrate our techniques work well.

The Yahoo!

Markus Klems, Adam Silberstein, Jianjun Chen +5 more

Information Systems · · cited 22× · DOI ↗ · more like this ·

Sherpa is a large-scale distributed and globally replicated multi-tenant cloud data storage system. Sherpa scales by horizontally partitioning data into tablets and distributing these tablets across multiple servers. While Sherpa scales for increasing workload sizes by adding servers, it is vulnerable to load imbalance among tablets that cause hotspots to develop on just a few servers.

PNUTS in Flight: Web-Scale Data Serving at Yahoo

Adam Silberstein, Jianjun Chen, David Lomax +5 more

Information Systems · IEEE Internet Computing · · cited 22× · DOI ↗ · more like this ·

Data management for stateful Web applications is extremely challenging. Applications must scale as they grow in popularity, serve their content with low latency on a global scale, and be highly available, even in the face of hardware failures. This need has generated a new class of Internet-scale data management systems. Yahoo has more than 100 user-facing applications and numerous internal platforms. To meet its data management needs, it built the PNUTS system. Here, the authors review PNUTS' growing adoption, point to specific applications, and detail several of PNUTS' features.

Feed following

Adam Silberstein, Ashwin Machanavajjhala, Raghu Ramakrishnan

Computer Networks and Communications · Databases and Social Networks · · cited 20× · DOI ↗ · more like this ·

Internet users spend billions of minutes per month on sites like Facebook and Twitter. These sites support feed following, where users "follow" activity streams associated with other users and entities. Followers get personalized feeds that blend streams produced by those followed. The emphasis on recency and relevance, and the highly variable fan-out of the follows graph, make this feature difficult to implement at the scale seen in major social networks.

Making Sense of Suppressions and Failures in Sensor Data: A Bayesian Approach.

Adam Silberstein, Alan E. Gelfand, Kamesh Munagala +2 more

Computer Networks and Communications · Very Large Data Bases · · cited 18× · DOI ↗ · more like this ·

Bambusa pervariabilis × Dendrocalamopisis grandis blight is caused by a toxin produced by the fungus Arthrinium phaeospermum. In this study, a toxin fraction (P1-2-2) with an estimated molecular mass of 31 kDa was purified from a culture filtrate of this fungus by ammonium sulfate precipitation, Sephadex G-50 gel chromatography, Q Sepharose Fast Flow anion exchange resin, and Sephadex G-75 chromatography. The N-terminal amino acid sequence (i.e., H(2)N-Gln-Val-Arg-Asp-Arg-Leu-Glu-Ser-Thr) determined by Edman degradation showed homology to known serine alkaline proteases. The purified protein was named AP-toxin. Effects of the purified protein toxin on total phenol, flavonoid, total nucleic acid, DNA, RNA, soluble protein, and soluble sugar content, as well as DNase and RNase activities and disease index, were analyzed in different bamboo varieties by the impregnation method. The toxin…

A batch of PNUTS

Adam Silberstein, Russell Sears, Wenchao Zhou +1 more

Information Systems · · cited 13× · DOI ↗ · more like this ·

Cloud data management systems are growing in prominence, particularly at large Internet companies like Google, Yahoo!, and Amazon, which prize them for their scalability and elasticity. Each of these systems trades off between low-latency serving performance and batch processing throughput. In this paper, we discuss our experience running batch-oriented Hadoop on top of Yahoo's serving-oriented PNUTS system instead of the standard HDFS file system. Though PNUTS is optimized for and primarily used for serving, a number of applications at Yahoo! must run batch-oriented jobs that read or write data that is stored in PNUTS.

Adaptively parallelizing distributed range queries

Ýmir Vigfússon, Adam Silberstein, Brian F. Cooper +1 more

Signal Processing · Proceedings of the VLDB Endowment · · cited 11× · DOI ↗ · more like this ·

We consider the problem of how to best parallelize range queries in a massive scale distributed database. In traditional systems the focus has been on maximizing parallelism, for example by laying out data to achieve the highest throughput. However, in a massive scale database such as our PNUTS system [11] or BigTable [10], maximizing parallelism is not necessarily the best strategy: the system has more than enough servers to saturate a single client by returning results faster than the client can consume them, and when there are multiple concurrent queries, maximizing parallelism for all of them will cause disk contention, reducing everybody's performance. How can we find the right parallelism level for each query in order to achieve high, consistent throughput for all queries? We propose an adaptive approach with two aspects. First, we adaptively determine the ideal parallelism for a…

Untangling cluster management with Helix

Kishore Gopalakrishna, Shi Lu, Zhen Zhang +4 more

Information Systems · · cited 10× · DOI ↗ · more like this ·

Distributed data systems systems are used in a variety of settings like online serving, offline analytics, data transport, and search, among other use cases. They let organizations scale out their workloads using cost-effective commodity hardware, while retaining key properties like fault tolerance and scalability. At LinkedIn we have built a number of such systems. A key pattern we observe is that even though they may serve different purposes, they tend to have a lot of common functionality, and tend to use common building blocks in their architectures. One such building block that is just beginning to receive attention is cluster management, which addresses the complexity of handling a dynamic, large-scale system with many servers. Such systems must handle software and hardware failures, setup tasks such as bootstrapping data, and operational issues such as data placement, load…

Cache refreshing for online social news feeds

Xiao Bai, Flavio Junqueira, Adam Silberstein

Computer Networks and Communications · · cited 6× · DOI ↗ · more like this ·

Several social networking applications enable users to view the events generated by other users, typically friends in the social network, in the form of ``news feeds''. Friends and events are typically maintained per user and cached in memory to enable efficient generation of news feeds. Caching user friends and events, however, raises concerns about the freshness of news feeds as users may not observe the most recent events when cache content becomes stale. Mechanisms to keep cache content fresh are thus critical for user satisfaction while computing news feeds efficiently through caching.

HotROD: Managing grid storage with on-demand replication

Sriram Rao, Benjamin Reed, Adam Silberstein

Computer Networks and Communications · · cited 6× · DOI ↗ · more like this ·

Enterprises (such as, Yahoo!, LinkedIn, Facebook) operate their own compute/storage infrastructure, which is effectively a “private cloud”. The private cloud consists of multiple clusters, each of which is managed independently. With HDFS, whenever data is stored in the cluster, it is replicated within the cluster for availability. Unfortunately, for datasets shared across the enterprise, this leads to the problem of over-replication within the private cloud. An analysis of Yahoo!'s HDFS usage suggests that the disk space consumed due to replication of shared datasets is substantial (viz., to the tune of PB's of storage). New data sets are typically popular and requested by many processing jobs in (different) clusters. This demand is satisfied by copying the dataset to each of the clusters. As data sets age, however, they get used less and become cold. We then have the opposite problem…

Shepherding social feed generation with Sheep

Flavio Junqueira, Vincent Leroy, Marco Serafini +1 more

Computer Networks and Communications · · cited 6× · DOI ↗ · more like this ·

Social feeds are used in many popular Web applications. They let users produce events, as well as read feeds containing the events generated by their friends. This paper investigates the design of an in-memory platform to manage social feeds. We show that straightforward memcache implementations suffer from a low throughput due to bandwidth bottlenecks. Following these observations, we propose Sheep, a system to support applications based on social feeds that leverages data aggregation and co-location to alleviate these bottlenecks. We show experimentally that Sheep outperforms memcache implementations by a factor of 7.

PNUTS to Sherpa

Brian F. Cooper, P. P. S. Narayan, Raghu Ramakrishnan +7 more

Computer Networks and Communications · Proceedings of the VLDB Endowment · · cited 5× · DOI ↗ · more like this ·

In this paper, we look back at the evolution of Yahoo!'s geo-replicated cloud data store from a research project called PNUTS to a globally deployed production system called Sherpa, share some of the lessons learned along the way, and finally, compare PNUTS with current operational cloud stores.