Author

Greg Little

28 papers · 2000–2014

2000peak 5 in 20072014

Frequent co-authors

Papers

showing 1–25 of 28 RSS

Soylent

Michael S. Bernstein, Greg Little, Robert C. Miller +5 more

Computer Science Applications · DSpace@MIT (Massachusetts Institute of Technology) · 3 Oct 2010 · cited 706× · DOI ↗ · more like this ·

This paper introduces architectural and interaction patterns for integrating crowdsourced human contributions directly into user interfaces. We focus on writing and editing, complex endeavors that span many levels of conceptual and pragmatic activity. Authoring tools offer help with pragmatics, but for higher-level help, writers commonly turn to other people. We thus present Soylent, a word processing interface that enables writers to call on Mechanical Turk workers to shorten, proofread, and otherwise edit parts of their documents on demand. To improve worker quality, we introduce the Find-Fix-Verify crowd programming pattern, which splits tasks into a series of generation and review stages. Evaluation studies demonstrate the feasibility of crowdsourced editing and investigate questions of reliability, cost, wait time, and work time for edits.

VizWiz

Jeffrey P. Bigham, Chandrika Jayant, Hanjie Ji +7 more

Computer Science Applications · DSpace@MIT (Massachusetts Institute of Technology) · 26 Apr 2010 · cited 584× · DOI ↗ · more like this ·

Visual information pervades our environment. Vision is used to decide everything from what we want to eat at a restaurant and which bus route to take to whether our clothes match and how long until the milk expires. Individually, the inability to interpret such visual information is a nuisance for blind people who often have effective, if inefficient, work-arounds to overcome them. Collectively, however, they can make blind people less independent. Specialized technology addresses some problems in this space, but automatic approaches cannot yet answer the vast majority of visual questions that blind people may have. VizWiz addresses this shortcoming by using the Internet connections and cameras on existing smartphones to connect blind people and their questions to remote paid workers' answers. VizWiz is designed to have low latency and low cost, making it both competitive with expensive…

TurKit

Greg Little, Lydia B. Chilton, Max Goldman +1 more

Computer Science Applications · 28 Jun 2009 · cited 313× · DOI ↗ · more like this ·

Mechanical Turk (MTurk) is an increasingly popular web service for paying people small rewards to do human computation tasks. Current uses of MTurk typically post independent parallel tasks. We are exploring an alternative iterative paradigm, in which workers build on or evaluate each other's work. We describe TurKit, a new toolkit for deploying iterative tasks to MTurk, with a familiar imperative programming paradigm that effectively uses MTurk workers as subroutines.

Cascade

Lydia B. Chilton, Greg Little, Darren Edge +2 more

Computer Science Applications · 27 Apr 2013 · cited 239× · DOI ↗ · more like this ·

Taxonomies are a useful and ubiquitous way of organizing information. However, creating organizational hierarchies is difficult because the process requires a global understanding of the objects to be categorized. Usually one is created by an individual or a small group of people working together for hours or even days. Unfortunately, this centralized approach does not work well for the large, quickly changing datasets found on the web. Cascade is an automated workflow that allows crowd workers to spend as little at 20 seconds each while collectively making a taxonomy. We evaluate Cascade and show that on three datasets its quality is 80-90% of that of experts. Cascade has a competitive cost to expert information architects, despite taking six times more human labor. Fortunately, this labor can be parallelized such that Cascade will run in as fast as four minutes instead of hours or…

Exploring iterative and parallel human computation processes

Greg Little, Lydia B. Chilton, Max Goldman +1 more

Computer Science Applications · 25 Jul 2010 · cited 172× · DOI ↗ · more like this ·

Services like Amazon's Mechanical Turk have opened the door for exploration of processes that outsource computation to humans. These human computation processes hold tremendous potential to solve a variety of problems in novel and interesting ways. However, we are only just beginning to understand how to design such processes. This paper explores two basic approaches: one where workers work alone in parallel and one where workers iteratively build on each other's work. We present a series of experiments exploring tradeoffs between each approach in several problem domains: writing, brainstorming, and transcription. In each of our experiments, iteration increases the average quality of responses. The increase is statistically significant in writing and brainstorming. However, in brainstorming and transcription, it is not clear that iteration is the best overall approach, in part because…

Web wallet

Min Wu, Robert C. Miller, Greg Little

Information Systems · 2006 · cited 163× · DOI ↗ · more like this ·

We introduce a new anti-phishing solution, the Web Wallet. The Web Wallet is a browser sidebar which users can use to submit their sensitive information online. It detects phishing attacks by determining where users intend to submit their information and suggests an alternative safe path to their intended site if the current site does not match it. It integrates security questions into the user's workflow so that its protection cannot be ignored by the user. We conducted a user study on the Web Wallet prototype and found that the Web Wallet is a promising approach. In the study, it significantly decreased the spoof rate of typical phishing attacks from 63% to 7%, and it effectively prevented all phishing attacks as long as it was used. A majority of the subjects successfully learned to depend on the Web Wallet to submit their login information. However, the study also found that…

Koala

Greg Little, Tessa Lau, Allen Cypher +3 more

Artificial Intelligence · 29 Apr 2007 · cited 162× · DOI ↗ · more like this ·

We present Koala, a system that enables users to capture, share, automate, and personalize business processes on the web. Koala is a collaborative programming-by-demonstration system that records, edits, and plays back user interactions as pseudo-natural language scripts that are both human- and machine-interpretable. Unlike previous programming by demonstration systems, Koala leverages sloppy programming that interprets pseudo-natural language instructions (as opposed to formal syntactic statements) in the context of a given web page's elements and actions. Koala scripts are automatically stored in the Koalescence wiki, where a community of users can share, run, and collaboratively develop their "how-to" knowledge. Koala also takes advantage of corporate and personal data stores to automatically generalize and instantiate user-specific data, so that scripts created by one user are…

Real-time collaborative coding in a web IDE

Max Goldman, Greg Little, Robert C. Miller

Information Systems · DSpace@MIT (Massachusetts Institute of Technology) · 16 Oct 2011 · cited 119× · DOI ↗ · more like this ·

This paper describes Collabode, a web-based Java integrated development environment designed to support close, synchronous collaboration between programmers. We examine the problem of collaborative coding in the face of program compilation errors introduced by other users which make collaboration more difficult, and describe an algorithm for error-mediated integration of program code. Concurrent editors see the text of changes made by collaborators, but the errors reported in their view are based only on their own changes. Editors may run the program at any time, using only error-free edits supplied so far, and ignoring incomplete or otherwise error-generating changes. We evaluate this algorithm and interface on recorded data from previous pilot experiments with Collabode, and via a user study with student and professional programmers. We conclude that it offers appreciable benefits…

Keyword programming in java

Greg Little, Robert C. Miller

Information Systems · 5 Nov 2007 · cited 96× · DOI ↗ · more like this ·

Keyword programming is a novel technique for reducing the need to remember details of programming language syntax and APIs, by translating a small number of keywords provided by the user into a valid expression. Prior work has demonstrated the feasibility and merit of this approach in limited domains. This paper presents a new algorithm that scales to the much larger domain of general-purpose Java programming. We tested the algorithm by extracting keywords from method calls in open source projects, and found that it could accurately reconstruct over 90% of the original expressions. We also conducted a study using keywords generated by users, whose results suggest that users can obtain correct Java code using keyword queries as accurately as they can write the correct Java code themselves

Translating keyword commands into executable code

Greg Little, Robert C. Miller

Software · 15 Oct 2006 · cited 78× · DOI ↗ · more like this ·

Modern applications provide interfaces for scripting, but many users do not know how to write script commands. However, many users are familiar with the idea of entering keywords into a web search engine. Hence, if a user is familiar with the vocabulary of an application domain, we anticipate that they could write a set of keywords expressing a command in that domain. For instance, in the web browsing domain, a user might enter click search button. We call expressions of this form keyword commands, and we present a novel approach for translating keyword commands directly into executable code. Our prototype of this system in the web browsing domain translates click search button into the Chickenfoot code click(findButton("search")). This code is then executed in the context of a web browser to carry out the effect. We also present an implementation of this system in…

Soylent: A Word Processor with a Crowd Inside

Michael S. Bernstein, Greg Little, Robert C. Miller +6 more

Computer Science Applications · 2010 · cited 68× · OpenAlex ↗ · more like this ·

Collabode

Max Goldman, Greg Little, Robert C. Miller

Information Systems · DSpace@MIT (Massachusetts Institute of Technology) · 21 May 2011 · cited 60× · DOI ↗ · more like this ·

Collaborating programmers should use a development environment designed specifically for collaboration, not the same one designed for solo programmers with a few collaborative processes and tools tacked on. This paper describes Collabode, a web-based Java integrated development environment built to support close, synchronous collaboration between programmers. We discuss three collaboration models in which participants take on distinct roles: micro-outsourcing to combine small contributions from many assistants; test-driven pair programming for effective pairwise development; and a mobile instructor connected to the work of many students. In particular, we report very promising preliminary results using Collabode to support micro-outsourcing.

TurKit: Tools for iterative tasks on mechanical turk

Greg Little

Communication · September 2009 · cited 48× · DOI ↗ · more like this ·

Mechanical Turk (MTurk) is an increasingly popular web service for paying people small rewards to do human computation tasks. Current uses of MTurk typically post independent parallel tasks. I am exploring an alternative iterative paradigm, in which workers build on or evaluate each other's work. Part of my proposal is a toolkit called TurKit which facilitates deployment of iterative tasks on MTurk. I want to explore using this technology as a new form of end-user programming, where end-users are writing ldquoprogramsrdquo that are really instructions executed by humans on MTurk.

Seaweed

Lydia B. Chilton, Clayton T. Sims, Max Goldman +2 more

Computer Science Applications · 28 Jun 2009 · cited 31× · DOI ↗ · more like this ·

Seaweed is a web application for experimental economists with no programming background to design two-player symmetric games in a visual-oriented interface. Games are automatically published to the web where players can play against each other remotely and game play is logged so that the game's designer can analyze the data. The design and implementation challenge in Seaweed is to provide an end user programming environment that creates games responsive to events and controlled by logic without the designer understanding programming concepts such as events and synchronization, or being burdened by specifying low-level programming detail. Seaweed achieves this by providing high-level visual representations for variables, control flow, and logic, and by automating behaviors for event handling, synchronization, and function evaluation. Seaweed's evaluation demonstrates that Amazon's…

The methodology and an application to fight against Unicode attacks

Anthony Y. Fu, Xiaotie Deng, Liu Wenyin +1 more

Information Systems · 2006 · cited 31× · DOI ↗ · more like this ·

Unicode is becoming a dominant character representation format for information processing. This presents a very dangerous usability and security problem for many applications. The problem arises because many characters in the UCS (Universal Character Set) are visually and/or semantically similar to each other. This presents a mechanism for malicious people to carry out Unicode Attacks, which include spam attacks, phishing attacks, and web identity attacks. In this paper, we address the potential attacks, and propose a methodology for countering them. To evaluate the feasibility of our methodology, we construct a Unicode Character Similarity List (UC-SimList). We then implement a visual and semantic based edit distance (VSED), as well as a visual and semantic based Knuth-Morris-Pratt algorithm (VSKMP), to detect Unicode attacks. We develop a prototype Unicode attack detection tool…

Beyond independent agreement: a tournament selection approach for quality assurance of human computation tasks

Yuan Sun, Shourya Roy, Greg Little

Computer Science Applications · 2011 · cited 25× · OpenAlex ↗ · more like this ·

Quality assurance remains a key topic in human computation research field. Prior work indicates independent agreement is effective for low difficulty tasks, but has limitations. This paper addresses this problem by proposing a tournament selection based quality control process. The experimental results from this paper show that the human are better at identifying the correct answers than producing them themselves.

Compacting cuts

Robert D. Carr, Goran Konjevod, Greg Little +2 more

Computational Theory and Mathematics · ACM Transactions on Algorithms · July 2009 · cited 13× · DOI ↗ · more like this ·

For a graph ( V , E ), existing compact linear formulations for the minimum cut problem require Θ(| V || E |) variables and constraints and can be interpreted as a composition of | V | − 1 polyhedra for minimum s - t cuts in much the same way as early approaches to finding globally minimum cuts relied on | V | − 1 calls to a minimum s - t cut algorithm. We present the first formulation to beat this bound, one that uses O (| V | 2 ) variables and O (| V | 3 ) constraints. An immediate consequence of our result is a compact linear relaxation with O (| V | 2 ) constraints and O (| V | 3 ) variables for enforcing global connectivity constraints. This relaxation is as strong as standard cut-based relaxations and has applications in solving traveling salesman problems by integer programming as well as finding approximate solutions for survivable network design problems using Jain's iterative…

Compacting cuts: a new linear formulation for minimum cut

Robert D. Carr, Goran Konjevod, Greg Little +2 more

Computational Theory and Mathematics · Symposium on Discrete Algorithms · 7 Jan 2007 · cited 6× · DOI ↗ · more like this ·

For a graph (V,E), existing compact linear formulations for the minimum cut problem require Θ(|V ||E|) variables and constraints and can be interpreted as a composition of |V | − 1 polyhedra for minimum s-t cuts in much the same way as early approaches to finding globally minimum cuts relied on |V | − 1 calls to a minimum s-t cut algorithm. We present the first formulation to beat this bound, one that uses O(|V | 2) variables and O(|V | 3) constraints. An immediate consequence of our result is a compact linear relaxation with O(|V | 2) constraints and O(|V | 3) variables for enforcing global connectivity constraints. This relaxation is as strong as standard cut-based relaxations and have applications in solving traveling salesman problems by integer programming as well as finding approximate solutions for survivable network design problems using Jain’s iterative rounding method. Another…

Inky: a sloppy command line for the web with rich visual feedback

Robert C. Miller, Victoria Chou, Michael S. Bernstein +4 more

Sociology and Political Science · ePrints Soton (University of Southampton) · 2008 · cited 4× · OpenAlex ↗ · more like this ·

We present Inky, a command line for shortcut access to common web tasks. Inky aims to capture the efficiency benefits of typed commands while mitigating their usability problems. Inky commands have little or no new syntax to learn, and the system displays rich visual feedback while the user is typing, including missing parameters and con-textual information automatically clipped from the target web site. Inky is an example of a new kind of hybrid be-tween a command line and a GUI interface. We describe the design and implementation of two prototypes of this idea, and report the results of a field study. ACM Classification: H5.2 [Information interfaces and

Heads in the Cloud: Challenges and Opportunities in Human Computation

Robert C. Miller, Greg Little, Michael Bernstein +4 more

Computer Science Applications · 22 Jan 2014 · cited 0× · OpenAlex ↗ · more like this ·

Crowd computing is rapidly becoming an essential part of the landscape of modern computing. Crowd computing encompasses the interaction among large numbers of people facilitated by software systems and increasingly ubiquitous networking technology. Crowds are powering intellectual enterprises (Wikipedia), real-time information media (Twitter), prediction markets (Intrade), and online labor

Cascade: Crowdsourcing Taxonomy Creation

Lydia B. Chilton, Greg Little, Darren Edge +2 more

Computer Science Applications · 16 Jul 2013 · cited 0× · OpenAlex ↗ · more like this ·

Taxonomies are a useful and ubiquitous way of organizing information. However, creating organizational hierarchies is difficult because the process requires a global understanding of the objects to be categorized. Usually one is created by an individual or a small group of people working together for hours or even days. Unfortunately, this centralized approach does not work well for the large, quickly-changing datasets found on the web. Cascade is an automated workflow that creates a taxonomy from the collective efforts of crowd workers who spend as little as 20 seconds each. We evaluate Cascade and show that on three datasets its quality is 80-90 % of that of experts. The cost of Cascade is competitive with expert information architects, despite taking six times more human labor. Fortunately, this labor can be parallelized such that Cascade will run in as fast as five minutes instead…

General Terms

Greg Little, Lydia B. Chilton, Max Goldman +1 more

Sociology and Political Science · 14 Jan 2012 · cited 0× · OpenAlex ↗ · more like this ·

Services like Amazon’s Mechanical Turk have opened the door for exploration of processes that outsource computation to humans. These human computation processes hold tremendous potential to solve a variety of problems in novel and interesting ways. However, we are only just beginning to understand how to design such processes. This paper explores two basic approaches: one where workers work alone in parallel and one where workers iteratively build on each other’s work. We present a series of experiments exploring tradeoffs between each approach in several problem domains: writing, brainstorming, and transcription. In each of our experiments, iteration increases the average quality of responses. The increase is statistically significant in writing and brainstorming. However, in brainstorming and transcription, it is not clear that iteration is the best overall approach, in part because…

Anthony Y. Fu 1,2 Context Sensitive Password

Robert C. Miller, Greg Little, Min Wu

Information Systems · 22 Mar 2011 · cited 0× · OpenAlex ↗ · more like this ·

There is a long history of people typing passwords into HTML textboxes to verify their identities. However, textboxes are easy to spoof by phishing websites. We propose a twist on the traditional password input field which is more secure, while remaining easyto-use.

Programming with Human Computation

Greg Little, Robert C. Miller

Computer Science Applications · DSpace@MIT (Massachusetts Institute of Technology) · 2011 · cited 0× · OpenAlex ↗ · more like this ·

Amazon’s Mechanical Turk provides a programmatically accessible micro-task market, allowing a program to hire human workers. This has opened the door to a rich field of research in human computation where programs orchestrate the efforts of humans to help solve problems. This thesis explores challenges that programmers face in this space: both technical challenges like managing high-latency, as well as psychological challenges like designing effective interfaces for human workers. We offer tools and experiments to overcome these challenges in an effort to help future researchers better understand and harness the power of human computation. The main tool this thesis offers is the crash-and-rerun programming model for managing high-latency tasks on MTurk, along with the TurKit toolkit which implements crash-and-rerun. TurKit provides a straightforward imperative programming environment…

Experimentation, Languages

Greg Little, Robert C. Miller

Information Systems · 13 Mar 2010 · cited 0× · OpenAlex ↗ · more like this ·

Keyword programming is a novel technique for reducing the need to remember details of programming language syntax and APIs, by translating a small number of keywords provided by the user into a valid expression. Prior work has demonstrated the feasibility and merit of this approach in limited domains. This paper presents a new algorithm that scales to the much larger domain of general-purpose Java programming. We tested the algorithm by extracting keywords from method calls in open source projects, and found that it could accurately reconstruct over 90 % of the original expressions. We also conducted a study using keywords generated by users, whose results suggest that users can obtain correct Java code using keyword queries as accurately as they can write the correct Java code themselves.