arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Sumit Basu

72 papers · 1995–2025

1995: 11996: 11997: 41998: 11999: 32000: 02001: 22002: 42003: 32004: 32005: 12006: 52007: 32008: 32009: 22010: 22011: 12012: 62013: 22014: 32015: 32016: 42017: 12018: 22019: 02020: 32021: 32022: 22023: 12024: 22025: 1
1995peak 6 in 20122025

Frequent co-authors

Papers

showing 1–25 of 72 RSS

Learning from the Wisdom of Crowds by Minimax Entropy

Dengyong Zhou, Sumit Basu, Yi Mao +1 more

Computer Science Applications · Neural Information Processing Systems · · cited 320× · OpenAlex ↗ · more like this ·

An important way to make large training sets is to gather noisy labels from crowds of nonexperts. We propose a minimax entropy principle to improve the quality of these labels. Our method assumes that labels are generated by a probability distribution over workers, items, and labels. By maximizing the entropy of this distribution, the method naturally infers item confusability and worker expertise. We infer the ground truth by minimizing the entropy of this distribution, which we show minimizes the Kullback-Leibler (KL) divergence between the probability distribution and the unknown truth. We show that a simple coordinate descent scheme can optimize minimax entropy. Empirically, our results are substantially better than previously published methods for the same problem. 1

Motion regularization for model-based head tracking

Sumit Basu, Irfan Essa, Alex Pentland

Computer Vision and Pattern Recognition · · cited 195× · DOI ↗ · more like this ·

This paper describes a method for the robust tracking of rigid head motion from video. This method uses a 3D ellipsoidal model of the head and interprets the optical flow in terms of the possible rigid motions of the model. This method is robust to large angular and translational motions of the head and is not subject to the singularities of a 2D model. The method has been successfully applied to heads with a variety of shapes, hair styles, etc. This method also has the advantage of accurately capturing the 3D motion parameters of the head. This accuracy is shown through comparison with a ground truth synthetic sequence (a rendered 3D animation of a model head). In addition, the ellipsoidal model is robust to small variations in the initial fit, enabling the automation of the model initialization. Lastly, due to its consideration of the entire 3D aspect of the head, the tracking is very…

Powergrading: a Clustering Approach to Amplify Human Effort for Short Answer Grading

Sumit Basu, Chuck Jacobs, Lucy Vanderwende

Computer Vision and Pattern Recognition · Transactions of the Association for Computational Linguistics · · cited 164× · DOI ↗ · more like this ·

We introduce a new approach to the machine-assisted grading of short answer questions. We follow past work in automated grading by first training a similarity metric between student responses, but then go on to use this metric to group responses into clusters and subclusters. The resulting groupings allow teachers to grade multiple responses with a single action, provide rich feedback to groups of similar answers, and discover modalities of misunderstanding among students; we refer to this amplification of grader effort as “powergrading.” We develop the means to further reduce teacher effort by automatically performing actions when an answer key is available. We show results in terms of grading progress with a small “budget” of human actions, both from our method and an LDA-based approach, on a test corpus of 10 questions answered by 698 respondents.

MySong

Ian Simon, Dan Morris, Sumit Basu

Computer Vision and Pattern Recognition · · cited 155× · DOI ↗ · more like this ·

We introduce MySong, a system that automatically chooses chords to accompany a vocal melody. A user with no musical experience can create a song with instrumental accompaniment just by singing into a microphone, and can experiment with different styles and chord patterns using interactions designed to be intuitive to non-musicians.

Modeling, tracking and interactive animation of faces and heads//using input from video

Irfan Essa, Sumit Basu, Trevor Darrell +1 more

Computer Vision and Pattern Recognition · · cited 133× · DOI ↗ · more like this ·

We describe tools that use measurements from video for the extraction of facial modeling and animation parameters, head tracking, and real time interactive facial animation. These tools share common goals but rely on varying details of physical and geometric modeling and in their input measurement system. Accurate facial modeling involves fine details of geometry and muscle coarticulation. By coupling pixel by pixel measurements of surface motion to a physically based face model and a muscle control model, we have been able to obtain detailed spatio temporal records of both the displacement of each point on the facial surface and the muscle control required to produce the observed facial motion. We discuss the importance of this visually extracted representation in terms of realistic facial motion synthesis. A similar method that uses an ellipsoidal model of the head coupled with…

Selective supervision: guiding supervised learning with decision-theoretic active learning

Ashish Kapoor, Eric Horvitz, Sumit Basu

Artificial Intelligence · International Joint Conference on Artificial Intelligence · · cited 121× · OpenAlex ↗ · more like this ·

An inescapable bottleneck with learning from large data sets is the high cost of labeling training data. Unsupervised learning methods have promised to lower the cost of tagging by leveraging notions of similarity among data points to assign tags. However, unsupervised and semi-supervised learning techniques often provide poor results due to errors in estimation. We look at methods that guide the allocation of human effort for labeling data so as to get the greatest boosts in discriminatory power with increasing amounts of work. We focus on the application of value of information to Gaussian Process classifiers and explore the effectiveness of the method on the task of classifying voice messages.

BLEWS: Using Blogs to Provide Context for News Articles

Michael Gamon, Sumit Basu, Dmitriy Belenko +2 more

Artificial Intelligence · Proceedings of the International AAAI Conference on Web and Social Media · · cited 99× · DOI ↗ · more like this ·

An overwhelming number of news articles are available every day via the internet. Unfortunately, it is impossible for us to peruse more than a handful; furthermore it is difficult to ascertain an article's social context, i.e., is it popular, what sorts of people are reading it, etc. In this paper, we develop a system to address this problem in the restricted domain of political news by harnessing implicit and explicit contextual information from the blogosphere. Specifically, we track thousands of blogs and the news articles they cite, collapsing news articles that have highly overlapping content. We then tag each article with the number of blogs citing it, the political orientation of those blogs, and the level of emotional charge expressed in the blog posts that link to the news article. We summarize and present the results to the user via a novel visualization which displays this…

3D modeling and tracking of human lip motions

Sumit Basu, Nuria Oliver, Alex Pentland

Signal Processing · · cited 65× · DOI ↗ · more like this ·

We address the problem of tracking and reconstructing 3D human lip motions from a 2D view. This problem is challenging due both to the complex nature of lip motions and the minimal data available from a raw video stream of the face. We counter both of these difficulties with statistical approaches. We first build a physically-based 3D model of lips and train it to cover only the subspace of lip motions. We then track this model in video by finding the shape within the subspace that maximizes the posterior probability of the model given the observed features. In this study, the features are the likelihoods of the lip and non-lip color classes: we iteratively derive forces from these values to apply to the physical model and converge to the final solution. Because of the full 3D nature of the model, this framework allows us to track the lips from any head pose. In addition, because of the…

A linked-HMM model for robust voicing and speech detection

Sumit Basu

Signal Processing · 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03). · · cited 57× · DOI ↗ · more like this ·

We present a novel method for simultaneous voicing and speech detection based on a linked-HMM architecture, with robust features that are independent of the signal energy. Because this approach models the change in dynamics between speech and nonspeech regions, it is robust to low sampling rates, significant levels of additive noise, and large distances from the microphone. We demonstrate the performance of our method in a variety of testing conditions and also compare it to other methods reported in the literature.

Teaching Classification Boundaries to Humans

Sumit Basu, Janara Christensen

Artificial Intelligence · Proceedings of the AAAI Conference on Artificial Intelligence · · cited 56× · DOI ↗ · more like this ·

Given a classification task, what is the best way to teach the resulting boundary to a human? While machine learning techniques can provide excellent methods for finding the boundary, including the selection of examples in an online setting, they tell us little about how we would teach a human the same task. We propose to investigate the problem of example selection and presentation in the context of teaching humans, and explore a variety of mechanisms in the interests of finding what may work best. In particular, we begin with the baseline of random presentation and then examine combinations of several mechanisms: the indication of an example’s relative difficulty, the use of the shaping heuristic from the cognitive science literature (moving from easier examples to harder ones), and a novel kernel-based “coverage model” of the subject’s mastery of the task. From our experiments on 54…

A cascade image transform for speaker independent automatic speechreading

Gerasimos Potamianos, Ashish Verma, C. Neti +2 more

Signal Processing · · cited 53× · DOI ↗ · more like this ·

We propose a three-stage pixel based visual front end for automatic speechreading (lipreading) that results in improved recognition performance of spoken words or phonemes. The proposed algorithm is a cascade of three transforms applied to a three-dimensional video region of interest that contains the speaker's mouth area. The first stage is a typical image compression transform that achieves a high "energy", reduced-dimensionality representation of the video data. The second stage is a linear discriminant analysis based data projection, which is applied to a concatenation of a small number of consecutive image transformed video data. The third stage is a data rotation by means of a maximum likelihood linear transform. Such a transform optimizes the likelihood of the observed data under the assumption of their class conditional Gaussian distribution with diagonal covariance. We apply…

Mind the Gap: Learning to Choose Gaps for Question Generation

Lee A. Becker, Sumit Basu, Lucy Vanderwende

Artificial Intelligence · North American Chapter of the Association for Computational Linguistics · · cited 50× · OpenAlex ↗ · more like this ·

Not all learning takes place in an educational setting: more and more self-motivated learners are turning to on-line text to learn about new topics. Our goal is to provide such learners with the well-known benefits of testing by automatically generating quiz questions for online text. Prior work on question generation has focused on the grammaticality of generated questions and generating effective multiple-choice distractors for individual question targets, both key parts of this problem. Our work focuses on the complementary aspect of determining what part of a sentence we should be asking about in the first place; we call this “gap selection. ” We address this problem by asking human judges about the quality of questions generated from a Wikipedia-based corpus, and then training a model to effectively replicate these judgments. Our data shows that good gaps are of variable length and…

Audio-visual large vocabulary continuous speech recognition in the broadcast domain

Sumit Basu, C. Neti, Nitendra Rajput +3 more

Signal Processing · · cited 37× · DOI ↗ · more like this ·

Considers the problem of combining visual cues with audio signals for the purpose of improved automatic machine recognition of speech. Although significant progress has been made in the machine transcription of large-vocabulary continuous speech (LVCSR) over the last few years, the technology to date is most effective only under controlled conditions, such as low noise, speaker-dependent recognition, read speech (as opposed to conversational speech), etc. On the other hand, while augmenting the recognition of speech utterances with visual cues has attracted the attention of researchers over the last couple of years, most efforts in this domain can be considered to be only preliminary in the sense that, unlike LVCSR efforts, tasks have been limited to small vocabularies (e.g. commands, digits) and often to speaker-dependent training or isolated word speech, where word boundaries are…

Exposing parameters of a trained dynamic model for interactive music creation

Dan Morris, Ian Simon, Sumit Basu

Signal Processing · National Conference on Artificial Intelligence · · cited 28× · OpenAlex ↗ · more like this ·

As machine learning (ML) systems emerge in end-user applications, learning algorithms and classifiers will need to be robust to an increasingly unpredictable operating environment. In many cases, the parameters governing a learning system cannot be optimized for every user scenario, nor can users typically manipulate parameters defined in the space and terminology of ML. Conventional approaches to user-oriented ML systems have typically hidden this complexity from users by automating parameter adjustment. We propose a new paradigm, in which model and algorithm parameters are exposed directly to end-users with intuitive labels, suitable for applications where parameters cannot be automatically optimized or where there is additional motivation – such as creative flexibility – to expose, rather than fix or automatically adapt, learning parameters. In our CHI 2008 paper, we introduced and…

Assisting Users with Clustering Tasks by Combining Metric Learning and Classification

Sumit Basu, Danyel Fisher, Steven M. Drucker +1 more

Computer Vision and Pattern Recognition · Proceedings of the AAAI Conference on Artificial Intelligence · · cited 27× · DOI ↗ · more like this ·

Interactive clustering refers to situations in which a human labeler is willing to assist a learning algorithm in automatically clustering items. We present a related but somewhat different task, assisted clustering, in which a user creates explicit groups of items from a large set and wants suggestions on what items to add to each group. While the traditional approach to interactive clustering has been to use metric learning to induce a distance metric, our situation seems equally amenable to classification. Using clusterings of documents from human subjects, we found that one or the other method proved to be superior for a given cluster, but not uniformly so. We thus developed a hybrid mechanism for combining the metric learner and the classifier. We present results from a large number of trials based on human clusterings, in which we show that our combination scheme matches and often…

A three-dimensional model of human lip motions trained from video

Sumit Basu, Alex Pentland

Signal Processing · · cited 27× · DOI ↗ · more like this ·

We present a 3D model of human lips and develop a framework for training it from real data. The model starts off with generic physics specified with the finite element method and "learns" the correct physics through observations. The model's physics allow physically-based regularization between sparse observation points and the resulting set of deformations are used to derive the correct physical modes of the model. Preliminary results showing the model's ability to reconstruct lip shapes from sparse data are shown. The resulting model can be used for both analysis and synthesis.

Vision-Steered Beam Forming and Transaural Rendering for the Artificial Life Interactive Video Environment (ALIVE)

Michael A. Casey, William G. Gardner, Sumit Basu

Computer Vision and Pattern Recognition · Journal of the Audio Engineering Society · · cited 20× · OpenAlex ↗ · more like this ·

This paper describes the audio component of a virtual reality system that uses remote sensing to free the user from body-mounted tracking equipment. Position information is obtained from a camera and used to constrain a beam-forming microphone array, for far-field speech input, and a two-speaker transaural audio system for rendering 3D audio.

Learning to generalize for complex selection tasks

Alan Ritter, Sumit Basu

Artificial Intelligence · · cited 18× · DOI ↗ · more like this ·

Selection tasks are common in modern computer interfaces: we are often required to select a set of files, emails, data entries, and the like. File and data browsers have sorting and block selection facilities to make these tasks easier, but for complex selections there is little to aid the user without writing complex search queries. We propose an interactive machine learning solution to this problem called "smart selection," in which the user selects and deselects items as inputs to a selection classifier which attempts at each step to correctly generalize to the user's target state. Furthermore, we take advantage of our data on how users perform selection tasks over many sessions, and use it to train a label regressor that models their generalization behavior: we call this process learning to generalize. We then combine the user's explicit labels as well the label regressor outputs in…

Extension of the nu-SVM range for classification

Fernando Pérez‐Cruz, Jason Weston, Djl Herrmann +6 more

Artificial Intelligence · Acta Haematologica · · cited 18× · DOI ↗ · more like this ·

Lymphocytes from children with Chediak-Higashi syndrome (CHS) have impaired natural killer (NK) activity and lack antibody-dependent cell-mediated cytotoxicity. Study of T cells bearing an alternate T cell receptor comprised of gamma- and delta-chains, which typically demonstrate NK activity in vitro, was undertaken in CHS patients. We demonstrate that the cellular machinery for lysis of target cells in vitro is present in CHS-derived gamma delta T cell clones. We also show that the proportion of gamma delta T cells among peripheral blood mononuclear cells is significantly increased in CHS, the first example of a specific immunodeficiency disorder with a relative expansion of these T cells.

Learning communities: connectivity and dynamics of interacting agents

Tanzeem Choudhury, Brian Clarkson, Sumit Basu +1 more

Artificial Intelligence · · cited 16× · DOI ↗ · more like this ·

Intelligent agents need to learn how the communication structure evolves within interacting groups and how to influence the groups overall behavior. We are developing methods to automatically and unobtrusively learn the social network structure that arises within a human group based on wearable sensors. Computational models of group interaction dynamics are derived from data gathered using wearable sensors. The questions we are exploring are: Can we tell who influences whom? Can we quantify this amount of influence? How can we modify group interactions to promote better information diffusion? The goal is real-time learning and modification of social network relationships by applying statistical machine learning techniques to data obtained from unobtrusive wearable sensors.

Why did my pc suddenly slow down

Sumit Basu, John Dunagan, Greg Smith

Computer Networks and Communications · USENIX workshop on Tackling computer systems problems with machine learning techniques · · cited 13× · OpenAlex ↗ · more like this ·

Users are often frustrated when they encounter a sudden decrease in the responsiveness of their personal computers. However, it is often difficult to pinpoint a particular offending process and the resource it is overconsuming, even when such a simple explanation does exist. We present preliminary results from several weeks of PC usage showing that user-perceived unresponsiveness often has such a simple explanation and that simple statistical models often suffice to pinpoint the problem. The statistical models we build use all the performance counters for all running processes. When the user expresses frustration at a given time point, we can use these models to determine which processes are acting most anomalously, and in turn which features of those processes are most anomalous. We present an investigative tool that ranks processes and features according to their degree of anomaly…

AUDIO ANALOGIES: CREATING NEW MUSIC FROM AN EXISTING PERFORMANCE BY CONCATENATIVE SYNTHESIS

Ian Simon, Sumit Basu, David Salesin +1 more

Signal Processing · The Journal of the Abraham Lincoln Association · · cited 13× · OpenAlex ↗ · more like this ·

This paper describes a method for creating new music by concatenative synthesis. Given a MIDI score and an audio recording of an example piece of monophonic music, our method synthesizes audio to correspond with a new MIDI score. The algorithm we use is based on concatenative synthesis, commonly used for generating speech. Two versions of our algorithm are explored, one in which individual notes from the example piece are concatenated, and one in which pairs of adjacent notes from the example piece are concatenated. We examine the range of example pieces and target scores for which each version of our algorithm yields good results. Our underlying framework remains general enough to be applicable to other problems, such as rendering a stylized version of the target score, and other types of sound analogies. 1.

Vision-Steered Audio for Interactive Environments

Sumit Basu, Michael A. Casey, William A. Gardner +2 more

Computer Vision and Pattern Recognition · · cited 12× · OpenAlex ↗ · more like this ·

We present novel techniques for obtaining and producing audio information in an interactive virtual environment using vision information. These techniques are free of mechanisms that would encumber the user, such as clip-on microphones, headphones, etc. Methods are described for both extracting sound from a given position in space and for rendering an "auditory scene," i.e., given a user location, producing sounds that appear to the user to becoming from an arbitrary point in 3-D space. In both cases, vision information about user position is used to guide the algorithms, resulting in solutions to problems that are difficult and often impossible to robustly solve in the auditory domain alone.

Automatically Extracting Fields from Unknown Network Protocols

Karthik Gopalratnam, Sumit Basu, John Dunagan +1 more

Computer Networks and Communications · · cited 12× · OpenAlex ↗ · more like this ·

There are thousands of network protocols in active use on the internet. System administrators often need to extract information from particular fields in such protocols without having sufficient information or time to programatically parse the packets. We propose an active learning framework to perform this extraction in an unknown protocol, in which the user presents the system with a small number of labeled instances. Our system then automatically generates an abundance of features and negative examples; we then use a boosting approach for feature selection and classifier combination. The system then displays its results for the user to correct and/or add new examples and iterate. In our preliminary experiments on DNS queries and responses, we achieve less than 0.1% generalization error using only a handful of labeled examples and thus a minimum of user effort. This translates to…