arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Li-wei He

56 papers · 1996–2026

1996: 11997: 11998: 21999: 42000: 42001: 22002: 12003: 12004: 42005: 32006: 62007: 32008: 22009: 02010: 42011: 42012: 22013: 12014: 12015: 12016: 12017: 02018: 02019: 02020: 02021: 02022: 02023: 32024: 22025: 22026: 1
1996peak 6 in 20062026

Frequent co-authors

Papers

showing 1–25 of 56 RSS

Layered depth images

Jonathan Shade, Steven J. Gortler, Li-wei He +1 more

Computer Graphics and Computer-Aided Design · Digital Access to Scholarship at Harvard (DASH) (Harvard University) · · cited 1,100× · DOI ↗ · more like this ·

In this paper we present a set of efficient image based rendering methods capable of rendering multiple frames per second on a PC. The first method warps Sprites with Depth representing smooth surfaces without the gaps found in other techniques. A second method for more general scenes performs warping from an intermediate representation called a Layered Depth Image (LDI). An LDI is a view of the scene from a single input camera view, but with multiple pixels along each line of sight. The size of the representation grows only linearly with the observed depth complexity in the scene. Moreover, because the LDI data are represented in a single image coordinate system, McMillan's warp ordering algorithm can be successfully adapted. As a result, pixels are drawn in the output image in back-to-front order. No z-buffer is required, so alphacompositing can be done efficiently without…

Rendering with concentric mosaics

Heung‐Yeung Shum, Li-wei He

Computer Graphics and Computer-Aided Design · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · · cited 467× · DOI ↗ · more like this ·

This paper presents a novel 3D plenoptic function, which we call concentric mosaics. We constrain camera motion to planar concentric circles, and create concentric mosaics using a manifold mosaic for each circle (i.e., composing slit images taken at different locations). Concentric mosaics index all input image rays naturally in 3 parameters: radius, rotation angle and vertical elevation. Novel views are rendered by combining the appropriate captured rays in an efficient manner at rendering time. Although vertical distortions exist in the rendered images, they can be alleviated by depth correction. Like panoramas, concentric mosaics do not require recovering geometric and photometric scene models. Moreover, concentric mosaics provide a much richer user experience by allowing the user to move freely in a circular region and observe significant parallax and lighting changes. Compared with…

High-quality linear interpolation for demosaicing of Bayer-patterned color images

H.S. Malvar, Li-wei He, Ross Cutler

Computer Vision and Pattern Recognition · · cited 432× · DOI ↗ · more like this ·

This paper introduces a new interpolation technique for demosaicing of color images produced by single-CCD digital cameras. We show that the proposed simple linear filter can lead to an improvement in PSNR of over 5.5 dB when compared to bilinear demosaicing, and about 0.7 dB improvement in R and B interpolation when compared to a recently introduced linear interpolator. The proposed filter also outperforms most nonlinear demosaicing algorithms, without the artifacts due to nonlinear processing, and a much reduced computational complexity.

The virtual cinematographer

Li-wei He, Michael F. Cohen, David Salesin

Computer Vision and Pattern Recognition · · cited 310× · DOI ↗ · more like this ·

This paper presents a paradigm for automatically generating complete camera specifications for capturing events in virtual 3D environments in real-time. We describe a fully implemented system, called the Virtual Cinematographer, and demonstrate its application in a virtual "party" setting. Cinematographic expertise, in the form of film idioms, is encoded as a set of small hierarchically organized finite state machines. Each idiom is responsible for capturing a particular type of scene, such as three virtual actors conversing or one actor moving across the environment. The idiom selects shot types and the timing of transitions between shots to best communicate events as they unfold. A set of camera modules, shared by the idioms, is responsible for the low-level geometric placement of specific cameras for each shot. The camera modules are also responsible for making subtle changes in the…

Distributed meetings

Ross Cutler, Yong Rui, Anoop Gupta +7 more

Computer Vision and Pattern Recognition · · cited 227× · DOI ↗ · more like this ·

The common meeting is an integral part of everyday life for most workgroups. However, due to travel, time, or other constraints, people are often not able to attend all the meetings they need to. Teleconferencing and recording of meetings can address this problem. In this paper we describe a system that provides these features, as well as a user study evaluation of the system. The system uses a variety of capture devices (a novel 360° camera, a whiteboard camera, an overview camera, and a microphone array) to provide a rich experience for people who want to participate in a meeting from a distance. The system is also combined with speaker clustering, spatial indexing, and time compression to provide a rich experience for people who miss a meeting and want to watch it afterward.

Auto-summarization of audio-video presentations

Li-wei He, Elizabeth Sanocki, Anoop Gupta +1 more

Computer Vision and Pattern Recognition · · cited 222× · DOI ↗ · more like this ·

As streaming audio-video technology becomes widespread, there is a dramatic increase in the amount of multimedia content available on the net. Users face a new challenge: How to examine large amounts of multimedia content quickly. One technique that can enable quick overview of multimedia is video summaries; that is, a shorter version assembled by picking important segments from the original. We evaluate three techniques for automatic creation of summaries for online audio-video presentations. These techniques exploit information in the audio signal (e.g., pitch and pause information), knowledge of slide transition points in the presentation, and information about access patterns of previous users. We report a user study that compares automatically generated summaries that are 20%-25% the length of full presentations to author generated summaries. Users learn from the computer-generated…

Distributed nonnegative matrix factorization for web-scale dyadic data analysis on mapreduce

Chao Liu, Hung‐Chih Yang, Jinliang Fan +2 more

Computer Networks and Communications · · cited 207× · DOI ↗ · more like this ·

The Web abounds with dyadic data that keeps increasing by every single second. Previous work has repeatedly shown the usefulness of extracting the interaction structure inside dyadic data [21, 9, 8]. A commonly used tool in extracting the underlying structure is the matrix factorization, whose fame was further boosted in the Netflix challenge [26]. When we were trying to replicate the same success on real-world Web dyadic data, we were seriously challenged by the scalability of available tools. We therefore in this paper report our efforts on scaling up the nonnegative matrix factorization (NMF) technique. We show that by carefully partitioning the data and arranging the computations to maximize data locality and parallelism, factorizing a tens of millions by hundreds of millions matrix with billions of nonzero cells can be accomplished within tens of hours. This result effectively…

Browsing digital video

Francis C. Li, Anoop Gupta, Elizabeth Sanocki +2 more

Computer Vision and Pattern Recognition · · cited 148× · DOI ↗ · more like this ·

Video in digital format played on programmable devices presents opportunities for significantly enhancing the user's viewing experience. For example, time compression and pause removal can shorten the viewing time for a video, textual and visual indices can allow personalized navigation through the content, and random-access digital storage allows instantaneous seeks into the content. To understand user behavior when such capabilities are available, we built a software video browsing application that combines many such features. We present results from a user study where users browsed video in six different categories: classroom lectures, conference presentations, entertainment shows, news, sports, and travel. Our results show that the most frequently used features were time compression, pause removal, and navigation using shot boundaries. Also, the behavior was different depending on…

Optimal rare query suggestion with implicit user feedback

Yang Song, Li-wei He

Information Systems · · cited 85× · DOI ↗ · more like this ·

Query suggestion has been an effective approach to help users narrow down to the information they need. However, most of existing studies focused on only popular/head queries. Since rare queries possess much less information (e.g., clicks) than popular queries in the query logs, it is much more difficult to efficiently suggest relevant queries to a rare query. In this paper, we propose an optimal rare query suggestion framework by leveraging implicit feedbacks from users in the query logs. Our model resembles the principle of pseudo-relevance feedback which assumes that top-returned results by search engines are relevant. However, we argue that the clicked URLs and skipped URLs contain different levels of information and thus should be treated differently. Hence, our framework optimally combines both the click and skip information from users and uses a random walk model to optimize the…

Time-compression

Nosa Omoigui, Li-wei He, Anoop Gupta +2 more

Computer Vision and Pattern Recognition · · cited 85× · DOI ↗ · more like this ·

With the proliferation of online multimedia content and the popularity of multimedia streaming systems, it is increasingly useful to be able to skim and browse multimedia quickly. A key technique that enables quick browsing of multimedia is time-compression. Prior research has described how speech can be time-compressed (shortened in duration) while preserving the pitch of the audio. However, client-server systems providing this functionality have not been available.

An automated end-to-end lecture capture and broadcasting system

Cha Zhang, Rui Yong, J. P. Wickersham Crawford +1 more

Sociology and Political Science · ACM Transactions on Multimedia Computing Communications and Applications · · cited 83× · DOI ↗ · more like this ·

Remote viewing of lectures presented to a live audience is becoming increasingly popular. At the same time, the lectures can be recorded for subsequent on-demand viewing over the Internet. Providing such services, however, is often prohibitive due to the labor-intensive cost of capturing and pre/post-processing. This article presents a complete automated end-to-end system that supports capturing, broadcasting, viewing, archiving and searching of presentations. Specifically, we describe a system architecture that minimizes the pre- and post-production time, and a fully automated lecture capture system called iCam2 that synchronously captures all contents of the lecture, including audio, video, and presentation material. No staff is needed during lecture capture and broadcasting, so the operational cost of the system is negligible. The system has been used on a daily basis for more than 4…

Building an intelligent camera management system

Yong Rui, Li-wei He, Anoop Gupta +1 more

Computer Vision and Pattern Recognition · · cited 68× · DOI ↗ · more like this ·

Given rapid improvements in storage devices, network infrastructure and streaming-media technologies, a large number of corporations and universities are recording lectures and making them available online for anytime, anywhere access. However, producing high-quality lecture videos is still labor intensive and expensive. Fortunately, recent technology advances are making it feasible to build automated camera management systems to capture lectures. In this paper we report our design of such a system, including system configuration, audio-visual tracking techniques, software architecture, and user study. Motivated by different roles in a professional video production team, we have developed a multi-cinematographer single-director camera management system. The system performs lecturer tracking, audience tracking, and video editing all fully automatically, and offers quality close to that…

Query suggestion by constructing term-transition graphs

Yang Song, Dengyong Zhou, Li-wei He

Information Systems · · cited 65× · DOI ↗ · more like this ·

Query suggestion is an interactive approach for search engines to better understand users information need. In this paper, we propose a novel query suggestion framework which leverages user re-query feedbacks from search engine logs. Specifically, we mined user query reformulation activities where the user only modifies part of the query by (1) adding terms after the query, (2) deleting terms within the query, or (3) modifying terms to new terms. We build a term-transition graph based on the mined data. Two models are proposed which address topic-level and term-level query suggestions, respectively. In the first topic-based unsupervised Pagerank model, we perform random walk on each of the topic-based term-transition graph and calculate the Pagerank for each term within a topic. Given a new query, we suggest relevant queries based on its topic distribution and term-transition…

A task level metric for measuring web search satisfaction and its application on improving relevance estimation

Ahmed H. Yousef, Yang Song, Li-wei He

Information Systems · · cited 55× · DOI ↗ · more like this ·

Understanding the behavior of satisfied and unsatisfied Web search users is very important for improving users search experience. Collecting labeled data that characterizes search behavior is a very challenging problem. Most of the previous work used a limited amount of data collected in lab studies or annotated by judges lacking information about the actual intent. In this work, we performed a large scale user study where we collected explicit judgments of user satisfaction with the entire search task. Results were analyzed using sequence models that incorporate user behavior to predict whether the user ended up being satisfied with a search or not. We test our metric on millions of queries collected from real Web search traffic and show empirically that user behavior models trained using explicit judgments of user satisfaction outperform several other search quality metrics. The…

Evaluating the effectiveness of search task trails

Zhen Liao, Yang Song, Li-wei He +1 more

Information Systems · · cited 52× · DOI ↗ · more like this ·

In this paper, we introduce "task trail" as a new concept to understand user search behaviors. We define task to be an atomic user information need. Web search logs have been studied mainly at session or query level where users may submit several queries within one task and handle several tasks within one session. Although previous studies have addressed the problem of task identification, little is known about the advantage of using task over session and query for search applications. In this paper, we conduct extensive analyses and comparisons to evaluate the effectiveness of task trails in three search applications: determining user satisfaction, predicting user search interests, and query suggestion. Experiments are conducted on large scale datasets from a commercial search engine. Experimental results show that: (1) Sessions and queries are not as precise as tasks in determining…

Post-ranking query suggestion by diversifying search results

Yang Song, Dengyong Zhou, Li-wei He

Information Systems · · cited 51× · DOI ↗ · more like this ·

Query suggestion refers to the process of suggesting related queries to search engine users. Most existing researches have focused on improving the relevance of suggested queries. In this paper, we introduce the concept of diversifying the content of the search results from suggested queries while keeping the suggestion relevant. Our framework first retrieves a set of query candidates from search engine logs using random walk and other techniques. We then re-rank the suggested queries by ranking them in the order which maximizes the diversification function that measures the difference between the original search results and the results from suggested queries. The diversification function we proposed includes features like ODP category, URL and domain similarity and so on. One important outcome from our research which contradicts with most existing researches is that, with the increase…

Energy-Based Sound Source Localization and Gain Normalization for Ad Hoc Microphone Arrays

Zicheng Liu, Zhengyou Zhang, Li-wei He +1 more

Signal Processing · · cited 46× · DOI ↗ · more like this ·

We present an energy-based technique to estimate both microphone and speaker/talker locations from an ad hoc network of microphones. An example of such ad hoc microphone network is a set of microphones built in the laptops that some meeting participants bring in a meeting room. Compared with traditional sound source localization approaches based on time of flight, our technique does not require accurate synchronization, and it does not require each laptop to emit special signals. We estimate the meeting participants' positions based on average energies of their speech signals. In addition, we present a technique, which is independent of the volumes of the speakers, to estimate the relative gains of the microphones. This is crucial to aggregate various audio channels from the ad hoc microphone network into a single stream for audio conferencing.

Designing presentations for on-demand viewing

Li-wei He, Jonathan Grudin, Anoop Gupta

Sociology and Political Science · · cited 42× · DOI ↗ · more like this ·

Increasingly often, presentations are given before a live audience, while simultaneously being viewed remotely and recorded for subsequent viewing on-demand over the Web. How should video presentations be designed for web access? How is video accessed and used online? Does optimal design for live and on-demand audiences conflict? We examined detailed behavior patterns of more than 9000 on-demand users of a large corpus of professionally prepared presentations. We find that as many people access these talks on-demand as attend live. Online access patterns differ markedly from live attendance. People watch less overall and skip to different parts of a talk. Speakers designing presentations for viewing on-demand should emphasize key points early in the talk and early within each slide, use slide titles that reveal the talk structure and are meaningful outside the flow of the talk. In some…

Why take notes? Use the whiteboard capture system

Li-wei He, Zicheng Liu, Zhengyou Zhang

Computer Vision and Pattern Recognition · 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03). · · cited 36× · DOI ↗ · more like this ·

The paper describes our recently developed system which captures both whiteboard content and audio signals of a meeting using a digital still camera and a microphone. Our system can be retrofitted to any existing whiteboard. It computes the time stamps of pen strokes on the whiteboard by analyzing the sequence of captured snapshots. It also automatically produces a set of key frames representing all the written content on the whiteboard before each erasure. Therefore, the whiteboard content serves as a visual index to browse the audio meeting efficiently. It is a complete system which not only captures the whiteboard content, but also helps users to view and manage the captured meeting content efficiently and securely.

The Virtual Cinematographer: A Paradigm for Automatic Real-Time Camera Control and Directing

Li-wei He, Michael F. Cohen, David Salesin

Computer Vision and Pattern Recognition · ACM eBooks · · cited 35× · DOI ↗ · more like this ·

This paper presents a paradigm for automatically generating complete camera specifications for capturing events in virtual 3D environments in real-time. We describe a fully implemented system, called the Virtual Cinematographer, and demonstrate its application in a virtual "party" setting. Cinematographic expertise, in the form of film idioms, is encoded as a set of small hierarchically organized finite state machines. Each idiom is responsible for capturing a particular type of scene, such as three virtual actors conversing or one actor moving across the environment. The idiom selects shot types and the timing of transitions between shots to best communicate events as they unfold. A set of camera modules, shared by the idioms, is responsible for the low-level geometric placement of specific cameras for each shot. The camera modules are also responsible for making subtle changes in the…

Comparing presentation summaries

Li-wei He, Elizabeth Sanocki, Anoop Gupta +1 more

Computer Vision and Pattern Recognition · · cited 32× · DOI ↗ · more like this ·

As more audio and video technical presentations go online, it becomes imperative to give users effective summarization and skimming tools so that they can find the presentation they want and browse through it quickly. In a previous study, we reported three automated methods for generating audio-video summaries and a user evaluation of those methods. An open question remained about how well various text/image only techniques will compare to the audio-video summarizations. This study attempts to fill that gap.

Real-Time Whiteboard Capture and Processing Using a Video Camera for Remote Collaboration

Li-Wei He, Zhengyou Zhang

Sociology and Political Science · IEEE Transactions on Multimedia · · cited 31× · DOI ↗ · more like this ·

This paper describes our recently developed system which captures pen strokes on physical whiteboards in real time using an off-the-shelf video camera. Unlike many existing tools, our system does not instrument the pens or the whiteboard. It analyzes the sequence of captured video images in real time, classifies the pixels into whiteboard background, pen strokes and foreground objects (e.g., people in front of the whiteboard), extracts newly written pen strokes, and corrects the color to make the whiteboard completely white. This allows us to transmit whiteboard contents using very low bandwidth to remote meeting participants. Combined with other teleconferencing tools such as voice conference and application sharing, our system becomes a powerful tool to share ideas during online meetings

Exploring benefits of non-linear time compression

Li-wei He, Anoop Gupta

Computer Vision and Pattern Recognition · · cited 29× · DOI ↗ · more like this ·

In comparison to text, audio-video content is much more challenging to browse. Time-compression has been suggested as a key technology that can support browsing-time compression speeds up the playback of audio-video content without causing the pitch to change. Simple forms of time-compression are starting to appear in commercial streaming-media products from Microsoft and Real Networks.In this paper we explore the potential benefits of more recent and advanced types of time compression, called non-linear time compression. The most advanced of these algorithms exploit fine-grain structure of human speech (e.g., phonemes) to differentially speedup segments of speech, so that the overall speedup can be higher. In this paper we explore what are the actual gains achieved by end-users from these advanced algorithms. Our results indicate that the gains are actually quite small in common cases…

Note-taking with a camera: whiteboard scanning and image enhancement

Zhengyou Zhang, Li-wei He

Computer Vision and Pattern Recognition · · cited 27× · DOI ↗ · more like this ·

The paper describes a system for scanning the content on a whiteboard into a computer by a digital camera and also for enhancing the visual quality of whiteboard images. Because digital cameras are becoming accessible to average users, more and more people use digital cameras to take pictures of whiteboards instead of copying manually, thus increasing productivity significantly. However, the images are usually taken from an angle, resulting in undesired perspective distortion. They also contain other distracting regions such as walls. We have developed a system that automatically locates the boundary of a whiteboard, crops out the whiteboard region, rectifies it into a rectangle, and corrects the color to make the whiteboard completely white. In the case where a single image is not enough (e.g., large whiteboard and low-res camera), we have developed a robust feature-based technique to…

Task Trail: An Effective Segmentation of User Search Behavior

Zhen Liao, Yang Song, Yalou Huang +2 more

Information Systems · IEEE Transactions on Knowledge and Data Engineering · · cited 25× · DOI ↗ · more like this ·

In this paper, we introduce “task trail” to understand user search behaviors. We define a task to be an atomic user information need, whereas a task trail represents all user activities within that particular task, such as query reformulations, URL clicks. Previously, web search logs have been studied mainly at session or query level where users may submit several queries within one task and handle several tasks within one session. Although previous studies have addressed the problem of task identification, little is known about the advantage of using task over session or query for search applications. In this paper, we conduct extensive analyses and comparisons to evaluate the effectiveness of task trails in several search applications: determining user satisfaction, predicting user search interests, and suggesting related queries. Experiments on large scale data sets of a commercial…