arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Matthew Loper

11 papers · 2004–2025

2004: 22005: 02006: 12007: 22008: 12009: 02010: 02011: 02012: 02013: 02014: 12015: 12016: 02017: 02018: 02019: 12020: 02021: 12022: 02023: 02024: 02025: 1
2004peak 2 in 20042025

Frequent co-authors

Papers

showing 1–11 of 11 RSS

Progressively-Refined Reflectance Functions from Natural Illumination

Wojciech Matusik, Matthew Loper, Hanspeter Pfister

Computer Graphics and Computer-Aided Design · Digital Access to Scholarship at Harvard (DASH) (Harvard University) · · cited 45× · DOI ↗ · more like this ·

In this paper we present a simple, robust, and efficient algorithm for estimating reflectance fields (i.e., a description of the transport of light through a scene) for a fixed viewpoint using images of the scene under known natural illumination. Our algorithm treats the scene as a black-box linear system that transforms an input signal (the incident light) into an output signal (the reflected light). The algorithm is hierarchical - it progressively refines the approximation of the reflectance field with an increasing number of training samples until the required precision is reached. Our method relies on a new representation for reflectance fields. This representation is compact, can be progressively refined, and quickly computes the relighting of scenes with complex illumination. Our representation and the corresponding algorithm allow us to efficiently estimate the reflectance fields…

Tracking human motion and actions for interactive robots

Odest Chadwicke Jenkins, Germán González, Matthew Loper

Computer Vision and Pattern Recognition · · cited 30× · DOI ↗ · more like this ·

A method is presented for kinematic pose estimation and action recognition from monocular robot vision through the use of dynamical human motion vocabularies. We propose the utilization of dynamical motion vocabularies towards bridging the decision making of observed humans and information from robot sensing. Our motion vocabulary is comprised of learned primitives that structure the action space for decision making and describe human movement dynamics. Given image observations over time, each primitive infers on pose independently using its prediction density on movement dynamics in the context of a particle filter. Pose estimates from a set of primitives inferencing in parallel are arbitrated to estimate the action being performed. The efficacy of our approach is demonstrated through tracking and action recognition over extended motion trials. Results evidence the robustness of the…

INTERACTIVE HUMAN POSE AND ACTION RECOGNITION USING DYNAMICAL MOTION PRIMITIVES

Odest Chadwicke Jenkins, Germán González, Matthew Loper

Computer Vision and Pattern Recognition · International Journal of Humanoid Robotics · · cited 23× · DOI ↗ · more like this ·

There is currently a division between real-world human performance and the decision making of socially interactive robots. This circumstance is partially due to the difficulty in estimating human cues, such as pose and gesture, from robot sensing. Towards bridging this division, we present a method for kinematic pose estimation and action recognition from monocular robot vision through the use of dynamical human motion vocabularies. Our notion of a motion vocabulary is comprised of movement primitives that structure a human's action space for decision making and predict human movement dynamics. Through prediction, such primitives can be used to both generate motor commands for specific actions and perceive humans performing those actions. In this paper, we focus specifically on the perception of human pose and performed actions using a known vocabulary of primitives. Given image…

Learning Realistic Human Reposing using Cyclic Self-Supervision with 3D Shape, Pose, and Appearance Consistency

Soubhik Sanyal, Alex Vorobiov, Timo Bolkart +5 more

cs.CV · · cited 20× · arXiv ↗ · more like this ·

Synthesizing images of a person in novel poses from a single image is a highly ambiguous task. Most existing approaches require paired training images; i.e. images of the same person with the same clothing in different poses. However, obtaining sufficiently large datasets with paired data is challenging and costly. Previous methods that forego paired supervision lack realism. We propose a self-supervised framework named SPICE (Self-supervised Person Image CrEation) that closes the image quality gap with supervised methods. The key insight enabling self-supervision is to exploit 3D information about the human body in several ways. First, the 3D body shape must remain unchanged when reposing. Second, representing body pose in 3D enables reasoning about self occlusions. Third, 3D body parts that are visible before and after reposing, should have similar appearance features. Once trained…

FlowCap: 2D Human Pose from Optical Flow

Javier Romero, Matthew Loper, Michael J. Black

Computer Vision and Pattern Recognition · Lecture notes in computer science · · cited 17× · DOI ↗ · more like this ·

We estimate 2D human pose from video using only optical flow. The key insight is that dense optical flow can provide information about 2D body pose. Like range data, flow is largely invariant to appearance but unlike depth it can be directly computed from monocular video. We demonstrate that body parts can be detected from dense flow using the same random forest approach used by the Microsoft Kinect. Unlike range data, however, when people stop moving, there is no optical flow and they effectively disappear. To address this, our FlowCap method uses a Kalman filter to propagate body part positions and velocities over time and a regression method to predict 2D body pose from part centers. No range sensor is required and FlowCap estimates 2D human pose from monocular video sources containing human motion. Such sources include hand-held phone cameras and archival television video. We…

Monocular virtual trajectory estimation with dynamical primitives

Odest Chadwicke Jenkins, Germán González, Matthew Loper

Computer Vision and Pattern Recognition · · cited 3× · OpenAlex ↗ · more like this ·

We present a method for monocular kinematic pose es-timation and activity recognition from video for move-ment imitation. Learned vocabularies of kinematic mo-tion primitives are used emulate the function of hypoth-esized neuroscientific models for spinal fields and mir-ror neurons in the process of imitation. For imitation, we assume the movement of a demonstrator is produced through a “virtual trajectory ” specifying desired body poses over time. Each pose represents a decomposi-tion into mirror neuron firing coefficients that specify the attractive dynamics to this configuration through a linear combination of primitives. Each primitive is a nonlinear dynamical system that predicts expected mo-tion with respect to an underlying activity. Our aim is to invert this process by estimating a demonstrator’s vir-tual trajectory from monocular image observations in a bottom-up fashion. At…

The Informed Sampler: A Discriminative Approach to Bayesian Inference in Generative Computer Vision Models

Varun Jampani, Sebastian Nowozin, Matthew Loper +1 more

cs.CV · · cited 1× · arXiv ↗ · more like this ·

Computer vision is hard because of a large variability in lighting, shape, and texture; in addition the image signal is non-additive due to occlusion. Generative models promised to account for this variability by accurately modelling the image formation process as a function of latent variables with prior beliefs. Bayesian posterior inference could then, in principle, explain the observation. While intuitively appealing, generative models for computer vision have largely failed to deliver on that promise due to the difficulty of posterior inference. As a result the community has favoured efficient discriminative approaches. We still believe in the usefulness of generative models in computer vision, but argue that we need to leverage existing discriminative or even heuristic computer vision methods. We implement this idea in a principled way with an "informed sampler" and in careful…

Research Comprehensive Final Exam: Action Recognition on a Mobile Robotic Platform

Matthew Loper

Computer Vision and Pattern Recognition · · cited 0× · OpenAlex ↗ · more like this ·

We present an approach for recognizing human action from a mobile platform. Three components are essential to our action recognition pipeline. First, a new off-the-shelf active sensor is employed to obtain observations. Next, we use our observables to generate a set of pose-inspired features. Finally, we use these features to obtain a history over actions, with the use of a Hidden Markov Model and a Gaussian likelihood function. The proposed approach is insensitive to the changing lighting conditions which can be troublesome for color-based methods. To our knowledge, we are the first to develop a mobile, uninstrumented, color-invariant 3D bodymodel based approach for action recognition. 1

Progressively-Refined Reflectance Functions from Natural Illumination

H. W. Jensen, A. Keller (editors, Wojciech Matusik +5 more

Computer Graphics and Computer-Aided Design · · cited 0× · OpenAlex ↗ · more like this ·

In this paper we present a simple, robust and efficient algorithm for estimating reflectance fields (i.e., a description of teh transport of light through a scene) for a fixed viewpoint using images of the scene under known natural illumination. Our algorithm treats the scene as a black-box linear system that transforms an input signal (the incident light) into an output signal (the reflected light). The algorithm is hierarchical- it progressively refines the approximation of the reflectance field with an increasing number of training samples until the required precision is reached. Our method relies on a new representation for reflectance fields. Thsi representation is compact, can be progressively refined, and quickly computes the relighting of scenes with complex illumination. Our representation and the corresponding algorithm allow us to efficiently estimate the reflectance fields…

Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactions

Kaifeng Zhang, Shuo Sha, Hanxiao Jiang +7 more

cs.RO · · arXiv ↗ · more like this ·

Robotic manipulation policies are advancing rapidly, but their direct evaluation in the real world remains costly, time-consuming, and difficult to reproduce, particularly for tasks involving deformable objects. Simulation provides a scalable and systematic alternative, yet existing simulators often fail to capture the coupled visual and physical complexity of soft-body interactions. We present a real-to-sim policy evaluation framework that constructs soft-body digital twins from real-world videos and renders robots, objects, and environments with photorealistic fidelity using 3D Gaussian Splatting. We validate our approach on representative deformable manipulation tasks, including plush toy packing, rope routing, and T-block pushing, demonstrating that simulated rollouts correlate strongly with real-world execution performance and reveal key behavioral patterns of learned policies. Our…

FACSIMILE: Fast and Accurate Scans From an Image in Less Than a Second

David Smith, Matthew Loper, Xiaochen Hu +2 more

cs.CV · · arXiv ↗ · more like this ·

Current methods for body shape estimation either lack detail or require many images. They are usually architecturally complex and computationally expensive. We propose FACSIMILE (FAX), a method that estimates a detailed body from a single photo, lowering the bar for creating virtual representations of humans. Our approach is easy to implement and fast to execute, making it easily deployable. FAX uses an image-translation network which recovers geometry at the original resolution of the image. Counterintuitively, the main loss which drives FAX is on per-pixel surface normals instead of per-pixel depth, making it possible to estimate detailed body geometry without any depth supervision. We evaluate our approach both qualitatively and quantitatively, and compare with a state-of-the-art method.