arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Odest

102 papers · 2000–2026

2000: 22001: 02002: 12003: 32004: 22005: 12006: 32007: 92008: 102009: 02010: 52011: 22012: 42013: 22014: 02015: 12016: 02017: 22018: 82019: 62020: 82021: 52022: 82023: 82024: 52025: 42026: 3
2000peak 10 in 20082026

Frequent co-authors

Papers

showing 1–25 of 102 RSS

A spatio-temporal extension to Isomap nonlinear dimension reduction

Odest Chadwicke Jenkins, Maja J. Matarić

Computer Vision and Pattern Recognition · · cited 187× · DOI ↗ · more like this ·

We present an extension of Isomap nonlinear dimension reduction (Tenenbaum et al., 2000) for data with both spatial and temporal relationships. Our method, ST-Isomap, augments the existing Isomap framework to consider temporal relationships in local neighborhoods that can be propagated globally via a shortest-path mechanism. Two instantiations of ST-Isomap are presented for sequentially continuous and segmented data. Results from applying ST-Isomap to real-world data collected from human motion performance and humanoid robot teleoperation are also presented.

Deriving action and behavior primitives from human motion data

Odest Chadwicke Jenkins, Maja J. Matarić

Computer Vision and Pattern Recognition · · cited 165× · DOI ↗ · more like this ·

We address the problem of creating basis behaviors for modularizing humanoid robot control and representing human activity. These behaviors, called perceptual-motor primitives, serve as a substrate for linking a system's perception of human activities and the ability to perform those activities. We present a data-driven method for deriving perceptual-motor action and behavior primitives from human motion capture data. In order to find these primitives, we employ a spatio-temporal non-linear dimension reduction technique on a set of motion segments. From this transformation, motions representing the same action can be clustered and generalized. Further dimension reduction iterations are applied to derive extended-duration behaviors.

Physical simulation for probabilistic motion tracking

Marek Vondrak, Leonid Sigal, Odest Chadwicke Jenkins

Computer Vision and Pattern Recognition · Defense Technical Information Center (DTIC) · · cited 116× · DOI ↗ · more like this ·

Human motion tracking is an important problem in computer vision. Most prior approaches have concentrated on efficient inference algorithms and prior motion models; however, few can explicitly account for physical plausibility of recovered motion. The primary purpose of this work is to enforce physical plausibility in the tracking of a single articulated human subject. Towards this end, we propose a full-body 3D physical simulation-based prior that explicitly incorporates motion control and dynamics into the Bayesian filtering framework. We consider the humanpsilas motion to be generated by a ldquocontrol looprdquo. In this control loop, Newtonian physics approximates the rigid-body motion dynamics of the human and the environment through the application and integration of forces. Collisions generate interaction forces to prevent physically impossible hypotheses. This allows us to…

VLMbench: A Compositional Benchmark for Vision-and-Language Manipulation

Kaizhi Zheng, Xiaotong Chen, Odest Chadwicke Jenkins +1 more

cs.RO · · cited 107× · arXiv ↗ · more like this ·

Benefiting from language flexibility and compositionality, humans naturally intend to use language to command an embodied agent for complex tasks such as navigation and object manipulation. In this work, we aim to fill the blank of the last mile of embodied agents -- object manipulation by following human guidance, e.g., "move the red mug next to the box while keeping it upright." To this end, we introduce an Automatic Manipulation Solver (AMSolver) system and build a Vision-and-Language Manipulation benchmark (VLMbench) based on it, containing various language instructions on categorized robotic manipulation tasks. Specifically, modular rule-based task templates are created to automatically generate robot demonstrations with language instructions, consisting of diverse object shapes and appearances, action types, and motion constraints. We also develop a keypoint-based model 6D-CLIPort…

Markerless kinematic model and motion capture from volume sequences

Chi-Wei Chun, Odest Chadwicke Jenkins, Maja J. Matarić

Computer Vision and Pattern Recognition · · cited 88× · DOI ↗ · more like this ·

An approach for model-free markerless motion capture of articulated kinematic structures is presented. This approach is centered our method for generating underlying nonlinear axes (or a skeleton curve) from the volume of an arbitrary rigid-body model. We describe the use of skeleton curves for deriving a kinematic model and motion (in the form of joint angles over time) from a captured volume sequence. Our motion capture method uses a skeleton curve, found in each frame of a volume sequence, to automatically determine kinematic postures. These postures are then aligned to determine a common kinematic model for the volume sequence. The derived kinematic model is then reapplied to each frame in the volume sequence to find the motion suited to this model. We demonstrate our method for several types of motion from synthetically generated volume sequences with arbitrary kinematic topology…

Incremental learning of subtasks from unsegmented demonstration

Daniel H. Grollman, Odest Chadwicke Jenkins

Artificial Intelligence · · cited 87× · DOI ↗ · more like this ·

We propose to incrementally learn the segmentation of a demonstrated task into subtasks and the individual subtask policies themselves simultaneously. Previous robot learning from demonstration techniques have either learned the individual subtasks in isolation, combined known subtasks, or used knowledge of the overall task structure to perform segmentation. Our infinite mixture of experts approach instead automatically infers an appropriate partitioning (number of subtasks and assignment of data points to each one) directly from the data. We illustrate the applicability of our technique by learning a suitable set of subtasks from the demonstration of a finite-state machine robot soccer goal scorer.

Learning Behavior Trees From Demonstration

Kevin French, Shiyu Wu, Tianyang Pan +2 more

Artificial Intelligence · · cited 78× · DOI ↗ · more like this ·

Robotic Learning from Demonstration (LfD) allows anyone, not just experts, to program a robot for an arbitrary task. Many LfD methods focus on low level primitive actions such as manipulator trajectories. Complex multistep task with many primitive actions must be learned from demonstration if LfD is to encompass the full range of task a user may desire. Existing methods represent the high level task in various forms including, finite state machines, decision trees, formal logic, among others. Behavior trees are proposed as an alternative representation of high level task. Behavior trees are an execution model for the control of a robot designed for real time execution, modularity, and, consequently, transparency. Real time execution allows the robot to reactively perform the task. Modularity allows the reuse of learned primitive actions and high level task in new situations, speeding up…

Semantic Linking Maps for Active Visual Object Search

Zhen Zeng, Adrian Röfer, Odest Chadwicke Jenkins

cs.RO · · cited 63× · arXiv ↗ · more like this ·

We aim for mobile robots to function in a variety of common human environments. Such robots need to be able to reason about the locations of previously unseen target objects. Landmark objects can help this reasoning by narrowing down the search space significantly. More specifically, we can exploit background knowledge about common spatial relations between landmark and target objects. For example, seeing a table and knowing that cups can often be found on tables aids the discovery of a cup. Such correlations can be expressed as distributions over possible pairing relationships of objects. In this paper, we propose an active visual object search strategy method through our introduction of the Semantic Linking Maps (SLiM) model. SLiM simultaneously maintains the belief over a target object's location as well as landmark objects' locations, while accounting for probabilistic inter-object…

Sparse incremental learning for interactive robot control policy estimation

Daniel H. Grollman, Odest Chadwicke Jenkins

Artificial Intelligence · · cited 58× · DOI ↗ · more like this ·

We are interested in transferring control policies for arbitrary tasks from a human to a robot. Using interactive demonstration via teleoperation as our transfer scenario, we cast learning as statistical regression over sensor-actuator data pairs. Our desire for interactive learning necessitates algorithms that are incremental and realtime. We examine locally weighted projection regression, a popular robotic learning algorithm, and sparse online Gaussian processes in this domain on one synthetic and several robot-generated data sets. We evaluate each algorithm in terms of function approximation, learned task performance, and scalability to large data sets.

Primitive-Based Movement Classification for Humanoid Imitation

Odest Chadwicke Jenkins, Maja J. Matarić, Stefan Weber

Computer Vision and Pattern Recognition · · cited 54× · OpenAlex ↗ · more like this ·

. Motor control is a complex problem and imitation is a powerful mechanism for acquiring new motor skills. In this paper, we describe perceptuo-motor primitives, a biologically-inspired notion for a basis set of perceptual and motor routines. Primitives serve as a vocabulary for classifying and imitating observed human movements, and are derived from the imitator's motor repertoire. We describe a model of imitation based on such primitives and demonstrate the feasibility of the model in a constrained implementation. We present approximate motion reconstruction generated from visually captured data of typically imitated tasks taken from aerobics, dancing, and athletics. 1 Introduction Imitation is a powerful mechanism for acquiring new skills. It involves an intricate interaction between perceptual and motor mechanisms, both of which are complex in themselves. Research into…

Learning from demonstration using a multi-valued function regressor for time-series data

Jesse Butterfield, Sarah Osentoski, Graylin Jay +1 more

Artificial Intelligence · · cited 41× · DOI ↗ · more like this ·

Using data collected from human teleoperation, our goal is to learn a control policy that maps perception to actuation. Such policies are potentially multi-valued with regard to perception with a single input mapping to multiple outputs depending on the user's objective at a particular time. We propose a multi-valued function regressor to learn a larger class of robot control policies from human demonstration and extend the Hierarchical Dirichlet Process Hidden Markov Model to discover latent variables representing unknown objectives in the demonstrated data and the transitions between these objectives. Each of these objectives requires only a single-valued policy function, and thus can be learned with a Gaussian process function regressor. The learned transitions between these objectives determine the correct actuation where the complete policy function is multi-valued. We present the…

Learning robot soccer skills from demonstration

Daniel H. Grollman, Odest Chadwicke Jenkins

Artificial Intelligence · · cited 39× · DOI ↗ · more like this ·

We seek to enable users to teach personal robots arbitrary tasks so that the robot can better perform as the user desires without explicit programming. Robot learning from demonstration is an approach well-suited to this paradigm, as a robot learns new tasks from observations of the task itself. Many current robot learning algorithms require the existence of basic behaviors that can be combined to perform the desired task. However, robots that exist in the world for long timeframes and learn many tasks over their lifetime may exhaust this basis set and need to move beyond it. In particular, we are interested in a robot that must learn to perform an unknown task for which its built in behaviors may not be appropriate. We demonstrate a learning paradigm that is capable of learning both low-level motion primitives (locomotion and manipulation) and high-level tasks built on top of them from…

Automated Derivation of Primitives for Movement Classification

Ajo Fod, Maja J. Matarić, Odest Chadwicke Jenkins

Computer Vision and Pattern Recognition · · cited 34× · OpenAlex ↗ · more like this ·

1 Introduction Programming and interacting with robots, especially humanoid ones, is a complex problem. Using learning toaddress this problem is a popular approach, but the high dimensionality of humanoid control makes the approach prohibitively slow. Imitation, the process of learning new movement patterns and skills by observation, is a promis-ing alternative. The ability to imitate enables a robot to greatly reduce the space of possible trajectories to a subset that approximates that of the observed demonstration. Refinement through trial and error is still likely to be re-quired, but in a greatly reduced learning space.

RoboFrameNet: Verb-centric semantics for actions in robot middleware

Brian J. Thomas, Odest Chadwicke Jenkins

Artificial Intelligence · · cited 31× · DOI ↗ · more like this ·

Advancements in robotics have led to an ever-growing repertoire of software capabilities (e.g., recognition, mapping, and object manipulation). However, robotic capabilities grow, the complexity of operating and interacting with such robots increases (such as through speech, gesture, scripting, or programming). Language-based communication can offer users the ability to work with physically and computationally complex robots without diminishing the robot's inherent capability. However, it remains an open question how to build a common ground between natural language and goal-directed robot actions, particularly in a way that scales with the growth of robot capabilities. We examine using semantic frames - a linguistics concept which describes scenes being acted out - as a conceptual stepping stone between natural language and robot action. We examine the scalability of this solution…

Tracking human motion and actions for interactive robots

Odest Chadwicke Jenkins, Germán González, Matthew Loper

Computer Vision and Pattern Recognition · · cited 30× · DOI ↗ · more like this ·

A method is presented for kinematic pose estimation and action recognition from monocular robot vision through the use of dynamical human motion vocabularies. We propose the utilization of dynamical motion vocabularies towards bridging the decision making of observed humans and information from robot sensing. Our motion vocabulary is comprised of learned primitives that structure the action space for decision making and describe human movement dynamics. Given image observations over time, each primitive infers on pose independently using its prediction density on movement dynamics in the context of a particle filter. Pose estimates from a set of primitives inferencing in parallel are arbitrated to estimate the action being performed. The efficacy of our approach is demonstrated through tracking and action recognition over extended motion trials. Results evidence the robustness of the…

LIT: Light-field Inference of Transparency for Refractive Object Localization

Zheming Zhou, Xiaotong Chen, Odest Chadwicke Jenkins

cs.RO · · cited 26× · arXiv ↗ · more like this ·

Translucency is prevalent in everyday scenes. As such, perception of transparent objects is essential for robots to perform manipulation. Compared with texture-rich or texture-less Lambertian objects, transparency induces significant uncertainty on object appearances. Ambiguity can be due to changes in lighting, viewpoint, and backgrounds, each of which brings challenges to existing object pose estimation algorithms. In this work, we propose LIT, a two-stage method for transparent object pose estimation using light-field sensing and photorealistic rendering. LIT employs multiple filters specific to light-field imagery in deep networks to capture transparent material properties, with robust depth and pose estimators based on generative sampling. Along with the LIT algorithm, we introduce the light-field transparent object dataset ProLIT for the tasks of recognition, localization and pose…

GRIP: Generative Robust Inference and Perception for Semantic Robot Manipulation in Adversarial Environments

Xiaotong Chen, Rui Chen, Zhiqiang Sui +4 more

cs.RO · · cited 26× · arXiv ↗ · more like this ·

Recent advancements have led to a proliferation of machine learning systems used to assist humans in a wide range of tasks. However, we are still far from accurate, reliable, and resource-efficient operations of these systems. For robot perception, convolutional neural networks (CNNs) for object detection and pose estimation are recently coming into widespread use. However, neural networks are known to suffer overfitting during training process and are less robust within unseen conditions, which are especially vulnerable to adversarial scenarios. In this work, we propose Generative Robust Inference and Perception (GRIP) as a two-stage object detection and pose estimation system that aims to combine relative strengths of discriminative CNNs and generative inference methods to achieve robust estimation. Our results show that a second stage of sample-based generative inference is able to…

Dynamical Simulation Priors for Human Motion Tracking

Marek Vondrak, Leonid Sigal, Odest Chadwicke Jenkins

Computer Vision and Pattern Recognition · IEEE Transactions on Pattern Analysis and Machine Intelligence · · cited 26× · DOI ↗ · more like this ·

We propose a simulation-based dynamical motion prior for tracking human motion from video in presence of physical ground-person interactions. Most tracking approaches to date have focused on efficient inference algorithms and/or learning of prior kinematic motion models; however, few can explicitly account for the physical plausibility of recovered motion. Here, we aim to recover physically plausible motion of a single articulated human subject. Toward this end, we propose a full-body 3D physical simulation-based prior that explicitly incorporates a model of human dynamics into the Bayesian filtering framework. We consider the motion of the subject to be generated by a feedback “control loop” in which Newtonian physics approximates the rigid-body motion dynamics of the human and the environment through the application and integration of interaction forces, motor forces, and gravity.…

Efficient nonparametric belief propagation for pose estimation and manipulation of articulated objects

Karthik Desingh, Shiyang Lu, Anthony W. Opipari +1 more

Human-Computer Interaction · Science Robotics · · cited 25× · DOI ↗ · more like this ·

Robots working in human environments often encounter a wide range of articulated objects, such as tools, cabinets, and other jointed objects. Such articulated objects can take an infinite number of possible poses, as a point in a potentially high-dimensional continuous space. A robot must perceive this continuous pose to manipulate the object to a desired pose. This problem of perception and manipulation of articulated objects remains a challenge due to its high dimensionality and multimodal uncertainty. Here, we describe a factored approach to estimate the poses of articulated objects using an efficient approach to nonparametric belief propagation. We consider inputs as geometrical models with articulation constraints and observed RGBD (red, green, blue, and depth) sensor data. The described framework produces object-part pose beliefs iteratively. The problem is formulated as a…

INTERACTIVE HUMAN POSE AND ACTION RECOGNITION USING DYNAMICAL MOTION PRIMITIVES

Odest Chadwicke Jenkins, Germán González, Matthew Loper

Computer Vision and Pattern Recognition · International Journal of Humanoid Robotics · · cited 23× · DOI ↗ · more like this ·

There is currently a division between real-world human performance and the decision making of socially interactive robots. This circumstance is partially due to the difficulty in estimating human cues, such as pose and gesture, from robot sensing. Towards bridging this division, we present a method for kinematic pose estimation and action recognition from monocular robot vision through the use of dynamical human motion vocabularies. Our notion of a motion vocabulary is comprised of movement primitives that structure a human's action space for decision making and predict human movement dynamics. Through prediction, such primitives can be used to both generate motor commands for specific actions and perceive humans performing those actions. In this paper, we focus specifically on the perception of human pose and performed actions using a known vocabulary of primitives. Given image…

Axiomatic particle filtering for goal-directed robotic manipulation

Zhiqiang Sui, Odest Chadwicke Jenkins, Karthik Desingh

Artificial Intelligence · · cited 22× · DOI ↗ · more like this ·

Manipulation tasks involving sequential pick-and-place actions in human environments remains an open problem for robotics. Central to this problem is the inability for robots to perceive in cluttered environments, where objects are physically touching, stacked, or occluded from the view. Such physical interactions currently prevent robots from distinguishing individual objects such that goal-directed reasoning over sequences of pick-and-place actions can be performed. Addressing this problem, we introduce the Axiomatic Particle Filter (APF) as a method for axiomatic state estimation to simultaneously perceive objects in clutter and perform sequential reasoning for manipulation. The APF estimates state as a scene graph, consisting of symbolic spatial relations between objects in the robot's world. Assuming known object geometries, the APF is able to infer a distribution over possible…

Multi-robot belief propagation for distributed robot allocation

Jonas Schwertfeger, Odest Chadwicke Jenkins

Artificial Intelligence · · cited 19× · DOI ↗ · more like this ·

Establishing collaborative behavior is an important factor in coordinating teams of robots. Multi-robot task allocation is one aspect of group coordination that deals with the assignment of robots to subtasks. Toward this end, we propose multi-robot belief propagation (MRBP), a synthesis of distributed probabilistic inference and notions of theory-of-mind for the purpose of multi-robot task allocation. MRBP does not rely upon a central planner or fixed decision hierarchy, but allows to inter task assignments locally through Bayesian belief propagation. The belief propagation algorithm provides us with a tool to incorporate task-related intentions and beliefs of robots into the assignment process. An assignment process in which each individual robot probabilistically decides, based on its own observations and the intentions and beliefs of other robots, which task it should allocate…

Plenoptic Monte Carlo Object Localization for Robot Grasping under Layered Translucency

Zheming Zhou, Zhiqiang Sui, Odest Chadwicke Jenkins

cs.RO · · cited 16× · arXiv ↗ · more like this ·

In order to fully function in human environments, robot perception will need to account for the uncertainty caused by translucent materials. Translucency poses several open challenges in the form of transparent objects (e.g., drinking glasses), refractive media (e.g., water), and diffuse partial occlusions (e.g., objects behind stained glass panels). This paper presents Plenoptic Monte Carlo Localization (PMCL) as a method for localizing object poses in the presence of translucency using plenoptic (light-field) observations. We propose a new depth descriptor, the Depth Likelihood Volume (DLV), and its use within a Monte Carlo object localization algorithm. We present results of localizing and manipulating objects with translucent materials and objects occluded by layers of translucency. Our PMCL implementation uses observations from a Lytro first generation light field camera to allow a…

EURECA: Enhanced Understanding of Real Environments via Crowd Assistance

Sai R. Gouravajhala, Jinyeong Yim, Karthik Desingh +3 more

Computer Vision and Pattern Recognition · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · · cited 15× · DOI ↗ · more like this ·

Indoor robots hold the promise of automatically handling mundane daily tasks, helping to improve access for people with disabilities, and providing on-demand access to remote physical environments. Unfortunately, the ability to understand never-before-seen objects in scenes where new items may be added (e.g., purchased) or altered (e.g., damaged) on a regular basis remains an open challenge for robotics. In this paper, we introduce EURECA, a mixed-initiative system that leverages online crowds of human contributors to help robots robustly identify 3D point cloud segments corresponding to user-referenced objects in near real-time. EURECA allows robots to understand multi-object 3D scenes on-the-fly (in ~40 seconds) by providing groups of non-expert crowd workers with intelligent tools that can segment objects more quickly (~70% faster) and more accurately (6% higher F1 score) than…

Roomba Pac-Man: Teaching Autonomous Robotics through Embodied Gaming

Brendan Charles Dickinson, Odest Chadwicke Jenkins, Mark Moseley +2 more

Computer Vision and Pattern Recognition · · cited 13× · OpenAlex ↗ · more like this ·

We present an approach to teaching autonomous robotics to upper-level undergraduates through the medium of embodied games. As part of a developing course at Brown University, we have created the Roomba Pac-Man task to introduce students to different approaches to autonomous robot control in the context of a specific task. Roomba Pac-Man has been developed using commodity hardware from which students explore standard methods in robotics, namely subsumption, localization, and path planning. Our development of Roomba Pac-Man is founded upon grounding robotics in a compelling and accessible application in a non-contrived real-world environment in a manner than can be reproduced, giving students a sense of ownership.