arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Ruohan Zhang

86 papers · 2015–2026

2015: 12016: 12017: 12018: 42019: 32020: 42021: 32022: 52023: 132024: 152025: 152026: 21
2015peak 21 in 20262026

Frequent co-authors

Papers

showing 1–25 of 86 RSS

Open X-Embodiment: Robotic Learning Datasets and RT-X Models

Embodiment Collaboration, Abby O'Neill, Abdul Rehman +291 more

cs.RO · · cited 1,206× · arXiv ↗ · more like this ·

Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for many applications. Can such a consolidation happen in robotics? Conventionally, robotic learning methods train a separate model for every application, every robot, and even every environment. Can we instead train generalist X-robot policy that can be adapted efficiently to new robots, tasks, and environments? In this paper, we provide datasets in standardized data formats and models to make it possible to explore this possibility in the context of robotic manipulation, alongside experimental results that provide an example of effective X-robot policies. We assemble a dataset from 22 different…

VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models

Wenlong Huang, Chen Wang, Ruohan Zhang +3 more

cs.RO · · cited 1,196× · arXiv ↗ · more like this ·

Large language models (LLMs) are shown to possess a wealth of actionable knowledge that can be extracted for robot manipulation in the form of reasoning and planning. Despite the progress, most still rely on pre-defined motion primitives to carry out the physical interactions with the environment, which remains a major bottleneck. In this work, we aim to synthesize robot trajectories, i.e., a dense sequence of 6-DoF end-effector waypoints, for a large variety of manipulation tasks given an open-set of instructions and an open-set of objects. We achieve this by first observing that LLMs excel at inferring affordances and constraints given a free-form language instruction. More importantly, by leveraging their code-writing capabilities, they can interact with a vision-language model (VLM) to compose 3D value maps to ground the knowledge into the observation space of the agent. The…

New Generation Deep Learning for Video Object Detection: A Survey

Licheng Jiao, Ruohan Zhang, Fang Liu +4 more

Computer Vision and Pattern Recognition · IEEE Transactions on Neural Networks and Learning Systems · · cited 252× · DOI ↗ · more like this ·

Video object detection, a basic task in the computer vision field, is rapidly evolving and widely used. In recent years, deep learning methods have rapidly become widespread in the field of video object detection, achieving excellent results compared with those of traditional methods. However, the presence of duplicate information and abundant spatiotemporal information in video data poses a serious challenge to video object detection. Therefore, in recent years, many scholars have investigated deep learning detection algorithms in the context of video data and have achieved remarkable results. Considering the wide range of applications, a comprehensive review of the research related to video object detection is both a necessary and challenging task. This survey attempts to link and systematize the latest cutting-edge research on video object detection with the goal of classifying and…

A Survey on Cross-domain Recommendation: Taxonomies, Methods, and Future Directions

Tianzi Zang, Yanmin Zhu, Haobing Liu +2 more

cs.IR · · cited 210× · arXiv ↗ · more like this ·

Traditional recommendation systems are faced with two long-standing obstacles, namely, data sparsity and cold-start problems, which promote the emergence and development of Cross-Domain Recommendation (CDR). The core idea of CDR is to leverage information collected from other domains to alleviate the two problems in one domain. Over the last decade, many efforts have been engaged for cross-domain recommendation. Recently, with the development of deep learning and neural networks, a large number of methods have emerged. However, there is a limited number of systematic surveys on CDR, especially regarding the latest proposed methods as well as the recommendation scenarios and recommendation tasks they address. In this survey paper, we first proposed a two-level taxonomy of cross-domain recommendation which classifies different recommendation scenarios and recommendation tasks. We then…

Leveraging Human Guidance for Deep Reinforcement Learning Tasks

Ruohan Zhang, Faraz Torabi, Lin Guan +2 more

cs.AI · · cited 100× · arXiv ↗ · more like this ·

Reinforcement learning agents can learn to solve sequential decision tasks by interacting with the environment. Human knowledge of how to solve these tasks can be incorporated using imitation learning, where the agent learns to imitate human demonstrated decisions. However, human guidance is not limited to the demonstrations. Other types of guidance could be more suitable for certain tasks and require less human effort. This survey provides a high-level overview of five recent learning frameworks that primarily rely on human guidance other than conventional, step-by-step action demonstrations. We review the motivation, assumption, and implementation of each framework. We then discuss possible future research directions.

Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation

Lin Guan, Mudit Verma, Sihang Guo +2 more

cs.AI · · cited 57× · arXiv ↗ · more like this ·

Human explanation (e.g., in terms of feature importance) has been recently used to extend the communication channel between human and agent in interactive machine learning. Under this setting, human trainers provide not only the ground truth but also some form of explanation. However, this kind of human guidance was only investigated in supervised learning tasks, and it remains unclear how to best incorporate this type of human knowledge into deep reinforcement learning. In this paper, we present the first study of using human visual explanations in human-in-the-loop reinforcement learning (HRL). We focus on the task of learning from feedback, in which the human trainer not only gives binary evaluative "good" or "bad" feedback for queried state-action pairs, but also provides a visual explanation by annotating relevant features in images. We propose EXPAND (EXPlanation AugmeNted…

Deep Learning-Based Object Tracking in Satellite Videos: A comprehensive survey with a new dataset

Yuxuan Li, Licheng Jiao, Zhongjian Huang +9 more

Computer Vision and Pattern Recognition · IEEE Geoscience and Remote Sensing Magazine · · cited 56× · DOI ↗ · more like this ·

As a fundamental task for research in satellite videos (SVs), object tracking is used to track the target of interest in traffic evaluation, military security, and so forth. The current satellite technology in the remote sensing field makes it possible to track moving targets with a relatively high frame rate and image resolution. However, objects under this special view are often small and blurry, making it hard to extract deep features effectively. As a result, quite a few deep learning (DL) methods were proposed for object tracking in SVs. In addition, evaluation criteria for daily life videos (DLVs) are not fully applicable to SVs, which always get low precision evaluation results for tiny objects. In this article, we make three contributions to the research on SVs. First, a new single object tracking (SOT) dataset, named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML"…

Plasma proteomic signatures of social isolation and loneliness associated with morbidity and mortality

Chun Shen, Ruohan Zhang, Jin‐Tai Yu +3 more

Health · Nature Human Behaviour · · cited 51× · DOI ↗ · more like this ·

The biology underlying the connection between social relationships and health is largely unknown. Here, leveraging data from 42,062 participants across 2,920 plasma proteins in the UK Biobank, we characterized the proteomic signatures of social isolation and loneliness through proteome-wide association study and protein co-expression network analysis. Proteins linked to these constructs were implicated in inflammation, antiviral responses and complement systems. More than half of these proteins were prospectively linked to cardiovascular disease, type 2 diabetes, stroke and mortality during a 14 year follow-up. Moreover, Mendelian randomization (MR) analysis suggested causal relationships from loneliness to five proteins, with two proteins (ADM and ASGR1) further supported by colocalization. These MR-identified proteins (GFRA1, ADM, FABP4, TNFRSF10A and ASGR1) exhibited broad…

Interaction Modeling with Multiplex Attention

Fan-Yun Sun, Isaac Kauvar, Ruohan Zhang +4 more

cs.LG · · cited 29× · arXiv ↗ · more like this ·

Modeling multi-agent systems requires understanding how agents interact. Such systems are often difficult to model because they can involve a variety of types of interactions that layer together to drive rich social behavioral dynamics. Here we introduce a method for accurately modeling multi-agent systems. We present Interaction Modeling with Multiplex Attention (IMMA), a forward prediction model that uses a multiplex latent graph to represent multiple independent types of interactions and attention to account for relations of different strengths. We also introduce Progressive Layer Training, a training strategy for this architecture. We show that our approach outperforms state-of-the-art models in trajectory forecasting and relation inference, spanning three multi-agent scenarios: social navigation, cooperative task achievement, and team sports. We further demonstrate that our…

An initial attempt of combining visual selective attention with deep reinforcement learning

Liu Yuezhang, Ruohan Zhang, Dana H. Ballard

cs.LG · · cited 22× · arXiv ↗ · more like this ·

Visual attention serves as a means of feature selection mechanism in the perceptual system. Motivated by Broadbent's leaky filter model of selective attention, we evaluate how such mechanism could be implemented and affect the learning process of deep reinforcement learning. We visualize and analyze the feature maps of DQN on a toy problem Catch, and propose an approach to combine visual selective attention with deep reinforcement learning. We experiment with optical flow-based attention and A2C on Atari games. Experiment results show that visual selective attention could lead to improvements in terms of sample efficiency on tested games. An intriguing relation between attention and batch normalization is also discovered.

Contrastive Multi-view Interest Learning for Cross-domain Sequential Recommendation

Tianzi Zang, Yanmin Zhu, Ruohan Zhang +3 more

Information Systems · ACM Transactions on Information Systems · · cited 21× · DOI ↗ · more like this ·

Cross-domain recommendation (CDR), which leverages information collected from other domains, has been empirically demonstrated to effectively alleviate data sparsity and cold-start problems encountered in traditional recommendation systems. However, current CDR methods, including those considering time information, do not jointly model the general and current interests within and across domains, which is pivotal for accurately predicting users’ future interactions. In this article, we propose a Contrastive learning-enhanced Multi-View interest learning model (CMVCDR) for cross-domain sequential recommendation. Specifically, we design a static view and a sequential view to model uses’ general interests and current interests, respectively. We divide a user’s general interest representation into a domain-invariant part and a domain-specific part. A cross-domain contrastive learning…

Task-Driven Graph Attention for Hierarchical Relational Object Navigation

Michael Lingelbach, Chengshu Li, Minjune Hwang +6 more

cs.AI · · cited 14× · arXiv ↗ · more like this ·

Embodied AI agents in large scenes often need to navigate to find objects. In this work, we study a naturally emerging variant of the object navigation task, hierarchical relational object navigation (HRON), where the goal is to find objects specified by logical predicates organized in a hierarchical structure - objects related to furniture and then to rooms - such as finding an apple on top of a table in the kitchen. Solving such a task requires an efficient representation to reason about object relations and correlate the relations in the environment and in the task goal. HRON in large scenes (e.g. homes) is particularly challenging due to its partial observability and long horizon, which invites solutions that can compactly store the past information while effectively exploring the scene. We demonstrate experimentally that scene graphs are the best-suited representation compared to…

Modeling sensory-motor decisions in natural behavior

Ruohan Zhang, Shun Zhang, Matthew H. Tong +4 more

Artificial Intelligence · PLoS Computational Biology · · cited 14× · DOI ↗ · more like this ·

Although a standard reinforcement learning model can capture many aspects of reward-seeking behaviors, it may not be practical for modeling human natural behaviors because of the richness of dynamic environments and limitations in cognitive resources. We propose a modular reinforcement learning model that addresses these factors. Based on this model, a modular inverse reinforcement learning algorithm is developed to estimate both the rewards and discount factors from human behavioral data, which allows predictions of human navigation behaviors in virtual reality with high accuracy across different subjects and with different tasks. Complex human navigation trajectories in novel environments can be reproduced by an artificial agent that is based on the modular model. This model provides a strategy for estimating the subjective value of actions and how they influence sensory-motor…

A Quantum Evolutionary Learning Tracker for Video

Ruohan Zhang, Licheng Jiao, Lingling Li +3 more

Computer Vision and Pattern Recognition · IEEE Transactions on Evolutionary Computation · · cited 11× · DOI ↗ · more like this ·

Video object tracking has been a popular area in the field of computer vision. As video data evolves, more special perspectives and challenging video data are constantly kept up to date. This poses challenges for object tracking tasks and places higher demands on the generalization capabilities of the models. In this article, we propose a novel quantum evolutionary learning tracker (QELT) for video. The model combines quantum evolution with deep networks for tracking video objects. The model uses a QELT to generate a reliable population of candidate regions and a deep network for classification. In particular, the quantum evolutionary predictor predicts the object motion state through rotation operator and trajectory inference, and provides motion state information for the tracker. The predictor can incorporate object history contextual information and can provide stable candidate…

Evolutionary Dual-Stream Transformer

Ruohan Zhang, Licheng Jiao, Lingling Li +3 more

Computer Vision and Pattern Recognition · IEEE Transactions on Cybernetics · · cited 9× · DOI ↗ · more like this ·

Vision transformers (ViTs) are rapidly evolving and are widely used in computer vision. However, high-performance ViTs require many computations, which limit their further development in the vision field. In this article, a novel evolutionary dual-stream transformer (E-DST) model is proposed to alleviate the computational resource demand problem. A hybrid attention mechanism structure is proposed for a DST model. The DST model uses a dual-branch structure to fuse convolutional and transformer features. Combining the features learned by the transformer and convolution effectively saves model computational resources. In addition, an evolutionary optimizer is proposed to optimize the parameters of the model. The excellent search ability of the evolutionary algorithm is utilized to optimize the transformer model parameters. The convergence of the evolutionary optimizer is proved in this…

Learning Attention Model From Human for Visuomotor Tasks

Luxin Zhang, Ruohan Zhang, Zhuode Liu +2 more

Human-Computer Interaction · Proceedings of the AAAI Conference on Artificial Intelligence · · cited 9× · DOI ↗ · more like this ·

A wealth of information regarding intelligent decision making is conveyed by human gaze and visual attention, hence, modeling and exploiting such information might be a promising way to strengthen algorithms like deep reinforcement learning. We collect high-quality human action and gaze data while playing Atari games. Using these data, we train a deep neural network that can predict human gaze positions and visual attention with high accuracy.

ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation

Wenlong Huang, Chen Wang, Yunzhu Li +2 more

cs.RO · · cited 8× · arXiv ↗ · more like this ·

Representing robotic manipulation tasks as constraints that associate the robot and the environment is a promising way to encode desired robot behaviors. However, it remains unclear how to formulate the constraints such that they are 1) versatile to diverse tasks, 2) free of manual labeling, and 3) optimizable by off-the-shelf solvers to produce robot actions in real-time. In this work, we introduce Relational Keypoint Constraints (ReKep), a visually-grounded representation for constraints in robotic manipulation. Specifically, ReKep is expressed as Python functions mapping a set of 3D keypoints in the environment to a numerical cost. We demonstrate that by representing a manipulation task as a sequence of Relational Keypoint Constraints, we can employ a hierarchical optimization procedure to solve for robot actions (represented by a sequence of end-effector poses in SE(3)) with a…

A Fast Evolutionary Knowledge Transfer Search for Multiscale Deep Neural Architecture

Ruohan Zhang, Licheng Jiao, Dan Wang +3 more

Artificial Intelligence · IEEE Transactions on Neural Networks and Learning Systems · · cited 8× · DOI ↗ · more like this ·

The emergence of neural architecture search (NAS) algorithms has removed the constraints on manually designed neural network architectures, so that neural network development no longer requires extensive professional knowledge, trial and error. However, the extremely high computational cost limits the development of NAS algorithms. In this article, in order to reduce computational costs and to improve the efficiency and effectiveness of evolutionary NAS (ENAS) is investigated. In this article, we present a fast ENAS framework for multiscale convolutional networks based on evolutionary knowledge transfer search (EKTS). This framework is novel, in that it combines global optimization methods with local optimization methods for search, and searches a multiscale network architecture. In this article, evolutionary computation is used as a global optimization algorithm with high robustness…

Primitive Skill-based Robot Learning from Human Evaluative Feedback

Ayano Hiranaka, Minjune Hwang, Sharon Lee +4 more

cs.RO · · cited 7× · arXiv ↗ · more like this ·

Reinforcement learning (RL) algorithms face significant challenges when dealing with long-horizon robot manipulation tasks in real-world environments due to sample inefficiency and safety issues. To overcome these challenges, we propose a novel framework, SEED, which leverages two approaches: reinforcement learning from human feedback (RLHF) and primitive skill-based reinforcement learning. Both approaches are particularly effective in addressing sparse reward issues and the complexities involved in long-horizon tasks. By combining them, SEED reduces the human effort required in RLHF and increases safety in training robot manipulation with RL in real-world settings. Additionally, parameterized skills provide a clear view of the agent's high-level intentions, allowing humans to evaluate skill choices before they are executed. This feature makes the training process even safer and more…

BEHAVIOR Vision Suite: Customizable Dataset Generation via Simulation

Yunhao Ge, Yihe Tang, Jiashu Xu +20 more

cs.CV · · cited 6× · arXiv ↗ · more like this ·

The systematic evaluation and understanding of computer vision models under varying conditions require large amounts of data with comprehensive and customized labels, which real-world vision datasets rarely satisfy. While current synthetic data generators offer a promising alternative, particularly for embodied AI tasks, they often fall short for computer vision tasks due to low asset and rendering quality, limited diversity, and unrealistic physical properties. We introduce the BEHAVIOR Vision Suite (BVS), a set of tools and assets to generate fully customized synthetic data for systematic evaluation of computer vision models, based on the newly developed embodied AI benchmark, BEHAVIOR-1K. BVS supports a large number of adjustable parameters at the scene level (e.g., lighting, object placement), the object level (e.g., joint configuration, attributes such as "filled" and "folded")…

BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation

Chengshu Li, Ruohan Zhang, Josiah Wong +32 more

cs.RO · · cited 6× · arXiv ↗ · more like this ·

We present BEHAVIOR-1K, a comprehensive simulation benchmark for human-centered robotics. BEHAVIOR-1K includes two components, guided and motivated by the results of an extensive survey on "what do you want robots to do for you?". The first is the definition of 1,000 everyday activities, grounded in 50 scenes (houses, gardens, restaurants, offices, etc.) with more than 9,000 objects annotated with rich physical and semantic properties. The second is OMNIGIBSON, a novel simulation environment that supports these activities via realistic physics simulation and rendering of rigid bodies, deformable bodies, and liquids. Our experiments indicate that the activities in BEHAVIOR-1K are long-horizon and dependent on complex manipulation skills, both of which remain a challenge for even state-of-the-art robot learning solutions. To calibrate the simulation-to-reality gap of BEHAVIOR-1K, we…

Attention Guided Imitation Learning and Reinforcement Learning

Ruohan Zhang

Human-Computer Interaction · Proceedings of the AAAI Conference on Artificial Intelligence · · cited 6× · DOI ↗ · more like this ·

&#x0D; &#x0D; &#x0D; We propose a framework that uses learned human visual attention model to guide the learning process of an imitation learning or reinforcement learning agent. We have collected high-quality human action and eye-tracking data while playing Atari games in a carefully controlled experimental setting. We have shown that incorporating a learned human gaze model into deep imitation learning yields promising results.&#x0D; &#x0D; &#x0D;

AGIL: Learning Attention from Human for Visuomotor Tasks

Ruohan Zhang, Zhuode Liu, Luxin Zhang +4 more

cs.CV · · cited 6× · arXiv ↗ · more like this ·

When intelligent agents learn visuomotor behaviors from human demonstrations, they may benefit from knowing where the human is allocating visual attention, which can be inferred from their gaze. A wealth of information regarding intelligent decision making is conveyed by human gaze allocation; hence, exploiting such information has the potential to improve the agents' performance. With this motivation, we propose the AGIL (Attention Guided Imitation Learning) framework. We collect high-quality human action and gaze data while playing Atari games in a carefully controlled experimental setting. Using these data, we first train a deep neural network that can predict human gaze positions and visual attention with high accuracy (the gaze network) and then train another network to predict human actions (the policy network). Incorporating the learned attention model from the gaze network into…

Efficiently Guiding Imitation Learning Agents with Human Gaze

Akanksha Saran, Ruohan Zhang, Elaine Schaertl Short +1 more

cs.LG · · cited 5× · arXiv ↗ · more like this ·

Human gaze is known to be an intention-revealing signal in human demonstrations of tasks. In this work, we use gaze cues from human demonstrators to enhance the performance of agents trained via three popular imitation learning methods -- behavioral cloning (BC), behavioral cloning from observation (BCO), and Trajectory-ranked Reward EXtrapolation (T-REX). Based on similarities between the attention of reinforcement learning agents and human gaze, we propose a novel approach for utilizing gaze data in a computationally efficient manner, as part of an auxiliary loss function, which guides a network to have higher activations in image regions where the human's gaze fixated. This work is a step towards augmenting any existing convolutional imitation learning agent's training with auxiliary gaze data. Our auxiliary coverage-based gaze loss (CGL) guides learning toward a better reward…

Global Policy Construction in Modular Reinforcement Learning

Ruohan Zhang, Zhao Song, Dana H. Ballard

Artificial Intelligence · Proceedings of the AAAI Conference on Artificial Intelligence · · cited 4× · DOI ↗ · more like this ·

We propose a modular reinforcement learning algorithm which decomposes a Markov decision process into independent modules. Each module is trained using Sarsa(lambda). We introduce three algorithms for forming global policy from modules policies, and demonstrate our results using a 2D grid world.