arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Zhenjun Yu

16 papers · 2022–2026

2022: 12023: 42024: 42025: 32026: 4
2022peak 4 in 20232026

Frequent co-authors

Papers

showing 1–16 of 16 RSS

DexTOG: Learning Task-Oriented Dexterous Grasp with Language

Jieyi Zhang, Wenqiang Xu, Zhenjun Yu +3 more

cs.RO · · cited 2× · arXiv ↗ · more like this ·

This study introduces a novel language-guided diffusion-based learning framework, DexTOG, aimed at advancing the field of task-oriented grasping (TOG) with dexterous hands. Unlike existing methods that mainly focus on 2-finger grippers, this research addresses the complexities of dexterous manipulation, where the system must identify non-unique optimal grasp poses under specific task constraints, cater to multiple valid grasps, and search in a high degree-of-freedom configuration space in grasp planning. The proposed DexTOG includes a diffusion-based grasp pose generation model, DexDiffu, and a data engine to support the DexDiffu. By leveraging DexTOG, we also proposed a new dataset, DexTOG-80K, which was developed using a shadow robot hand to perform various tasks on 80 objects from 5 categories, showcasing the dexterity and multi-tasking capabilities of the robotic hand. This research…

Structure-aware generalization for heterogeneous histopathology via prototype-based multiple instance learning

Zhenjun Yu, Zhelin Xia, Donghao Xu +7 more

Artificial Intelligence · npj Digital Medicine · · cited 1× · DOI ↗ · more like this ·

Accurate and generalizable cancer diagnosis from whole slide images (WSIs) remains challenging due to limited fine-grained annotations, complex tumor architectures, and domain shifts across scanners and institutions1. We introduce StructMIL, a structure-aware and prototype-driven multiple instance learning framework designed for robust and interpretable cancer detection and grading2. StructMIL integrates graph-based topological priors with histological context, employs prototype-enhanced pooling for stable and transparent predictions, and incorporates a unified domain-generalization strategy that combines contrastive alignment, adversarial confusion, and consistency regularization. Evaluated on Camelyon16 for breast cancer metastasis detection and PANDA for prostate cancer Gleason grading, StructMIL achieves state-of-the-art performance. On Camelyon16, StructMIL improves cross-center…

KDSeg: Segmentation Guided knowledge distillation for Anomaly Detection

Zhenjun Yu, Chunbo Zhang, Lin Sun

Artificial Intelligence · · cited 1× · DOI ↗ · more like this ·

Anomaly detection (AD) are crucial technologies for product quality inspection. Due to the powerful feature extraction capabilities of convolutional neural networks (CNN) which have been widely applied in industrial product quality inspection. However, the high cost of sample annotation and the scarcity of anomalous samples have increased the difficulty of this task. therefore, unsupervised anomaly detection are attracting widespread attention. Nevertheless, the current capabilities of unsupervised anomaly detection are insufficient, and the accuracy in segmenting anomalous regions is imprecise. To address the aforementioned issues, we propose the KDSeg method which integrates Teacher-Student framework and a segmentation network into one model and can be trained from only normal images. The approach makes full use of multi-scale information to improve the capabilities of student network…

MS-MANO: Enabling Hand Pose Tracking with Biomechanical Constraints

Pengfei Xie, Wenqiang Xu, Tutian Tang +2 more

cs.CV · · cited 1× · arXiv ↗ · more like this ·

This work proposes a novel learning framework for visual hand dynamics analysis that takes into account the physiological aspects of hand motion. The existing models, which are simplified joint-actuated systems, often produce unnatural motions. To address this, we integrate a musculoskeletal system with a learnable parametric hand model, MANO, to create a new model, MS-MANO. This model emulates the dynamics of muscles and tendons to drive the skeletal system, imposing physiologically realistic constraints on the resulting torque trajectories. We further propose a simulation-in-the-loop pose refinement framework, BioPR, that refines the initial estimated pose through a multi-layer perceptron (MLP) network. Our evaluation of the accuracy of MS-MANO and the efficacy of the BioPR is conducted in two separate parts. The accuracy of MS-MANO is compared with MyoSuite, while the efficacy of…

Abnormal Discrepancy-Guided Knowledge Distillation for Image Anomaly Detection

Zhenjun Yu, Lin Sun, Kai Wang +1 more

Artificial Intelligence · Journal of Imaging · · cited 0× · DOI ↗ · more like this ·

Knowledge distillation is a cornerstone of image anomaly detection for amplifying subtle defects via teacher-student discrepancy, yet existing methods rely on feature alignment loss that causes reconstruction error confusion and degrades accuracy. To address this critical limitation, this study proposes an abnormal discrepancy-guided knowledge distillation method (DiffKD) that differentially guides student feature reconstruction through channel-level discrepancy masks, leveraging normal features as supervisory signals and abnormal discrepancy features as constraints to enhance anomaly detection performance. The approach integrates a knowledge distillation network for feature reconstruction with a segmentation network for anomaly localization, while utilizing prior anomaly samples and synthetic anomaly samples to provide real-time training data of anomalous samples. Extensive evaluations…

Image Anomaly Detection via Knowledge Distillation With Cross‐Anomaly Sample Augmentation

Zhenjun Yu, Fengxiang Jin, Lin Sun +2 more

Artificial Intelligence · Concurrency and Computation Practice and Experience · · cited 0× · DOI ↗ · more like this ·

ABSTRACT Current image anomaly detection techniques mainly focus on using normal samples throughout the training phase to minimize the model's reliance on anomaly samples, overlooking the valuable information contained in anomalous samples that are often available in real‐world scenarios. However, lacking guidance from real anomaly samples, the model learns a decision boundary based exclusively on normal data, which inevitably results in an ambiguous boundary and poor generalization, rendering the model vulnerable to complex background interference and prone to high false alarm rates. To address this limitation, we propose a Few‐Shot Prior anomaly driven (FSPD) method that leverages a blend of prior anomalous samples and synthetic anomalies based on normal samples to enhance model training. Our approach integrates a knowledge distillation network for accurate feature reconstruction and…

Dynamic Reconstruction of Hand-Object Interaction with Distributed Force-aware Contact Representation

Zhenjun Yu, Wenqiang Xu, Pengfei Xie +4 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

We present ViTaM-D, a novel visual-tactile framework for reconstructing dynamic hand-object interaction with distributed tactile sensing to enhance contact modeling. Existing methods, relying solely on visual inputs, often fail to capture occluded interactions and object deformation. To address this, we introduce DF-Field, a distributed force-aware contact representation leveraging kinetic and potential energy in hand-object interactions. ViTaM-D first reconstructs interactions using a visual network with contact constraint, then refines contact details through force-aware optimization, improving object deformation modeling. To evaluate deformable object reconstruction, we introduce the HOT dataset, featuring 600 hand-object interaction sequences in a high-precision simulation environment. Experiments on DexYCB and HOT datasets show that ViTaM-D outperforms state-of-the-art methods in…

FlowDSnet: An End-to-End Scene Flow Based LiDAR De-Skewing Network

Xinrui Wu, Yixiang Zhu, Siyi Wang +3 more

Computer Vision and Pattern Recognition · · cited 0× · DOI ↗ · more like this ·

The motion distortion of LiDAR is the phenomenon caused by the movement of the LiDAR carrier. Such phenomenon makes the LiDAR point cloud information unreliable and seriously degrades the tasks that rely on LiDAR such as 3D object perception, localization, and mapping. In this work, a novel end-to-end network named FlowDSnet is proposed to compensate for the distortion of LiDAR point clouds in an end-to-end way. The proposed pipeline takes the original adjacent two frames of point clouds as input and recovers the undistorted coordinates of each point by predicting the scene flow in 3D space. Different from the traditional distortion removal algorithm, our method does not rely on the information of other sensors, such as the Inertial measurement unit (IMU), but applies a novel flow embedding algorithm combined with spatiotemporal interpolation to conduct distortion correction. And the…

FreeFly-Thinking : Aligning Chain-of-Thought Reasoning with Continuous UAV Navigation

Jiaxu Zhou, Shaobo Wang, Zhiyuan Yang +2 more

cs.CV · · arXiv ↗ · more like this ·

Vision-Language Navigation aims to enable agents to understand natural language instructions and carry out appropriate navigation actions in real-world environments. Most work focuses on indoor settings, with little research in complex outdoor scenes. Current UAV Vision-and-Language Navigation models typically act as black boxes without explicit reasoning. We introduce FreeFly-thinking, an end-to-end VLN framework that converts the UAV agent's egocentric images and language instructions into a series of actions, inspired by environment of urban architecture proposed by OpenFly. We first construct a UAV dataset for navigation task, and then performing natural language chain of thought. We adopt a two-stage training strategy: Supervised fine-tuning and Reinforcement fine-tuning. Experiments on unseen test demonstrate a strong performance, presenting robustness and efficiency in UAV…

FBI: Learning Dexterous In-hand Manipulation with Dynamic Visuotactile Shortcut Policy

Yijin Chen, Wenqiang Xu, Zhenjun Yu +4 more

cs.RO · · arXiv ↗ · more like this ·

Dexterous in-hand manipulation is a long-standing challenge in robotics due to complex contact dynamics and partial observability. While humans synergize vision and touch for such tasks, robotic approaches often prioritize one modality, therefore limiting adaptability. This paper introduces Flow Before Imitation (FBI), a visuotactile imitation learning framework that dynamically fuses tactile interactions with visual observations through motion dynamics. Unlike prior static fusion methods, FBI establishes a causal link between tactile signals and object motion via a dynamics-aware latent model. FBI employs a transformer-based interaction module to fuse flow-derived tactile features with visual inputs, training a one-step diffusion policy for real-time execution. Extensive experiments demonstrate that the proposed method outperforms the baseline methods in both simulation and the real…

AgentWorld: An Interactive Simulation Platform for Scene Construction and Mobile Robotic Manipulation

Yizheng Zhang, Zhenjun Yu, Jiaxin Lai +2 more

cs.RO · · arXiv ↗ · more like this ·

We introduce AgentWorld, an interactive simulation platform for developing household mobile manipulation capabilities. Our platform combines automated scene construction that encompasses layout generation, semantic asset placement, visual material configuration, and physics simulation, with a dual-mode teleoperation system supporting both wheeled bases and humanoid locomotion policies for data collection. The resulting AgentWorld Dataset captures diverse tasks ranging from primitive actions (pick-and-place, push-pull, etc.) to multistage activities (serve drinks, heat up food, etc.) across living rooms, bedrooms, and kitchens. Through extensive benchmarking of imitation learning methods including behavior cloning, action chunking transformers, diffusion policies, and vision-language-action models, we demonstrate the dataset's effectiveness for sim-to-real transfer. The integrated system…

DiPGrasp: Parallel Local Searching for Efficient Differentiable Grasp Planning

Wenqiang Xu, Jieyi Zhang, Tutian Tang +3 more

cs.RO · · arXiv ↗ · more like this ·

Grasp planning is an important task for robotic manipulation. Though it is a richly studied area, a standalone, fast, and differentiable grasp planner that can work with robot grippers of different DOFs has not been reported. In this work, we present DiPGrasp, a grasp planner that satisfies all these goals. DiPGrasp takes a force-closure geometric surface matching grasp quality metric. It adopts a gradient-based optimization scheme on the metric, which also considers parallel sampling and collision handling. This not only drastically accelerates the grasp search process over the object surface but also makes it differentiable. We apply DiPGrasp to three applications, namely grasp dataset construction, mask-conditioned planning, and pose refinement. For dataset generation, as a standalone planner, DiPGrasp has clear advantages over speed and quality compared with several classic…

TacIPC: Intersection- and Inversion-free FEM-based Elastomer Simulation For Optical Tactile Sensors

Wenxin Du, Wenqiang Xu, Jieji Ren +2 more

cs.RO · · arXiv ↗ · more like this ·

Tactile perception stands as a critical sensory modality for human interaction with the environment. Among various tactile sensor techniques, optical sensor-based approaches have gained traction, notably for producing high-resolution tactile images. This work explores gel elastomer deformation simulation through a physics-based approach. While previous works in this direction usually adopt the explicit material point method (MPM), which has certain limitations in force simulation and rendering, we adopt the finite element method (FEM) and address the challenges in penetration and mesh distortion with incremental potential contact (IPC) method. As a result, we present a simulator named TacIPC, which can ensure numerically stable simulations while accommodating direct rendering and friction modeling. To evaluate TacIPC, we conduct three tasks: pseudo-image quality assessment, deformed…

Precise Robotic Needle-Threading with Tactile Perception and Reinforcement Learning

Zhenjun Yu, Wenqiang Xu, Siqiong Yao +5 more

cs.RO · · arXiv ↗ · more like this ·

This work presents a novel tactile perception-based method, named T-NT, for performing the needle-threading task, an application of deformable linear object (DLO) manipulation. This task is divided into two main stages: Tail-end Finding and Tail-end Insertion. In the first stage, the agent traces the contour of the thread twice using vision-based tactile sensors mounted on the gripper fingers. The two-run tracing is to locate the tail-end of the thread. In the second stage, it employs a tactile-guided reinforcement learning (RL) model to drive the robot to insert the thread into the target needle eyelet. The RL model is trained in a Unity-based simulated environment. The simulation environment supports tactile rendering which can produce realistic tactile images and thread modeling. During insertion, the position of the poke point and the center of the eyelet are obtained through a…

Visual-Tactile Sensing for In-Hand Object Reconstruction

Wenqiang Xu, Zhenjun Yu, Han Xue +3 more

cs.CV · · arXiv ↗ · more like this ·

Tactile sensing is one of the modalities humans rely on heavily to perceive the world. Working with vision, this modality refines local geometry structure, measures deformation at the contact area, and indicates the hand-object contact state. With the availability of open-source tactile sensors such as DIGIT, research on visual-tactile learning is becoming more accessible and reproducible. Leveraging this tactile sensor, we propose a novel visual-tactile in-hand object reconstruction framework \textbf{VTacO}, and extend it to \textbf{VTacOH} for hand-object reconstruction. Since our method can support both rigid and deformable object reconstruction, no existing benchmarks are proper for the goal. We propose a simulation environment, VT-Sim, which supports generating hand-object interaction for both rigid and deformable objects. With VT-Sim, we generate a large-scale training dataset and…

RFUniverse: A Multiphysics Simulation Platform for Embodied AI

Haoyuan Fu, Wenqiang Xu, Ruolin Ye +7 more

cs.RO · · arXiv ↗ · more like this ·

Multiphysics phenomena, the coupling effects involving different aspects of physics laws, are pervasive in the real world and can often be encountered when performing everyday household tasks. Intelligent agents which seek to assist or replace human laborers will need to learn to cope with such phenomena in household task settings. To equip the agents with such kind of abilities, the research community needs a simulation environment, which will have the capability to serve as the testbed for the training process of these intelligent agents, to have the ability to support multiphysics coupling effects. Though many mature simulation software for multiphysics simulation have been adopted in industrial production, such techniques have not been applied to robot learning or embodied AI research. To bridge the gap, we propose a novel simulation environment named RFUniverse. This simulator can…