arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Jiaming Sun

50 papers · 1998–2026

1998: 11999: 02000: 02001: 12002: 02003: 02004: 12005: 02006: 12007: 22008: 02009: 02010: 02011: 02012: 12013: 12014: 02015: 02016: 02017: 02018: 22019: 12020: 32021: 42022: 72023: 82024: 82025: 52026: 3
1998peak 8 in 20232026

Frequent co-authors

Papers

showing 1–25 of 50 RSS

LoFTR: Detector-Free Local Feature Matching with Transformers

Jiaming Sun, Zehong Shen, Yuang Wang +2 more

cs.CV · · cited 2,110× · arXiv ↗ · more like this ·

We present a novel method for local image feature matching. Instead of performing image feature detection, description, and matching sequentially, we propose to first establish pixel-wise dense matches at a coarse level and later refine the good matches at a fine level. In contrast to dense methods that use a cost volume to search correspondences, we use self and cross attention layers in Transformer to obtain feature descriptors that are conditioned on both images. The global receptive field provided by Transformer enables our method to produce dense matches in low-texture areas, where feature detectors usually struggle to produce repeatable interest points. The experiments on indoor and outdoor datasets show that LoFTR outperforms state-of-the-art methods by a large margin. LoFTR also ranks first on two public benchmarks of visual localization among the published methods.

NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video

Jiaming Sun, Yiming Xie, Linghao Chen +2 more

cs.CV · · cited 406× · arXiv ↗ · more like this ·

We present a novel framework named NeuralRecon for real-time 3D scene reconstruction from a monocular video. Unlike previous methods that estimate single-view depth maps separately on each key-frame and fuse them later, we propose to directly reconstruct local surfaces represented as sparse TSDF volumes for each video fragment sequentially by a neural network. A learning-based TSDF fusion module based on gated recurrent units is used to guide the network to fuse features from previous fragments. This design allows the network to capture local smoothness prior and global shape prior of 3D surfaces when sequentially reconstructing the surfaces, resulting in accurate, coherent, and real-time surface reconstruction. The experiments on ScanNet and 7-Scenes datasets show that our system outperforms state-of-the-art methods in terms of both accuracy and speed. To the best of our knowledge…

Modeling Indirect Illumination for Inverse Rendering

Yuanqing Zhang, Jiaming Sun, Xingyi He +3 more

cs.CV · · cited 231× · arXiv ↗ · more like this ·

Recent advances in implicit neural representations and differentiable rendering make it possible to simultaneously recover the geometry and materials of an object from multi-view RGB images captured under unknown static illumination. Despite the promising results achieved, indirect illumination is rarely modeled in previous methods, as it requires expensive recursive path tracing which makes the inverse rendering computationally intractable. In this paper, we propose a novel approach to efficiently recovering spatially-varying indirect illumination. The key insight is that indirect illumination can be conveniently derived from the neural radiance field learned from input images instead of being estimated jointly with direct illumination and materials. By properly modeling the indirect illumination and visibility of direct illumination, interreflection- and shadow-free albedo can be…

HCLR-Net: Hybrid Contrastive Learning Regularization with Locally Randomized Perturbation for Underwater Image Enhancement

Jingchun Zhou, Jiaming Sun, Chongyi Li +5 more

Computer Vision and Pattern Recognition · International Journal of Computer Vision · · cited 165× · DOI ↗ · more like this ·

Underwater image enhancement presents a significant challenge due to the complex and diverse underwater environments that result in severe degradation phenomena such as light absorption, scattering, and color distortion. More importantly, obtaining paired training data for these scenarios is a challenging task, which further hinders the generalization performance of enhancement models. To address these issues, we propose a novel approach, the Hybrid Contrastive Learning Regularization (HCLR-Net). Our method is built upon a distinctive hybrid contrastive learning regularization strategy that incorporates a unique methodology for constructing negative samples. This approach enables the network to develop a more robust sample distribution. Notably, we utilize non-paired data for both positive and negative samples, with negative samples are innovatively reconstructed using local patch…

4K4D: Real-Time 4D View Synthesis at 4K Resolution

Zhen Xu, Sida Peng, Haotong Lin +5 more

cs.CV · · cited 135× · arXiv ↗ · more like this ·

This paper targets high-fidelity and real-time view synthesis of dynamic 3D scenes at 4K resolution. Recently, some methods on dynamic view synthesis have shown impressive rendering quality. However, their speed is still limited when rendering high-resolution images. To overcome this problem, we propose 4K4D, a 4D point cloud representation that supports hardware rasterization and enables unprecedented rendering speed. Our representation is built on a 4D feature grid so that the points are naturally regularized and can be robustly optimized. In addition, we design a novel hybrid appearance model that significantly boosts the rendering quality while preserving efficiency. Moreover, we develop a differentiable depth peeling algorithm to effectively learn the proposed model from RGB videos. Experiments show that our representation can be rendered at over 400 FPS on the DNA-Rendering…

Disp R-CNN: Stereo 3D Object Detection via Shape Prior Guided Instance Disparity Estimation

Jiaming Sun, Linghao Chen, Yiming Xie +4 more

cs.CV · · cited 117× · arXiv ↗ · more like this ·

In this paper, we propose a novel system named Disp R-CNN for 3D object detection from stereo images. Many recent works solve this problem by first recovering a point cloud with disparity estimation and then apply a 3D detector. The disparity map is computed for the entire image, which is costly and fails to leverage category-specific prior. In contrast, we design an instance disparity estimation network (iDispNet) that predicts disparity only for pixels on objects of interest and learns a category-specific shape prior for more accurate disparity estimation. To address the challenge from scarcity of disparity annotation in training, we propose to use a statistical shape model to generate dense disparity pseudo-ground-truth without the need of LiDAR point clouds, which makes our system more widely applicable. Experiments on the KITTI dataset show that, even when LiDAR ground-truth is not…

Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimation

Haotong Lin, Sida Peng, Jingxiao Chen +7 more

cs.CV · · cited 109× · arXiv ↗ · more like this ·

Prompts play a critical role in unleashing the power of language and vision foundation models for specific tasks. For the first time, we introduce prompting into depth foundation models, creating a new paradigm for metric depth estimation termed Prompt Depth Anything. Specifically, we use a low-cost LiDAR as the prompt to guide the Depth Anything model for accurate metric depth output, achieving up to 4K resolution. Our approach centers on a concise prompt fusion design that integrates the LiDAR at multiple scales within the depth decoder. To address training challenges posed by limited datasets containin both LiDAR depth and precise GT depth, we propose a scalable data pipeline that includes synthetic data LiDAR simulation and real data pseudo GT depth generation. Our approach sets new state-of-the-arts on the ARKitScenes and ScanNet++ datasets and benefits downstream applications…

Representing Long Volumetric Video with Temporal Gaussian Hierarchy

Zhen Xu, Yinghao Xu, Zhiyuan Yu +4 more

cs.CV · · cited 81× · arXiv ↗ · more like this ·

This paper aims to address the challenge of reconstructing long volumetric videos from multi-view RGB videos. Recent dynamic view synthesis methods leverage powerful 4D representations, like feature grids or point cloud sequences, to achieve high-quality rendering results. However, they are typically limited to short (1~2s) video clips and often suffer from large memory footprints when dealing with longer videos. To solve this issue, we propose a novel 4D representation, named Temporal Gaussian Hierarchy, to compactly model long volumetric videos. Our key observation is that there are generally various degrees of temporal redundancy in dynamic scenes, which consist of areas changing at different speeds. Motivated by this, our approach builds a multi-level hierarchy of 4D Gaussian primitives, where each level separately describes scene regions with different degrees of content change…

Distributed coordinated control scheme of UAV swarm based on heterogeneous roles

Jiang Zhao, Jiaming Sun, Zhihao Cai +2 more

Computer Networks and Communications · Chinese Journal of Aeronautics · · cited 45× · DOI ↗ · more like this ·

This paper proposes a new distributed coordinated control scheme based on heterogeneous roles for Unmanned Aerial Vehicle (UAV) swarm to achieve formation control. First, the framework of the distributed coordinated control scheme is designed on the basis of Distributed Model Predictive Control (DMPC). Then, the effect of heterogeneous roles including leader, coordinator and follower is discussed, and the role-based cost functions are developed to improve the performance of coordinated control for UAV swarm. Furthermore, a group of coordination strategies are proposed for UAVs with different roles to achieve swarm conflict resolution. Numerical simulations demonstrate that the presented distributed coordinated control scheme is effective to formulate and maintain the desired formation for the UAV swarm.

Relightable and Animatable Neural Avatar from Sparse-View Video

Zhen Xu, Sida Peng, Chen Geng +5 more

cs.CV · · cited 44× · arXiv ↗ · more like this ·

This paper tackles the challenge of creating relightable and animatable neural avatars from sparse-view (or even monocular) videos of dynamic humans under unknown illumination. Compared to studio environments, this setting is more practical and accessible but poses an extremely challenging ill-posed problem. Previous neural human reconstruction methods are able to reconstruct animatable avatars from sparse views using deformed Signed Distance Fields (SDF) but cannot recover material parameters for relighting. While differentiable inverse rendering-based methods have succeeded in material recovery of static objects, it is not straightforward to extend them to dynamic humans as it is computationally intensive to compute pixel-surface intersection and light visibility on deformed SDFs for inverse rendering. To solve this challenge, we propose a Hierarchical Distance Query (HDQ) algorithm…

Reconstructing Hand-Held Objects from Monocular Video

Di Huang, Xiaopeng Ji, Xingyi He +5 more

cs.CV · · cited 44× · arXiv ↗ · more like this ·

This paper presents an approach that reconstructs a hand-held object from a monocular video. In contrast to many recent methods that directly predict object geometry by a trained network, the proposed approach does not require any learned prior about the object and is able to recover more accurate and detailed object geometry. The key idea is that the hand motion naturally provides multiple views of the object and the motion can be reliably estimated by a hand pose tracker. Then, the object geometry can be recovered by solving a multi-view reconstruction problem. We devise an implicit neural representation-based method to solve the reconstruction problem and address the issues of imprecise hand pose estimation, relative hand-object motion, and insufficient geometry optimization for small objects. We also provide a newly collected dataset with 3D ground truth to validate the proposed…

Shape Prior Guided Instance Disparity Estimation for 3D Object Detection

Ling-Hao Chen, Jiaming Sun, Yiming Xie +6 more

Computer Vision and Pattern Recognition · IEEE Transactions on Pattern Analysis and Machine Intelligence · · cited 25× · DOI ↗ · more like this ·

In this paper, we propose a novel system named Disp R-CNN for 3D object detection from stereo images. Many recent works solve this problem by first recovering point clouds with disparity estimation and then apply a 3D detector. The disparity map is computed for the entire image, which is costly and fails to leverage category-specific prior. In contrast, we design an instance disparity estimation network (iDispNet) that predicts disparity only for pixels on objects of interest and learns a category-specific shape prior for more accurate disparity estimation. To address the challenge from scarcity of disparity annotation in training, we propose to use a statistical shape model to generate dense disparity pseudo-ground-truth without the need of LiDAR point clouds, which makes our system more widely applicable. Experiments on the KITTI dataset show that, when LiDAR ground-truth is not used…

Growth and Properties of Quaternary Alloy Magnetic Semiconductor (InGaMn)As

Shinobu Ohya, Hiromasa Shimizu, Yutaka Higo +2 more

cond-mat.mtrl-sci · · cited 23× · arXiv ↗ · more like this ·

We have studied growth and properties of quaternary alloy magnetic semiconductor (InGaMn)As grown both on GaAs substrates and on InP substrates by low-temperature molecular-beam epitaxy (LT-MBE). (InGaMn)As thin films were ferromagnetic below ~30 K, exhibiting strong magneto-optical effect. The lattice constant of [(InyGa1-y)1-xMnx]As, whose Mn concentration x is below 4%, is slightly smaller than that of InyGa1-yAs with the same In/Ga content ratio. We have also obtained very smooth surface morphology of nearly lattice matched (InGaMn)As thin films grown on InP substrates, which is important for application to thin-film type magneto-optical devices integrated with III-V opto-electronics.

Deep Reinforcement Learning-Based End-to-End Control for UAV Dynamic Target Tracking

Jiang Zhao, Han Liu, Jiaming Sun +4 more

Computer Vision and Pattern Recognition · Biomimetics · · cited 21× · DOI ↗ · more like this ·

Uncertainty of target motion, limited perception ability of onboard cameras, and constrained control have brought new challenges to unmanned aerial vehicle (UAV) dynamic target tracking control. In virtue of the powerful fitting ability and learning ability of the neural network, this paper proposes a new deep reinforcement learning (DRL)-based end-to-end control method for UAV dynamic target tracking. Firstly, a DRL-based framework using onboard camera image is established, which simplifies the traditional modularization paradigm. Secondly, neural network architecture, reward functions, and soft actor-critic (SAC)-based speed command perception algorithm are designed to train the policy network. The output of the policy network is denormalized and directly used as speed control command, which realizes the UAV dynamic target tracking. Finally, the feasibility of the proposed end-to-end…

Sociological Perspectives on Urban China

Xiangming Chen, Jiaming Sun

Political Science and International Relations · China Information · · cited 21× · DOI ↗ · more like this ·

This article provides an integrated review of sociological perspectives and research on seven areas of inquiry regarding change and continuity in urban China over the past two decades or so. We begin with an assessment of the sociological literature on stratification and inequality in light of the state-market debate and its extensions. Then we evaluate the research on social networks as a resilient resource that can influence social stratification and social change. Next we examine the more interdisciplinary research on migration and migrant labor, highlighting its sociological insights. This is followed by a critical look at housing studies that have revealed a new residential landscape in the Chinese city. Then we present our take on the scholarly contributions to urban consumption, followed by a presentation and appraisal of studies of changing urban governance that have focused on…

Semi-Dense Feature Matching With Transformers and its Applications in Multiple-View Geometry

Zehong Shen, Jiaming Sun, Yuang Wang +3 more

Computer Vision and Pattern Recognition · IEEE Transactions on Pattern Analysis and Machine Intelligence · · cited 18× · DOI ↗ · more like this ·

We present a novel method for local image feature matching. Instead of performing image feature detection, description, and matching sequentially, we propose to first establish pixel-wise dense matches at a coarse level and later refine the good matches at a fine level. In contrast to dense methods that use a cost volume to search correspondences, we use self and cross attention layers in Transformer to obtain feature descriptors that are conditioned on both images. The global receptive field provided by Transformer enables our method to produce dense matches in low-texture areas, where feature detectors usually struggle to produce repeatable interest points. The experiments on indoor and outdoor datasets show that LoFTR outperforms state-of-the-art methods by a large margin. We further adapt LoFTR to modern SfM systems and illustrate its application in multiple-view geometry. The…

Untangling a Global–Local Nexus: Sorting Out Residential Sorting in Shanghai

Xiangming Chen, Jiaming Sun

Urban Studies · Environment and Planning A Economy and Space · · cited 16× · DOI ↗ · more like this ·

The local ‘touchdown’ of globalization gives rise to many complex global–local nexuses, and understanding their nature, structure, and consequences presents a major analytical challenge for globalization research. This paper attempts to untangle one global–local nexus by examining the ‘sorting’ of people into residential or neighborhood spaces in globalizing Shanghai as a function of individual demographic and socioeconomic attributes and by examining personal global connectivity as a key relational variable. We begin with an overview of how local residential differentiation in general and particularly in Shanghai has evolved through the current phase of accelerated globalization and through the city's booming decade of the 1990s. Then, using survey data from the Pudong New Area of Shanghai in 2001, we present a statistical account and analysis of the increasingly varied and layered…

NeuralRecon: Real-Time Coherent 3D Scene Reconstruction From Monocular Video

Xi Chen, Jiaming Sun, Yiming Xie +2 more

Computer Vision and Pattern Recognition · IEEE Transactions on Pattern Analysis and Machine Intelligence · · cited 10× · DOI ↗ · more like this ·

We present a novel framework named NeuralRecon for real-time 3D scene reconstruction from a monocular video. Unlike previous methods that estimate single-view depth maps separately on each key-frame and fuse them later, we propose to directly reconstruct local surfaces represented as sparse TSDF volumes for each video fragment sequentially by a neural network. A learning-based TSDF fusion module based on gated recurrent units is used to guide the network to fuse features from previous fragments. This design allows the network to capture local smoothness prior and global shape prior of 3D surfaces when sequentially reconstructing the surfaces, resulting in accurate, coherent, and real-time surface reconstruction. The fused features can also be used to predict semantic labels, allowing our method to reconstruct and segment the 3D scene simultaneously. Furthermore, we purpose an efficient…

FreeTimeGS: Free Gaussian Primitives at Anytime Anywhere for Dynamic Scene Reconstruction

Yifan Wang, Peishan Yang, Zhen Xu +6 more

Computer Vision and Pattern Recognition · · cited 8× · DOI ↗ · more like this ·

This paper addresses the challenge of reconstructing dynamic 3D scenes with complex motions. Some recent works define 3D Gaussian primitives in the canonical space and use deformation fields to map canonical primitives to observation spaces, achieving real-time dynamic view synthesis. However, these methods often struggle to handle scenes with complex motions due to the difficulty of optimizing deformation fields. To overcome this problem, we propose FreeTimeGS, a novel 4D representation that allows Gaussian primitives to appear at arbitrary time and locations. In contrast to canonical Gaussian primitives, our representation possesses the strong flexibility, thus improving the ability to model dynamic 3D scenes. In addition, we endow each Gaussian primitive with an motion function, allowing it to move to neighboring regions over time, which reduces the temporal redundancy. Experiments…

SMAP: Single-Shot Multi-Person Absolute 3D Pose Estimation

Jianan Zhen, Qi Fang, Jiaming Sun +4 more

cs.CV · · cited 5× · arXiv ↗ · more like this ·

Recovering multi-person 3D poses with absolute scales from a single RGB image is a challenging problem due to the inherent depth and scale ambiguity from a single view. Addressing this ambiguity requires to aggregate various cues over the entire image, such as body sizes, scene layouts, and inter-person relationships. However, most previous methods adopt a top-down scheme that first performs 2D pose detection and then regresses the 3D pose and scale for each detected person individually, ignoring global contextual cues. In this paper, we propose a novel system that first regresses a set of 2.5D representations of body parts and then reconstructs the 3D absolute poses based on these 2.5D representations with a depth-aware part association algorithm. Such a single-shot bottom-up scheme allows the system to better learn and reason about the inter-person depth relationship, improving both…

Personal Global Connectivity and Consumer Behavior

Jiaming Sun, Xun Wang

Sociology and Political Science · Journal of International Consumer Marketing · · cited 5× · DOI ↗ · more like this ·

Abstract This paper aims to examine the relationship between global connectivity and consumer behavior in the city of Shanghai, China. Using Structural Equation Modeling analysis, we found that personal global connectivity has the strongest direct effect and significant indirect effect on global consumption by mediating other exogenous variables such as age, educational attainment, and income. The direct effects of education and income disappear when global connectivity is controlled, while the negative effect of age remains.

EasyVolcap: Accelerating Neural Volumetric Video Research

Zhen Xu, Tao Xie, Sida Peng +7 more

cs.CV · · cited 3× · arXiv ↗ · more like this ·

Volumetric video is a technology that digitally records dynamic events such as artistic performances, sporting events, and remote conversations. When acquired, such volumography can be viewed from any viewpoint and timestamp on flat screens, 3D displays, or VR headsets, enabling immersive viewing experiences and more flexible content creation in a variety of applications such as sports broadcasting, video conferencing, gaming, and movie productions. With the recent advances and fast-growing interest in neural scene representations for volumetric video, there is an urgent need for a unified open-source library to streamline the process of volumetric video capturing, reconstruction, and rendering for both researchers and non-professional users to develop various algorithms and applications of this emerging technology. In this paper, we present EasyVolcap, a Python & Pytorch library for…

An Improved Template Representation-based Transformer for Abstractive Text Summarization

Jiaming Sun, Yunli Wang, Zhoujun Li

Artificial Intelligence · · cited 3× · DOI ↗ · more like this ·

Text summarization plays an important role in various NLP applications. Using templates with generation methods is an effective way to address abstractive summarization. However, existing template-enhanced generation approaches use templates in a naive way and mainly adopt RNN-based Seq2Seq models, so they cannot make full use of valid information in the templates and suffer from templates' noise. To mitigate these problems, we propose a new abstractive summarization model called Summarization Transformer with Template-aware Representation (STTR), which uses a template-aware document encoding module and a document representation shifting loss to preserve the useful information and filter the noise of the template. The experiments on the Gigaword and LCSTS datasets show that our method outperforms baseline models and achieves a new state-of-the-art.

Displacement measurement with CCD Moiré method considering the relative rotation of gratings

Zhihao Shangguan, Chen Gui, Jiaming Sun +3 more

Computer Vision and Pattern Recognition · Optics Express · · cited 2× · DOI ↗ · more like this ·

Utilizing the periodic pixel configuration of the CCD/CMOS imaging sensor as the reference grating and the image of the real grating formed by the lens as the specimen grating, the CCD Moiré method directly outputs amplified Moiré fringes, facilitating high-resolution, full-field deformation measurement. Due to the amplification effect of CCD Moiré, even minor relative rotations of the two gratings can significantly affect Moiré imaging and introduce measurable errors. This paper mainly addresses the imaging and measurement challenges of in-plane rotation in the CCD Moiré method. Firstly, through theoretical analysis, simulations, and experimental studies, the imaging characteristics of CCD Moiré under various rotation conditions were systematically summarized and an optimal rotation angle range for accurate measurements was proposed. Secondly, the correlation between the deformation…

EmoEcho: Designing Emotion Mimicry Mechanics for Enhancing Social Engagement in Digital Games

Jiaming Sun, Zhuying Li, Xiaolan Peng

Human-Computer Interaction · · cited 2× · DOI ↗ · more like this ·

Social interaction plays a critical role in enhancing player experiences in bodily play. Although previous research has extensively explored physical mimicry in movement-based games for social engagement, the incorporation of emotional expressions, a significant aspect of bodily interaction, remains underexplored. This study aims to broaden the understanding of bodily play by integrating emotion mimicry, examining its impact on enhancing social connections. We present “EmoEcho”, a two-player 2D side-scrolling game that embeds emotion mimicry into its core gameplay. EmoEcho allows players to dynamically alter the game environment using facial expressions, fostering a unique shared interactive experience. A pilot study compares emotion-based and keyboard inputs, revealing that emotion input significantly enhances the social dynamics of gameplay. This paper contributes to digital play by…