arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Tianshu Yu

90 papers · 2003–2026

2003: 12004: 02005: 02006: 02007: 02008: 02009: 02010: 12011: 02012: 12013: 02014: 02015: 12016: 22017: 12018: 42019: 02020: 52021: 32022: 12023: 142024: 152025: 202026: 21
2003peak 21 in 20262026

Frequent co-authors

Papers

showing 1–25 of 90 RSS

Learning Language-guided Adaptive Hyper-modality Representation for Multimodal Sentiment Analysis

Haoyu Zhang, Yu Wang, Guanghao Yin +3 more

cs.AI · · cited 221× · arXiv ↗ · more like this ·

Though Multimodal Sentiment Analysis (MSA) proves effective by utilizing rich information from multiple sources (e.g., language, video, and audio), the potential sentiment-irrelevant and conflicting information across modalities may hinder the performance from being further improved. To alleviate this, we present Adaptive Language-guided Multimodal Transformer (ALMT), which incorporates an Adaptive Hyper-modality Learning (AHL) module to learn an irrelevance/conflict-suppressing representation from visual and audio features under the guidance of language features at different scales. With the obtained hyper-modality representation, the model can obtain a complementary and joint representation through multimodal fusion for effective MSA. In practice, ALMT achieves state-of-the-art performance on several popular datasets (e.g., MOSI, MOSEI and CH-SIMS) and an abundance of ablation…

Combinatorial Learning of Graph Edit Distance via Dynamic Embedding

Runzhong Wang, Tianqi Zhang, Tianshu Yu +2 more

cs.LG · · cited 69× · arXiv ↗ · more like this ·

Graph Edit Distance (GED) is a popular similarity measurement for pairwise graphs and it also refers to the recovery of the edit path from the source graph to the target graph. Traditional A* algorithm suffers scalability issues due to its exhaustive nature, whose search heuristics heavily rely on human prior knowledge. This paper presents a hybrid approach by combing the interpretability of traditional search-based techniques for producing the edit path, as well as the efficiency and adaptivity of deep embedding models to achieve a cost-effective GED solver. Inspired by dynamic programming, node-level embedding is designated in a dynamic reuse fashion and suboptimal branches are encouraged to be pruned. To this end, our method can be readily integrated into A* procedure in a dynamic fashion, as well as significantly reduce the computational burden with a learned heuristic. Experimental…

Frequency Representation Integration for Camouflaged Object Detection

C. Xie, Changqun Xia, Tianshu Yu +1 more

Computer Vision and Pattern Recognition · · cited 51× · DOI ↗ · more like this ·

Recent camouflaged object detection (COD) approaches have been proposed to accurately segment objects blended into surroundings. The most challenging and critical issue in COD is to find out the lines of demarcation between objects and background in the camouflage environment. Because of the similarity between the target object and the background, these lines are difficult to be found accurately. However, these are easy to be observed in different frequency components of the image. To this end, in this paper we rethink COD from the perspective of frequency components and propose a Frequency Representation Integration Network to mine informative cues from them. Specifically, we obtain high-frequency components from the original image by Laplacian pyramid-like decomposition, and then respectively send the image to a transformer-based encoder and frequency components to a tailored…

ASP: Learn a Universal Neural Solver!

Chenguang Wang, Zhouliang Yu, Stephen McAleer +2 more

cs.LG · · cited 47× · arXiv ↗ · more like this ·

Applying machine learning to combinatorial optimization problems has the potential to improve both efficiency and accuracy. However, existing learning-based solvers often struggle with generalization when faced with changes in problem distributions and scales. In this paper, we propose a new approach called ASP: Adaptive Staircase Policy Space Response Oracle to address these generalization issues and learn a universal neural solver. ASP consists of two components: Distributional Exploration, which enhances the solver's ability to handle unknown distributions using Policy Space Response Oracles, and Persistent Scale Adaption, which improves scalability through curriculum learning. We have tested ASP on several challenging COPs, including the traveling salesman problem, the vehicle routing problem, and the prize collecting TSP, as well as the real-world instances from TSPLib and CVRPLib.…

Learning deep graph matching with channel-independent embedding and Hungarian attention

Tianshu Yu, Runzhong Wang, Junchi Yan +1 more

Artificial Intelligence · International Conference on Learning Representations · · cited 46× · OpenAlex ↗ · more like this ·

Graph matching aims to establishing node-wise correspondence between two graphs, which is a classic combinatorial problem and in general NP-complete. Until very recently, deep graph matching methods start to resort to deep networks to achieve unprecedented matching accuracy. Along this direction, this paper makes two complementary contributions which can also be reused as plugin in existing works: i) a novel node and edge embedding strategy which stimulates the multi-head strategy in attention models and allows the information in each channel to be merged independently. In contrast, only node embedding is accounted in previous works; ii) a general masking mechanism over the loss function is devised to improve the smoothness of objective learning for graph matching. Using Hungarian algorithm, it dynamically constructs a structured and sparsely connected layer, taking into account the…

Generalizing graph matching beyond quadratic assignment model

Tianshu Yu, Junchi Yan, Yilin Wang +2 more

Computer Vision and Pattern Recognition · neural information processing systems · · cited 29× · OpenAlex ↗ · more like this ·

Graph matching has received persistent attention over decades, which can be formulated as a quadratic assignment problem (QAP). We show that a large family of functions, which we define as Separable Functions, can approximate discrete graph matching in the continuous domain asymptotically by varying the approximation controlling parameters. We also study the properties of global optimality and devise convex/concave-preserving extensions to the widely used Lawler's QAP form. Our theoretical findings show the potential for deriving new algorithms and techniques for graph matching. We deliver solvers based on two specific instances of Separable Functions, and the state-of-the-art performance of our method is verified on popular benchmarks.

View-aware Salient Object Detection for 360° Omnidirectional Image

Junjie Wu, Changqun Xia, Tianshu Yu +1 more

cs.CV · · cited 20× · arXiv ↗ · more like this ·

Image-based salient object detection (ISOD) in 360° scenarios is significant for understanding and applying panoramic information. However, research on 360° ISOD has not been widely explored due to the lack of large, complex, high-resolution, and well-labeled datasets. Towards this end, we construct a large scale 360° ISOD dataset with object-level pixel-wise annotation on equirectangular projection (ERP), which contains rich panoramic scenes with not less than 2K resolution and is the largest dataset for 360° ISOD by far to our best knowledge. By observing the data, we find current methods face three significant challenges in panoramic scenarios: diverse distortion degrees, discontinuous edge effects and changeable object scales. Inspired by humans' observing process, we propose a view-aware salient object detection method based on a Sample Adaptive View Transformer (SAVT) module with…

Face recognition based on subset selection via metric learning on manifold

Hong Shao, Shuang Chen, Jieyi Zhao +2 more

Computer Vision and Pattern Recognition · Frontiers of Information Technology & Electronic Engineering · · cited 20× · DOI ↗ · more like this ·

With the development of face recognition using sparse representation based classification (SRC), many relevant methods have been proposed and investigated. However, when the dictionary is large and the representation is sparse, only a small proportion of the elements contributes to the l 1-minimization. Under this observation, several approaches have been developed to carry out an efficient element selection procedure before SRC. In this paper, we employ a metric learning approach which helps find the active elements correctly by taking into account the interclass/intraclass relationship and manifold structure of face images. After the metric has been learned, a neighborhood graph is constructed in the projected space. A fast marching algorithm is used to rapidly select the subset from the graph, and SRC is implemented for classification. Experimental results show that our method…

Joint Cuts and Matching of Partitions in One Graph

Tianshu Yu, Junchi Yan, Jieyi Zhao +1 more

cs.CV · · cited 13× · arXiv ↗ · more like this ·

As two fundamental problems, graph cuts and graph matching have been investigated over decades, resulting in vast literature in these two topics respectively. However the way of jointly applying and solving graph cuts and matching receives few attention. In this paper, we first formalize the problem of simultaneously cutting a graph into two partitions i.e. graph cuts and establishing their correspondence i.e. graph matching. Then we develop an optimization algorithm by updating matching and cutting alternatively, provided with theoretical analysis. The efficacy of our algorithm is verified on both synthetic dataset and real-world images containing similar regions or structures.

Hessian based image structure adaptive gradient vector flow for parametric active contours

Yuanquan Wang, Wanyi Chen, Tianshu Yu +1 more

Computer Vision and Pattern Recognition · · cited 8× · DOI ↗ · more like this ·

Active contours have been one of the most successful methods for image segmentation during the last two decades, but one of the shortcomings of being unable to converge to concavity is a handicap to its effectiveness. In order to address this issue, the gradient vector flow (GVF) was put forth. Although there have been a great number of works on GVF, the image structure has seldom been incorporated into GVF algorithm. In this work, the image structure characterized by the Hessian matrix is incorporated into the GVF algorithm by reformulating the smoothness constraint of GVF into matrix form. In this way, the associated diffusion PDEs are anisotropic and the modified GVF snake can converge to very long concavity and preserve weak edge simultaneously. Experiments and comparisons are presented to demonstrate the properties of the proposed strategies.

Regional Features and Spatial Distribution of Fifty-Eight Ethnic Groups in Southwest China

Yuan Shen, Danyin Wang, Jiahui Wu +3 more

Sociology and Political Science · Sustainability · · cited 7× · DOI ↗ · more like this ·

Eco-environmental variability was the basis of ethnic diversity with a persistent influence on the regional development. The unique geographic conditions and multi-ethnic characteristics in southwest China were valuable for exploring sustainable development of ethnic regions. In this study, the regional features of distribution areas of ethnic groups in southwest China were analysed, and it was found that average altitude, slope, water coverage and water form ratio of each ethnic group differed apparently. Additionally, regional features of southern minorities were relatively stable, while those of northern minorities changed evidently from 1990 to 2010. Furthermore, taking the spatial coordinates and regional features as parameters, fifty-eight ethnic groups were clustered into six categories via spatial cluster analysis. Moreover, based on the county-level population data, the ethnic…

Determinant Regularization for Gradient-Efficient Graph Matching

Tianshu Yu, Junchi Yan, Baoxin Li

Computer Vision and Pattern Recognition · · cited 7× · DOI ↗ · more like this ·

Graph matching refers to finding vertex correspondence for a pair of graphs, which plays a fundamental role in many vision and learning related tasks. Directly applying gradient-based continuous optimization on graph matching can be attractive for its simplicity but calls for effective ways of converting the continuous solution to the discrete one under the matching constraint. In this paper, we show a novel regularization technique with the tool of determinant analysis on the matching matrix which is relaxed into continuous domain with gradient based optimization. Meanwhile we present a theoretical study on the property of our relaxation technique. Our paper strikes an attempt to understand the geometric properties of different regularization techniques and the gradient behavior during the optimization. We show that the proposed regularization is more gradient-efficient than…

Weakly Supervised Attention Learning for Textual Phrases Grounding

Zhiyuan Fang, Shu Kong, Tianshu Yu +1 more

cs.CV · · cited 7× · arXiv ↗ · more like this ·

Grounding textual phrases in visual content is a meaningful yet challenging problem with various potential applications such as image-text inference or text-driven multimedia interaction. Most of the current existing methods adopt the supervised learning mechanism which requires ground-truth at pixel level during training. However, fine-grained level ground-truth annotation is quite time-consuming and severely narrows the scope for more general applications. In this extended abstract, we explore methods to localize flexibly image regions from the top-down signal (in a form of one-hot label or natural languages) with a weakly supervised attention learning mechanism. In our model, two types of modules are utilized: a backbone module for visual feature capturing, and an attentive module generating maps based on regularized bilinear pooling. We construct the model in an end-to-end fashion…

Towards Robust Multimodal Sentiment Analysis with Incomplete Data

Haoyu Zhang, Wenbin Wang, Tianshu Yu

cs.CL · · cited 4× · arXiv ↗ · more like this ·

The field of Multimodal Sentiment Analysis (MSA) has recently witnessed an emerging direction seeking to tackle the issue of data incompleteness. Recognizing that the language modality typically contains dense sentiment information, we consider it as the dominant modality and present an innovative Language-dominated Noise-resistant Learning Network (LNLN) to achieve robust MSA. The proposed LNLN features a dominant modality correction (DMC) module and dominant modality based multimodal learning (DMML) module, which enhances the model's robustness across various noise scenarios by ensuring the quality of dominant modality representations. Aside from the methodical design, we perform comprehensive experiments under random data missing scenarios, utilizing diverse and meaningful settings on several popular datasets (\textit{e.g.,} MOSI, MOSEI, and SIMS), providing additional uniformity…

Graph matching with low-rank regularization

Tianshu Yu, Ruisheng Wang

Computer Vision and Pattern Recognition · · cited 4× · DOI ↗ · more like this ·

Graph matching is a widely researched topic which has been utilized in various applications of computer vision. Due to the combinatorial nature of graph matching, it is NP-hard to find an exact solution. So exact graph matching is always relaxed to inexact graph matching which seeks to find an approximate solution for the original problem. For a matching problem in quadratic form, semidefinite programming (SDP) relaxation is proven to be effective. However, previous SDP relaxation methods discard the constraint that the solution matrix is rank one, because the rank of a matrix is non-convex. In this paper, we explore some good properties of the solution matrix. By relaxing the rank into convex form using the properties, we propose to reformulate the graph matching with low rank constraint into a standard SDP, which can be easily solved. We test our method on both synthetic and real…

MolEmb: Multimodal Large Language Models Can Be Strong Molecular Embedding Models

Xinjian Zhao, Xiangru Jian, Yaoyao Xu +4 more

cs.AI · · cited 3× · arXiv ↗ · more like this ·

Molecular embedding models can serve as foundational infrastructure for computational chemistry and drug discovery, where reusable vector representations support property prediction, virtual screening, and retrieval. Most molecular encoders are specialist models built around a single molecular view, producing unconditional vectors with no language interface for varying the representation. We ask whether multimodal large language models (MLLMs), which natively process images, text, and symbolic inputs, can instead serve as \emph{general molecular embedding models} that produce embeddings conditioned on both a molecular profile and a natural-language semantic context. We introduce \textbf{MolEmb}, a lightweight framework that adapts MLLMs by aligning molecular profiles with textual descriptions in a shared embedding space using a bidirectional contrastive objective. The resulting…

FullTransNet: Full Transformer with Local-Global Attention for Video Summarization

Libin Lan, Lu Jiang, Tianshu Yu +2 more

cs.CV · · cited 3× · arXiv ↗ · more like this ·

Video summarization aims to generate a compact, informative, and representative synopsis of raw videos, which is crucial for browsing, analyzing, and understanding video content. Dominant approaches in video summarization primarily rely on recurrent or convolutional neural networks, and more recently on encoder-only transformer architectures. However, these methods typically suffer from several limitations in parallelism, modeling long-range dependencies, and providing explicit generative capabilities. To address these issues, we propose a transformer-like architecture named FullTransNet with two-fold ideas. First, it uses a full transformer with an encoder-decoder structure as an alternative architecture for video summarization. As the full transformer is specifically designed for sequence transduction tasks, its direct application to video summarization is both intuitive and…

Recognizing Video Events with Varying Rhythms

Yikang Li, Tianshu Yu, Baoxin Li

cs.CV · · cited 3× · arXiv ↗ · more like this ·

Recognizing Video events in long, complex videos with multiple sub-activities has received persistent attention recently. This task is more challenging than traditional action recognition with short, relatively homogeneous video clips. In this paper, we investigate the problem of recognizing long and complex events with varying action rhythms, which has not been considered in the literature but is a practical challenge. Our work is inspired in part by how humans identify events with varying rhythms: quickly catching frames contributing most to a specific event. We propose a two-stage \emph{end-to-end} framework, in which the first stage selects the most significant frames while the second stage recognizes the event using the selected frames. Our model needs only \emph{event-level labels} in the training stage, and thus is more practical when the sub-activity labels are missing or…

GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models

5 Team, Aohan Zeng, Xin Lv +168 more

cs.CL · · cited 2× · arXiv ↗ · more like this ·

We present GLM-4.5, an open-source Mixture-of-Experts (MoE) large language model with 355B total parameters and 32B activated parameters, featuring a hybrid reasoning method that supports both thinking and direct response modes. Through multi-stage training on 23T tokens and comprehensive post-training with expert model iteration and reinforcement learning, GLM-4.5 achieves strong performance across agentic, reasoning, and coding (ARC) tasks, scoring 70.1% on TAU-Bench, 91.0% on AIME 24, and 64.2% on SWE-bench Verified. With much fewer parameters than several competitors, GLM-4.5 ranks 3rd overall among all evaluated models and 2nd on agentic benchmarks. We release both GLM-4.5 (355B parameters) and a compact version, GLM-4.5-Air (106B parameters), to advance research in reasoning and agentic AI systems. Code, models, and more information are available at…

Impact of Shrinking Cities on Carbon Emission Efficiency in China

Tianshu Yu, Ling Li, Tao Li

Urban Studies · Sustainability · · cited 2× · DOI ↗ · more like this ·

The issue of urban carbon emission efficiency (CEE) has become a critical problem for global sustainable development, particularly in China, where the phenomenon of shrinking cities has emerged after rapid urbanization. Using panel data from 283 Chinese prefecture-level cities (2000–2016), we examine how urban shrinkage affects CEE through both direct and spatial spillover effects. Our findings show that urban shrinkage significantly improves CEE both directly and indirectly; when a city shrinks, it increases the local CEE by 0.0132%, while the contraction of adjacent cities enhances the local CEE by 0.0312%, leading to a total improvement of 0.0445%. However, the overall CEE in shrinking cities remains lower than the nationwide average, with values consistently below 0.5. The main determinants of CEE are GDP per capita and population size, which show significant direct positive effects…

Unsupervised Speech-text word-level alignment with Dynamic Programming

Tianshu Yu, Zihan Gong, Minghuan Tan +2 more

Artificial Intelligence · · cited 2× · DOI ↗ · more like this ·

Word-level alignment in speech-text pretraining has demonstrated significant effectiveness, particularly with models like SPECTRA that enhance cross-modal interactions and understand multi-turn dialog contexts.However, these advancements are constrained by a reliance on word-level annotated data, limiting their broader applicability and failing to fully exploit the vast amount of unannotated data available.This paper introduces an Unsupervised Speech-text word-level alignment with Dynamic Programming (USDP), which reduces the dependency on scarce annotated resources.We propose an iterative training method for USDP, inspired by the EM algorithm.This approach uses Dynamic Programming and EM principles to iteratively refine temporal alignment predictions.Initially, corresponding speech segments are identified based on the model's temporal predictions.A predictor then forecasts text words…

Speech-Text Dialog Pre-training for Spoken Dialog Understanding with Explicit Cross-Modal Alignment

Tianshu Yu, Haoyu Gao, Ting-En Lin +6 more

cs.CL · · cited 2× · arXiv ↗ · more like this ·

Recently, speech-text pre-training methods have shown remarkable success in many speech and natural language processing tasks. However, most previous pre-trained models are usually tailored for one or two specific tasks, but fail to conquer a wide range of speech-text tasks. In addition, existing speech-text pre-training methods fail to explore the contextual information within a dialogue to enrich utterance representations. In this paper, we propose Speech-text dialog Pre-training for spoken dialog understanding with ExpliCiT cRoss-Modal Alignment (SPECTRA), which is the first-ever speech-text dialog pre-training model. Concretely, to consider the temporality of speech modality, we design a novel temporal position prediction task to capture the speech-text alignment. This pre-training task aims to predict the start and end time of each textual word in the corresponding speech waveform.…

Deep Learning of Determinantal Point Processes via Proper Spectral Sub-gradient

Tianshu Yu, Yikang Li, Baoxin Li

Computer Vision and Pattern Recognition · International Conference on Learning Representations · · cited 2× · OpenAlex ↗ · more like this ·

Determinantal point processes (DPPs) is an effective tool to deliver diversity on multiple machine learning and computer vision tasks. Under deep learning framework, DPP is typically optimized via approximation, which is not straightforward and has some conflict with diversity requirement. We note, however, there has been no deep learning paradigms to optimize DPP directly since it involves matrix inversion which may result in highly computational instability. This fact greatly hinders the wide use of DPP on some specific objectives where DPP serves as a term to measure the feature diversity. In this paper, we devise a simple but effective algorithm to address this issue to optimize DPP term directly expressed with L-ensemble in spectral domain over gram matrix, which is more flexible than learning on parametric kernels. By further taking into account some geometric constraints, our…

Simultaneous Event Localization and Recognition in Surveillance Video

Yikang Li, Tianshu Yu, Baoxin Li

Computer Vision and Pattern Recognition · · cited 2× · DOI ↗ · more like this ·

The ubiquity of video-based surveillance demands automated approaches to analysis of ever-increasing video footages. Action/Event localization and recognition are two critical capabilities in surveillance video analysis, which have been largely addressed separately in the literature. In this paper, we propose an approach to simultaneously localize and recognize visual events from raw surveillance videos, employing an end-to-end learning strategy. Our approach formulates the task as weakly-supervised sequential semantic segmentation, in which we utilize a specific convolutional RNN to capture not only the appearance and the motion information but also their temporal evolution patterns. We tested our approach on the VIRAT 2.0 dataset. The experimental results, in comparison with relevant existing state-of-the-art, suggest that the proposed approach is promising in delivering a practical…