arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Lin Song

193 papers · 1990–2026

1990: 11991: 01992: 01993: 11994: 01995: 01996: 01997: 01998: 01999: 12000: 02001: 12002: 12003: 12004: 12005: 42006: 12007: 12008: 42009: 152010: 102011: 32012: 42013: 42014: 72015: 22016: 42017: 32018: 42019: 72020: 112021: 112022: 222023: 152024: 162025: 182026: 20
1990peak 22 in 20222026

Frequent co-authors

Papers

showing 1–25 of 193 RSS

YOLO-World: Real-Time Open-Vocabulary Object Detection

Tianheng Cheng, Lin Song, Yixiao Ge +3 more

cs.CV · · cited 1,113× · arXiv ↗ · more like this ·

The You Only Look Once (YOLO) series of detectors have established themselves as efficient and practical tools. However, their reliance on predefined and trained object categories limits their applicability in open scenarios. Addressing this limitation, we introduce YOLO-World, an innovative approach that enhances YOLO with open-vocabulary detection capabilities through vision-language modeling and pre-training on large-scale datasets. Specifically, we propose a new Re-parameterizable Vision-Language Path Aggregation Network (RepVL-PAN) and region-text contrastive loss to facilitate the interaction between visual and linguistic information. Our method excels in detecting a wide range of objects in a zero-shot manner with high efficiency. On the challenging LVIS dataset, YOLO-World achieves 35.4 AP with 52.0 FPS on V100, which outperforms many state-of-the-art methods in terms of both…

UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition

Xiaohan Ding, Yiyuan Zhang, Yixiao Ge +4 more

cs.CV · · cited 358× · arXiv ↗ · more like this ·

Large-kernel convolutional neural networks (ConvNets) have recently received extensive research attention, but two unresolved and critical issues demand further investigation. 1) The architectures of existing large-kernel ConvNets largely follow the design principles of conventional ConvNets or transformers, while the architectural design for large-kernel ConvNets remains under-addressed. 2) As transformers have dominated multiple modalities, it remains to be investigated whether ConvNets also have a strong universal perception ability in domains beyond vision. In this paper, we contribute from two aspects. 1) We propose four architectural guidelines for designing large-kernel ConvNets, the core of which is to exploit the essential characteristics of large kernels that distinguish them from small kernels - they can see wide without going deep. Following such guidelines, our proposed…

GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction

Rui Yang, Lin Song, Yanwei Li +4 more

cs.CV · · cited 339× · arXiv ↗ · more like this ·

This paper aims to efficiently enable Large Language Models (LLMs) to use multimodal tools. Advanced proprietary LLMs, such as ChatGPT and GPT-4, have shown great potential for tool usage through sophisticated prompt engineering. Nevertheless, these models typically rely on prohibitive computational costs and publicly inaccessible data. To address these challenges, we propose the GPT4Tools based on self-instruct to enable open-source LLMs, such as LLaMA and OPT, to use tools. It generates an instruction-following dataset by prompting an advanced teacher with various multi-modal contexts. By using the Low-Rank Adaptation (LoRA) optimization, our approach facilitates the open-source LLMs to solve a range of visual problems, including visual comprehension and image generation. Moreover, we provide a benchmark to evaluate the ability of LLMs to use tools, which is performed in both…

End-to-End Object Detection with Fully Convolutional Network

Jianfeng Wang, Lin Song, Zeming Li +3 more

cs.CV · · cited 248× · arXiv ↗ · more like this ·

Mainstream object detectors based on the fully convolutional network has achieved impressive performance. While most of them still need a hand-designed non-maximum suppression (NMS) post-processing, which impedes fully end-to-end training. In this paper, we give the analysis of discarding NMS, where the results reveal that a proper label assignment plays a crucial role. To this end, for fully convolutional detectors, we introduce a Prediction-aware One-To-One (POTO) label assignment for classification to enable end-to-end detection, which obtains comparable performance with NMS. Besides, a simple 3D Max Filtering (3DMF) is proposed to utilize the multi-scale features and improve the discriminability of convolutions in the local region. With these techniques, our end-to-end framework achieves competitive performance against many state-of-the-art detectors with NMS on COCO and CrowdHuman…

Learning Dynamic Routing for Semantic Segmentation

Yanwei Li, Lin Song, Yukang Chen +4 more

cs.CV · · cited 189× · arXiv ↗ · more like this ·

Recently, numerous handcrafted and searched networks have been applied for semantic segmentation. However, previous works intend to handle inputs with various scales in pre-defined static architectures, such as FCN, U-Net, and DeepLab series. This paper studies a conceptually new method to alleviate the scale variance in semantic representation, named dynamic routing. The proposed framework generates data-dependent routes, adapting to the scale distribution of each image. To this end, a differentiable gating function, called soft conditional gate, is proposed to select scale transform paths on the fly. In addition, the computational cost can be further reduced in an end-to-end manner by giving budget constraints to the gating function. We further relax the network level routing space to support multi-path propagations and skip-connections in each forward, bringing substantial network…

TACNet: Transition-Aware Context Network for Spatio-Temporal Action Detection

Lin Song, Shiwei Zhang, Gang Yu +1 more

cs.CV · · cited 92× · arXiv ↗ · more like this ·

Current state-of-the-art approaches for spatio-temporal action detection have achieved impressive results but remain unsatisfactory for temporal extent detection. The main reason comes from that, there are some ambiguous states similar to the real actions which may be treated as target actions even by a well-trained network. In this paper, we define these ambiguous samples as "transitional states", and propose a Transition-Aware Context Network (TACNet) to distinguish transitional states. The proposed TACNet includes two main components, i.e., temporal context detector and transition-aware classifier. The temporal context detector can extract long-term context information with constant time complexity by constructing a recurrent network. The transition-aware classifier can further distinguish transitional states by classifying action and transitional states simultaneously. Therefore…

Meta-Adapter: An Online Few-shot Learner for Vision-Language Model

Cheng Cheng, Lin Song, Ruoyi Xue +4 more

cs.CV · · cited 64× · arXiv ↗ · more like this ·

The contrastive vision-language pre-training, known as CLIP, demonstrates remarkable potential in perceiving open-world visual concepts, enabling effective zero-shot image recognition. Nevertheless, few-shot learning methods based on CLIP typically require offline fine-tuning of the parameters on few-shot samples, resulting in longer inference time and the risk of over-fitting in certain domains. To tackle these challenges, we propose the Meta-Adapter, a lightweight residual-style adapter, to refine the CLIP features guided by the few-shot samples in an online manner. With a few training samples, our method can enable effective few-shot learning capabilities and generalize to unseen data or tasks without additional fine-tuning, achieving competitive performance and high efficiency. Without bells and whistles, our approach outperforms the state-of-the-art online few-shot learning method…

High Point Walking for Health: Creating Built and Social Environments That Support Walking in a Public Housing Community

JAMES KRIEGER, Janice C. Rabkin, Denise Sharify +1 more

Transportation · American Journal of Public Health · · cited 64× · DOI ↗ · more like this ·

OBJECTIVES: We implemented and evaluated multiple interventions to increase walking activity at a multicultural public housing site. METHODS: A community-based participatory research partnership and community action teams assessed assets and barriers related to walking and developed multiple interventions to promote walking activity. Interventions included sponsoring walking groups, improving walking routes, providing information about walking options, and advocating for pedestrian safety. A pre-post study design was used to assess the changes in walking activity. RESULTS: Self-reported walking activity increased among walking group participants from 65 to 109 minutes per day (P = .001). The proportion that reported being at least moderately active for at least 150 minutes per week increased from 62% to 81% (P = .018). CONCLUSIONS: A multicomponent intervention developed through…

Learnable Tree Filter for Structure-preserving Feature Transform

Lin Song, Yanwei Li, Zeming Li +4 more

cs.CV · · cited 58× · arXiv ↗ · more like this ·

Learning discriminative global features plays a vital role in semantic segmentation. And most of the existing methods adopt stacks of local convolutions or non-local blocks to capture long-range context. However, due to the absence of spatial structure preservation, these operators ignore the object details when enlarging receptive fields. In this paper, we propose the learnable tree filter to form a generic tree filtering module that leverages the structural property of minimal spanning tree to model long-range dependencies while preserving the details. Furthermore, we propose a highly efficient linear-time algorithm to reduce resource consumption. Thus, the designed modules can be plugged into existing deep neural networks conveniently. To this end, tree filtering modules are embedded to formulate a unified framework for semantic segmentation. We conduct extensive ablation studies to…

Privacy-Preserving Threshold-Based Image Retrieval in Cloud-Assisted Internet of Things

Lin Song, Yinbin Miao, Jian Weng +3 more

Computer Vision and Pattern Recognition · IEEE Internet of Things Journal · · cited 47× · DOI ↗ · more like this ·

Encrypted image retrieval is a promising technique for achieving data confidentiality and searchability the in cloud-assisted Internet of Things (IoT) environment. However, most of the existing top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> ranked image retrieval solutions have low retrieval efficiency and may leak the values and orders of similarity scores to the cloud server. Hence, if a malicious server learns user background information through some improper means, then the malicious server can potentially infer user preferences and guess the most similar image content according to similarity scores. To solve the above challenges, we propose a privacy-preserving threshold-based image retrieval scheme using the convolutional neural network (CNN) model and a…

Dynamic Grained Encoder for Vision Transformers

Lin Song, Songyang Zhang, Songtao Liu +5 more

cs.CV · · cited 44× · arXiv ↗ · more like this ·

Transformers, the de-facto standard for language modeling, have been recently applied for vision tasks. This paper introduces sparse queries for vision transformers to exploit the intrinsic spatial redundancy of natural images and save computational costs. Specifically, we propose a Dynamic Grained Encoder for vision transformers, which can adaptively assign a suitable number of queries to each spatial region. Thus it achieves a fine-grained representation in discriminative regions while keeping high efficiency. Besides, the dynamic grained encoder is compatible with most vision transformer frameworks. Without bells and whistles, our encoder allows the state-of-the-art vision transformers to reduce computational complexity by 40%-60% while maintaining comparable performance on image classification. Extensive experiments on object detection and segmentation further demonstrate the…

BoxSnake: Polygonal Instance Segmentation with Box Supervision

Rui Yang, Lin Song, Yixiao Ge +1 more

cs.CV · · cited 40× · arXiv ↗ · more like this ·

Box-supervised instance segmentation has gained much attention as it requires only simple box annotations instead of costly mask or polygon annotations. However, existing box-supervised instance segmentation models mainly focus on mask-based frameworks. We propose a new end-to-end training technique, termed BoxSnake, to achieve effective polygonal instance segmentation using only box annotations for the first time. Our method consists of two loss functions: (1) a point-based unary loss that constrains the bounding box of predicted polygons to achieve coarse-grained segmentation; and (2) a distance-aware pairwise loss that encourages the predicted polygons to fit the object boundaries. Compared with the mask-based weakly-supervised methods, BoxSnake further reduces the performance gap between the predicted segmentation and the bounding box, and shows significant superiority on the…

Scale Space Histogram of Oriented Gradients for Human Detection

Ning He, Jiaheng Cao, Lin Song

Computer Vision and Pattern Recognition · · cited 39× · DOI ↗ · more like this ·

Human detection is the task of finding presence and position of human beings in images, In this paper, we apply scale space theory to detection human in still images. By integrating scale space theory with histogram of oriented gradients(HOG), we designed a new feature descriptor called scale space histogam of oriented gradients (SS-HOG). SS-HOG focus on the multiple scale property of describe an object. Using HOGs at multiple scale, SS-HOG encodes more information to discriminate human bodies from other object types than traditional uni-scale HOGs Experiments on INRIA person dataset demonstrate the effectiveness of our method.

GLNet: Global Local Network for Weakly Supervised Action Localization

Shiwei Zhang, Lin Song, Changxin Gao +1 more

Computer Vision and Pattern Recognition · IEEE Transactions on Multimedia · · cited 37× · DOI ↗ · more like this ·

In this paper, we address the challenging problem of weakly supervised spatio-temporal action localization for which only video-level action labels are available during training. To solve this problem, we propose an end-to-end Global Local Network (GLNet) to predict the probability distribution simultaneously in both spatial and temporal space. The proposed GLNet model includes two key components: a local spatial module and a global temporal module. The local spatial module aims to predict the frame-level spatial distribution by encoding short-term temporal information. In particular, we propose a Region Actionness Network (RAN) to select the target region boxes from the precomputed exhaustive proposals. The global temporal module can predict temporal distribution by a long-term temporal structure modelling. Specifically, we design a temporal fusion-and-excitation architecture on the…

Evidential classification of incomplete instance based on K-nearest centroid neighbor

Zongfang Ma, Zhe Liu, Chan Luo +1 more

Information Systems · Journal of Intelligent & Fuzzy Systems · · cited 36× · DOI ↗ · more like this ·

Classification of incomplete instance is a challenging problem due to the missing features generally cause uncertainty in the classification result. A new evidential classification method of incomplete instance based on adaptive imputation thanks to the framework of evidence theory. Specifically, the missing values of different incomplete instances in test set are adaptively estimated based on Shannon entropy and K-nearest centroid neighbors (KNCNs) technology. The single or multiple edited instances (with estimations) then are classified by the chosen classifier to get single or multiple classification results for the instances with different discounting (weighting) factors, and a new adaptive global fusion method finally is proposed to unify the different discounted results. The proposed method can well capture the imprecision degree of classification by submitting the instances that…

ViTrack: Efficient Tracking on the Edge for Commodity Video Surveillance Systems

Cheng Lin-song, Jiliang Wang, Yinghui Li

Computer Vision and Pattern Recognition · IEEE Transactions on Parallel and Distributed Systems · · cited 31× · DOI ↗ · more like this ·

Nowadays, video surveillance systems are widely deployed in various places, e.g., schools, parks, airports, roads, etc. However, existing video surveillance systems are far from full utilization due to high computation overhead in video processing. In this work, we present ViTrack, a framework for efficient multi-video tracking using computation resource on the edge for commodity video surveillance systems. In the heart of ViTrack lies a two layer spatial/temporal compressed target detection method to significantly reduce the computation overhead by combining videos from multiple cameras. Further, ViTrack derives the video relationship and camera information even in absence of camera location, direction, etc. To alleviate the impact of variant video quality and missing targets, ViTrack leverages a Markov Model based approach to efficiently recover missing information and finally derive…

NIPM-sWMF: Toward Efficient FPGA Design for High-Definition Large-Disparity Stereo Matching

Xuchong Zhang, Hongbin Sun, Shiqiang Chen +2 more

Computer Vision and Pattern Recognition · IEEE Transactions on Circuits and Systems for Video Technology · · cited 23× · DOI ↗ · more like this ·

Large disparity stereo matching is critical to the application of a stereo vision system especially for outdoor scenes. Nevertheless, how to efficiently design high accuracy large-disparity stereo matching on a field-programmable gate array (FPGA) is still a grand challenge. The computational complexity of previously proposed stereo matching is inevitably proportional to disparity range; hence their hardware designs become very inefficient when the disparity range is large. Motivated by the original PatchMatch and weighted median filtering (WMF) algorithms, this paper proposes a non-iterative PatchMatch and separable WMF (NIPM-sWMF) algorithm to significantly reduce the computational complexity of stereo matching and make it independent of disparity range. Moreover, we also propose a fully pipelined architecture design on FPGA that employs several hardware techniques to efficiently…

Generalization of Safe Optimal Control Actions on Networked Multiagent Systems

Lin Song, Neng Wan, Aditya Gahlawat +3 more

Artificial Intelligence · IEEE Transactions on Control of Network Systems · · cited 22× · DOI ↗ · more like this ·

In this article, we propose a unified framework to instantly generate a safe optimal control action for a new task from existing controllers on multiagent systems. The control action composition is achieved by taking a weighted mixture of the existing controllers according to the contribution of each component task. Instead of sophisticatedly tuning the cost parameters and other hyperparameters for safe and reliable behavior in the optimal control framework, the safety of each single-task solution is guaranteed using the control barrier functions (CBFs) for high relative degree stochastic systems, which constrains the system state within a known safe operation region where it originates from. Linearity of CBF constraints in control ensures the feasibility of safe control action composition. The discussed framework can immediately provide reliable solutions to new tasks by taking a…

Localization Algorithm and Implementation for Focal of Pulmonary Tuberculosis Chest Image

Yang Yang, Ya-Lin Song

Computer Vision and Pattern Recognition · · cited 22× · DOI ↗ · more like this ·

According to the technology of computer graphics and image processing combined with medical knowledge, and based on the feature analysis of X-ray chest films of pulmonary tuberculosis, the paper puts forward some algorithms. Some of these algorithms can establish a localizing frame of reference by some special methods of the rib image processing that are edge detecting method, boundary tracing method and information extracting method, and others can get the professional result of localization after implemented focal segmentation and focal localization. Successful application of the technology of computer graphics and image processing is finally implemented in the filed of medical image.

A shot boundary detection method based on color feature

Hua Zhang, Ruimin Hu, Lin Song

Computer Vision and Pattern Recognition · · cited 21× · DOI ↗ · more like this ·

Shot is usually used as basic unit for video content analysis, content-based indexing and retrieval. In this paper, a new method of shot boundary detection based on color feature is proposed aiming to obtain accurate detection with inexpensive cost. The technique is able to detect cut shot boundaries through the analysis of color histogram differences and an adaptive threshold that is based on a sliding window. For gradual transitions such as fades and dissolves, a preprocessing is introduced, and local histogram differences are quantified to binary values by selecting a threshold automatically with reference to the variation of histogram differences. The experimental results show that this proposed method can gain good precision and recall.

Towards Automatic Measurement of Probabilistic Processes

Lin Song, Yuxin Deng, Xiaojuan Cai

Computational Theory and Mathematics · · cited 18× · DOI ↗ · more like this ·

In this paper we propose a metric for finite processes in a probabilistic extension of CSP. The kernel of the metric corresponds to trace equivalence and most of the operators in the process algebra is shown to satisfy non-expansiveness property with respect to this metric. We also provide an algorithm to calculate the distance between two processes to a prescribed discount factor in polynomial time. The algorithm has been implemented in a tool that helps us to measure processes automatically.

Credal Transfer Learning With Multi-Estimation for Missing Data

Zongfang Ma, Zhe Liu, Yiru Zhang +2 more

Artificial Intelligence · IEEE Access · · cited 15× · DOI ↗ · more like this ·

Transfer learning (TL) has grown popular in recent years. It is effective to improve the classification accuracy in the target domain by using the training knowledge in the related domain (called source domain). However, the classification of missing data (or incomplete data) is a challenging task for TL because different strategies of imputation may have strong impacts on learning models. To address this problem, we propose credal transfer learning (CTL) with multi-estimation for missing data based on belief function theory by introducing uncertainty and imprecision in data imputation procedure. CTL mainly consists of three steps: Firstly, the query patterns are reasonably mapped into multiple versions in source domain to characterize the uncertainty caused by missing values. Afterwards, the multiple mapping patterns are classified in the source domain to obtain the corresponding…

Adaptation is Useless for Two Discrete Additive-Noise Two-Way Channels

Lin Song, Fady Alajaji, Tamas Linder

cs.IT · · cited 14× · arXiv ↗ · more like this ·

In two-way channels, each user transmits and receives at the same time. This allows each encoder to interactively adapt the current input to its own message and all previously received signals. Such coding approach can introduce correlation between inputs of different users, since all the users' outputs are correlated by the nature of the channel. However, for some channels, such adaptation in the coding scheme and its induced correlation among users are useless in the sense that they do not help enlarge the capacity region with respect to the standard coding method (where each user encodes only based on its own message). In this paper, it is shown that adaptation is not helpful for enlarging the capacity region of two classes of two-way discrete channels: the modulo additive-noise channel with memory and the multiple access/degraded broadcast channel.

DiffTune: Autotuning Through Autodifferentiation

Sheng Cheng, Minkyung Kim, Lin Song +4 more

Hardware and Architecture · IEEE Transactions on Robotics · · cited 13× · DOI ↗ · more like this ·

The performance of robots in high-level tasks depends on the quality of their lower level controller, which requires fine-tuning. However, the intrinsically nonlinear dynamics and controllers make tuning a challenging task when it is done by hand. In this article, we present DiffTune, a novel, gradient-based automatic tuning framework. We formulate the controller tuning as a parameter optimization problem. Our method unrolls the dynamical system and controller as a computational graph and updates the controller parameters through gradient-based optimization. The gradient is obtained using sensitivity propagation, which is the only method for gradient computation when tuning for a physical system instead of its simulated counterpart. Furthermore, we use <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math…

Privacy-Preserving Arbitrary Geometric Range Query in Mobile Internet of Vehicles

Yinbin Miao, Lin Song, Xinghua Li +3 more

Artificial Intelligence · IEEE Transactions on Mobile Computing · · cited 12× · DOI ↗ · more like this ·

The mobile Internet of Vehicles (IoVs) has great potential for intelligent transportation, and creates spatial data query demands to realize the value of data. Outsourcing spatial data to a cloud server eliminates the need for local computation and storage, but it leads to data security and privacy threats caused by untrusted third-parties. Existing privacy-preserving spatial range query solutions based on Homomorphic Encryption (HE) have been developed to increase security. However, in the single server model, the private key is held by the query user, which incurs high computation and communication burdens on query users due to multiple rounds of interactions. Moreover, exposing data access patterns to semi-honest servers is highly vulnerable to frequency and statistical attacks. To solve these issues, in this paper we propose a secure spatial location query within arbitrary geometric…