arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Shumin Han

30 papers · 2012–2025

2012: 12013: 02014: 02015: 02016: 02017: 12018: 12019: 12020: 32021: 102022: 42023: 62024: 22025: 1
2012peak 10 in 20212025

Frequent co-authors

Papers

showing 1–25 of 30 RSS

Student-Teacher Feature Pyramid Matching for Anomaly Detection

Guodong Wang, Shumin Han, Errui Ding +1 more

cs.CV · · cited 391× · arXiv ↗ · more like this ·

Anomaly detection is a challenging task and usually formulated as an one-class learning problem for the unexpectedness of anomalies. This paper proposes a simple yet powerful approach to this issue, which is implemented in the student-teacher framework for its advantages but substantially extends it in terms of both accuracy and efficiency. Given a strong model pre-trained on image classification as the teacher, we distill the knowledge into a single student network with the identical architecture to learn the distribution of anomaly-free images and this one-step transfer preserves the crucial clues as much as possible. Moreover, we integrate the multi-scale feature matching strategy into the framework, and this hierarchical feature matching enables the student network to receive a mixture of multi-level knowledge from the feature pyramid under better supervision, thus allowing to…

PP-YOLO: An Effective and Efficient Implementation of Object Detector

Xiang Long, Kaipeng Deng, Guanzhong Wang +8 more

cs.CV · · cited 347× · arXiv ↗ · more like this ·

Object detection is one of the most important areas in computer vision, which plays a key role in various practical scenarios. Due to limitation of hardware, it is often necessary to sacrifice accuracy to ensure the infer speed of the detector in practice. Therefore, the balance between effectiveness and efficiency of object detector must be considered. The goal of this paper is to implement an object detector with relatively balanced effectiveness and efficiency that can be directly applied in actual application scenarios, rather than propose a novel detection model. Considering that YOLOv3 has been widely used in practice, we develop a new object detector based on YOLOv3. We mainly try to combine various existing tricks that almost not increase the number of model parameters and FLOPs, to achieve the goal of improving the accuracy of detector as much as possible while ensuring that…

PP-YOLOv2: A Practical Object Detector

Xin Huang, Xinxin Wang, Wenyu Lv +10 more

cs.CV · · cited 165× · arXiv ↗ · more like this ·

Being effective and efficient is essential to an object detector for practical use. To meet these two concerns, we comprehensively evaluate a collection of existing refinements to improve the performance of PP-YOLO while almost keep the infer time unchanged. This paper will analyze a collection of refinements and empirically evaluate their impact on the final model performance through incremental ablation study. Things we tried that didn't work will also be discussed. By combining multiple effective refinements, we boost PP-YOLO's performance from 45.9% mAP to 49.5% mAP on COCO2017 test-dev. Since a significant margin of performance has been made, we present PP-YOLOv2. In terms of speed, PP-YOLOv2 runs in 68.9FPS at 640x640 input size. Paddle inference engine with TensorRT, FP16-precision, and batch size = 1 further improves PP-YOLOv2's infer speed, which achieves 106.5 FPS. Such a…

Prompt Tuning Inversion for Text-Driven Image Editing Using Diffusion Models

Wenkai Dong, Song Xue, Xiaoyue Duan +1 more

cs.CV · · cited 122× · arXiv ↗ · more like this ·

Recently large-scale language-image models (e.g., text-guided diffusion models) have considerably improved the image generation capabilities to generate photorealistic images in various domains. Based on this success, current image editing methods use texts to achieve intuitive and versatile modification of images. To edit a real image using diffusion models, one must first invert the image to a noisy latent from which an edited image is sampled with a target text prompt. However, most methods lack one of the following: user-friendliness (e.g., additional masks or precise descriptions of the input image are required), generalization to larger domains, or high fidelity to the input image. In this paper, we design an accurate and quick inversion technique, Prompt Tuning Inversion, for text-driven image editing. Specifically, our proposed editing method consists of a reconstruction stage…

Dual-stream Network for Visual Recognition

Mingyuan Mao, Renrui Zhang, Honghui Zheng +6 more

cs.CV · · cited 88× · arXiv ↗ · more like this ·

Transformers with remarkable global representation capacities achieve competitive results for visual tasks, but fail to consider high-level local pattern information in input images. In this paper, we present a generic Dual-stream Network (DS-Net) to fully explore the representation capacity of local and global pattern features for image classification. Our DS-Net can simultaneously calculate fine-grained and integrated features and efficiently fuse them. Specifically, we propose an Intra-scale Propagation module to process two different resolutions in each block and an Inter-Scale Alignment module to perform information interaction across features at dual scales. Besides, we also design a Dual-stream FPN (DS-FPN) to further enhance contextual information for downstream dense predictions. Without bells and whistles, the proposed DS-Net outperforms DeiT-Small by 2.4% in terms of top-1…

Oriented Object Detection with Transformer

Teli Ma, Mingyuan Mao, Honghui Zheng +6 more

cs.CV · · cited 70× · arXiv ↗ · more like this ·

Object detection with Transformers (DETR) has achieved a competitive performance over traditional detectors, such as Faster R-CNN. However, the potential of DETR remains largely unexplored for the more challenging task of arbitrary-oriented object detection problem. We provide the first attempt and implement Oriented Object DEtection with TRansformer ($\bf O^2DETR$) based on an end-to-end network. The contributions of $\rm O^2DETR$ include: 1) we provide a new insight into oriented object detection, by applying Transformer to directly and efficiently localize objects without a tedious process of rotated anchors as in conventional detectors; 2) we design a simple but highly efficient encoder for Transformer by replacing the attention mechanism with depthwise separable convolution, which can significantly reduce the memory and computational cost of using multi-scale features in the…

HS-ResNet: Hierarchical-Split Block on Convolutional Neural Network

Pengcheng Yuan, Shufei Lin, Cheng Cui +5 more

cs.CV · · cited 57× · arXiv ↗ · more like this ·

This paper addresses representational block named Hierarchical-Split Block, which can be taken as a plug-and-play block to upgrade existing convolutional neural networks, improves model performance significantly in a network. Hierarchical-Split Block contains many hierarchical split and concatenate connections within one single residual block. We find multi-scale features is of great importance for numerous vision tasks. Moreover, Hierarchical-Split block is very flexible and efficient, which provides a large space of potential network architectures for different applications. In this work, we present a common backbone based on Hierarchical-Split block for tasks: image classification, object detection, instance segmentation and semantic image segmentation/parsing. Our approach shows significant improvements over all these core tasks in comparison with the baseline. As shown in Figure1…

Layer-wise Searching for 1-bit Detectors

Sheng Xu, Junhe Zhao, Jinhu Lü +3 more

Computer Vision and Pattern Recognition · · cited 29× · DOI ↗ · more like this ·

1-bit detectors show great promise for resource-constrained embedded devices but often suffer from a significant performance gap compared with their real-valued counterparts. The primary reason lies in the error during binarization. This paper presents a layer-wise searching (LWS) strategy to generate 1-bit detectors that maintain a performance very close to the original real-valued model. The approach introduces angular and amplitude loss functions to increase detector capacity. At 1-bit layers, it exploits a differentiable binarization search (DBS) to minimize the angular error in a student-teacher framework. We also learn the scale factor by minimizing the amplitude loss in the same student-teacher framework. Extensive experiments show that LWS-Det outperforms state-of-the-art 1-bit detectors by a considerable margin on the PASCAL VOC and COCO datasets. For example, the LWS-Det…

Learning from Large-scale Noisy Web Data with Ubiquitous Reweighting for Image Classification

Jia Li, Yafei Song, Jianfeng Zhu +5 more

cs.CV · · cited 27× · arXiv ↗ · more like this ·

Many advances of deep learning techniques originate from the efforts of addressing the image classification task on large-scale datasets. However, the construction of such clean datasets is costly and time-consuming since the Internet is overwhelmed by noisy images with inadequate and inaccurate tags. In this paper, we propose a Ubiquitous Reweighting Network (URNet) that learns an image classification model from large-scale noisy data. By observing the web data, we find that there are five key challenges, i.e., imbalanced class sizes, high intra-classes diversity and inter-class similarity, imprecise instances, insufficient representative instances, and ambiguous class labels. To alleviate these challenges, we assume that every training instance has the potential to contribute positively by alleviating the data bias and noise via reweighting the influence of each instance according to…

Feature Rescaling and Fusion for Tiny Object Detection

Jingwei Liu, Yi Gu, Shumin Han +3 more

Computer Vision and Pattern Recognition · IEEE Access · · cited 16× · DOI ↗ · more like this ·

Recent years have witnessed rapid developments on computer vision, however, there are still challenges in detecting tiny objects in a large-scale background. The tiny objects knowledge become sparse and weak due to their tiny size, which makes the tiny objects difficult to be detected with the common approaches. In this paper, a new network named Specific Characteristics based Feature Rescaling and Fusion (SFRF) is designed to detect tiny persons in a broad horizon and massive background. Different from the methods in general, a Nonparametric Adaptive Dense Perceiving Algorithm (NADPA) is designed to automatically select and generate a new resized feature map with the high density distribution of tiny objects. Then, a method called Many-For-One strategy is used for feature fusion of the feature pyramid network (FPN) layers to improve the feature representation and detection. Finally, an…

UV-IDM: Identity-Conditioned Latent Diffusion Model for Face UV-Texture Generation

Hong Li, Yutang Feng, Song Xue +7 more

Computer Vision and Pattern Recognition · · cited 11× · DOI ↗ · more like this ·

3D face reconstruction aims at generating high-fidelity 3D face shapes and textures from single-view or multi-view images. However, current prevailing facial texture generation methods generally suffer from low-quality texture, identity information loss, and inadequate handling of occlusions. To solve these problems, we introduce an Identity-Conditioned Latent Diffusion Model for face UV-texture generation (UV-IDM) to generate photo-realistic textures based on the Basel Face Model (BFM). UV-IDM leverages the powerful texture generation capacity of a latent diffusion model (LDM) to obtain detailed facial textures. To preserve the identity during the reconstruction procedure, we design an identity-conditioned module that can utilize any in-the-wild image as a robust condition for the LDM to guide texture generation. UV-IDM can be easily adapted to different BFM-based methods as a…

Context Autoencoder for Self-Supervised Representation Learning

Xiaokang Chen, Mingyu Ding, Xiaodi Wang +7 more

cs.CV · · cited 9× · arXiv ↗ · more like this ·

We present a novel masked image modeling (MIM) approach, context autoencoder (CAE), for self-supervised representation pretraining. We pretrain an encoder by making predictions in the encoded representation space. The pretraining tasks include two tasks: masked representation prediction - predict the representations for the masked patches, and masked patch reconstruction - reconstruct the masked patches. The network is an encoder-regressor-decoder architecture: the encoder takes the visible patches as input; the regressor predicts the representations of the masked patches, which are expected to be aligned with the representations computed from the encoder, using the representations of visible patches and the positions of visible and masked patches; the decoder reconstructs the masked patches from the predicted encoded representations. The CAE design encourages the separation of learning…

CAE v2: Context Autoencoder with CLIP Target

Xinyu Zhang, Jiahui Chen, Junkun Yuan +10 more

cs.CV · · cited 8× · arXiv ↗ · more like this ·

Masked image modeling (MIM) learns visual representation by masking and reconstructing image patches. Applying the reconstruction supervision on the CLIP representation has been proven effective for MIM. However, it is still under-explored how CLIP supervision in MIM influences performance. To investigate strategies for refining the CLIP-targeted MIM, we study two critical elements in MIM, i.e., the supervision position and the mask ratio, and reveal two interesting perspectives, relying on our developed simple pipeline, context autodecoder with CLIP target (CAE v2). Firstly, we observe that the supervision on visible patches achieves remarkable performance, even better than that on masked patches, where the latter is the standard format in the existing MIM methods. Secondly, the optimal mask ratio positively correlates to the model size. That is to say, the smaller the model, the lower…

Innovative applications of single-atom catalysts in MgH2/Mg system to build high-efficiency hydrogen storage

Lixiang Ma, Lu Zhang, Zhuohan Zhang +3 more

Artificial Intelligence · Journal of Magnesium and Alloys · · cited 6× · DOI ↗ · more like this ·

• Advanced single-atom catalysts in MgH 2 : design, impact, catalytic mechanisms. • Challenges in activity, stability, metal loading of single-atom catalysts in MgH 2 . • Future research: novel reaction pathways, robust metal-support interface synergies. • Industrialization keys: multiscale simulation, interface engineering, synthesis. MgH 2 shows significant potential for a solid-state hydrogen storage medium due to the advantages of high hydrogen capacity, excellent reversibility, and low cost. However, its large-scale application still requires overcoming significant thermodynamic and kinetic hurdles. Catalyst design and optimization enhancements are crucial for the hydrogen storage properties of MgH 2 , wherein single-atom catalysts, characterized by their small size and high proportion of unsaturated coordination sites, have recently demonstrated a significant advance and…

Facial Expression Recognition Algorithm Based on Swin Transformer

Shumin Han, Heming Chang, Zhiguo Shi +1 more

Computer Vision and Pattern Recognition · · cited 6× · DOI ↗ · more like this ·

Expressions are an important non-verbal behavior for humans to convey their emotional information, reflecting their inner activities. As attention in Transformers has excellent feature representation capabilities, it is catching more interests in computer vision field. Swin Transformer is a successful implementation of transformer in computer vision field. This paper proposes a Swin Transformer-based emotion recognition algorithm called Swin Emotion. The algorithm utilizes Swin Transformer as the backbone network and fuses the varying-sized feature vectors extracted from multiple stages, taking into account both the detailed features of the image and its macroscopic characteristics while preserving the integrity of the modeling process. The multi-scale feature fusion network simultaneously uses both low-level and high-level features of images, thereby improving the model's perception…

PAFNet: An Efficient Anchor-Free Object Detector Guidance

Ying Xin, Guanzhong Wang, Mingyuan Mao +5 more

cs.CV · · cited 6× · arXiv ↗ · more like this ·

Object detection is a basic but challenging task in computer vision, which plays a key role in a variety of industrial applications. However, object detectors based on deep learning usually require greater storage requirements and longer inference time, which hinders its practicality seriously. Therefore, a trade-off between effectiveness and efficiency is necessary in practical scenarios. Considering that without constraint of pre-defined anchors, anchor-free detectors can achieve acceptable accuracy and inference speed simultaneously. In this paper, we start from an anchor-free detector called TTFNet, modify the structure of TTFNet and introduce multiple existing tricks to realize effective server and mobile solutions respectively. Since all experiments in this paper are conducted based on PaddlePaddle, we call the model as PAFNet(Paddle Anchor Free Network). For server side, PAFNet…

The 1st Tiny Object Detection Challenge:Methods and Results

Xuehui Yu, Zhenjun Han, Yuqi Gong +22 more

cs.CV · · cited 6× · arXiv ↗ · more like this ·

The 1st Tiny Object Detection (TOD) Challenge aims to encourage research in developing novel and accurate methods for tiny object detection in images which have wide views, with a current focus on tiny person detection. The TinyPerson dataset was used for the TOD Challenge and is publicly released. It has 1610 images and 72651 box-levelannotations. Around 36 participating teams from the globe competed inthe 1st TOD Challenge. In this paper, we provide a brief summary of the1st TOD Challenge including brief introductions to the top three methods.The submission leaderboard will be reopened for researchers that areinterested in the TOD challenge. The benchmark dataset and other information can be found at: https://github.com/ucas-vg/TinyBenchmark.

Private Blocking Technique for Multi-party Privacy-Preserving Record Linkage

Shumin Han, Derong Shen, Tiezheng Nie +2 more

Artificial Intelligence · Data Science and Engineering · · cited 6× · DOI ↗ · more like this ·

The process of matching and integrating records that relate to the same entity from one or more datasets is known as record linkage, and it has become an increasingly important subject in many application areas, including business, government and health system. The data from these areas often contain sensitive information. To prevent privacy breaches, ideally records should be linked in a private way such that no information other than the matching result is leaked in the process, and this technique is called privacy-preserving record linkage (PPRL). With the increasing data, scalability becomes the main challenge of PPRL, and many private blocking techniques have been developed for PPRL. They are aimed at reducing the number of record pairs to be compared in the matching process by removing obvious non-matching pairs without compromising privacy. However, most of them are designed for…

MAFormer: A Transformer Network with Multi-scale Attention Fusion for Visual Recognition

Yunhao Wang, Huixin Sun, Xiaodi Wang +6 more

cs.CV · · cited 4× · arXiv ↗ · more like this ·

Vision Transformer and its variants have demonstrated great potential in various computer vision tasks. But conventional vision transformers often focus on global dependency at a coarse level, which suffer from a learning challenge on global relationships and fine-grained representation at a token level. In this paper, we introduce Multi-scale Attention Fusion into transformer (MAFormer), which explores local aggregation and global feature extraction in a dual-stream framework for visual recognition. We develop a simple but effective module to explore the full potential of transformers for visual representation by learning fine-grained and coarse-grained features at a token level and dynamically fusing them. Our Multi-scale Attention Fusion (MAF) block consists of: i) a local window attention branch that learns short-range interactions within windows, aggregating fine-grained local…

Language-aware Multiple Datasets Detection Pretraining for DETRs

Jing Hao, Song Chen, Xiaodi Wang +1 more

cs.CV · · cited 3× · arXiv ↗ · more like this ·

Pretraining on large-scale datasets can boost the performance of object detectors while the annotated datasets for object detection are hard to scale up due to the high labor cost. What we possess are numerous isolated filed-specific datasets, thus, it is appealing to jointly pretrain models across aggregation of datasets to enhance data volume and diversity. In this paper, we propose a strong framework for utilizing Multiple datasets to pretrain DETR-like detectors, termed METR, without the need for manual label spaces integration. It converts the typical multi-classification in object detection into binary classification by introducing a pre-trained language model. Specifically, we design a category extraction module for extracting potential categories involved in an image and assign these categories into different queries by language embeddings. Each query is only responsible for…

Rethinking the Number of Shots in Robust Model-Agnostic Meta-Learning

Xiaoyue Duan, Guoliang Kang, Runqi Wang +4 more

cs.CV · · cited 2× · arXiv ↗ · more like this ·

Robust Model-Agnostic Meta-Learning (MAML) is usually adopted to train a meta-model which may fast adapt to novel classes with only a few exemplars and meanwhile remain robust to adversarial attacks. The conventional solution for robust MAML is to introduce robustness-promoting regularization during meta-training stage. With such a regularization, previous robust MAML methods simply follow the typical MAML practice that the number of training shots should match with the number of test shots to achieve an optimal adaptation performance. However, although the robustness can be largely improved, previous methods sacrifice clean accuracy a lot. In this paper, we observe that introducing robustness-promoting regularization into MAML reduces the intrinsic dimension of clean sample features, which results in a lower capacity of clean representations. This may explain why the clean accuracy of…

Probabilistic Ranking-Aware Ensembles for Enhanced Object Detections

Mingyuan Mao, Baochang Zhang, David Doermann +5 more

cs.CV · · cited 2× · arXiv ↗ · more like this ·

Model ensembles are becoming one of the most effective approaches for improving object detection performance already optimized for a single detector. Conventional methods directly fuse bounding boxes but typically fail to consider proposal qualities when combining detectors. This leads to a new problem of confidence discrepancy for the detector ensembles. The confidence has little effect on single detectors but significantly affects detector ensembles. To address this issue, we propose a novel ensemble called the Probabilistic Ranking Aware Ensemble (PRAE) that refines the confidence of bounding boxes from detectors. By simultaneously considering the category and the location on the same validation set, we obtain a more reliable confidence based on statistical probability. We can then rank the detected bounding boxes for assembly. We also introduce a bandit approach to address the…

Analysis on the Coupling Effect of Xi’an-Yinchuan High Speed Railway and Economy Development in Shanxi-Gansu-Ningxia District Railway and Economy Development in Shanxi-Gansu-Ningxia District

Shumin Han, Feng Han, Shaojin Zhang +2 more

Transportation · IOP Conference Series Earth and Environmental Science · · cited 2× · DOI ↗ · more like this ·

Abstract Rail transit has been playing a vital role in comprehensive transportation system, there goes without saying that rail transit will stimulate economic growth of the regions alongside railway lines, its coverage and development pattern will exert widespread and far-reaching impact on the area where railway travels through. In Xi’an-Yinchuan High-speed Railway construction setting, firstly, building an accessibility model based on the GIS platform and comparing analysis of traffic accessibility changes in the study area before and after the opening of the high-speed rail. Secondly, relying on the railway network and economic relations to establish comprehensive indicators System, build a coupling model of rail transit and regional economy in Shanxi-Gansu-Ningxia. Finally, predicting the coupling relationship between economy and transportation after the opening of high-speed rail.…

Att-FPA: Boosting Feature Perceive for Object Detection

Jingwei Liu, Yi Gu, Shumin Han +3 more

Computer Vision and Pattern Recognition · IEEE Access · · cited 2× · DOI ↗ · more like this ·

The deep convolutional networks have a great success in vision classification tasks. For object detection, tasks are divided into two subtasks: localization and classification. The detectors scan the whole image to generate object proposals relying on the predefined anchors or points, then classify and fine trim the proposals. The localization task plays an important role in object detection. The foreground objects and background can be easily confused under complex scenes in the existing approaches. Thus, the localization task results in a bad influence on the performance of classification problems. In order to enhance the object localization perceived, a novel method called Attentional Feature Perceive and Augmentation (Att-FPA) is constructed, which devotes setting up a feature dual perceive for localization and classification. Firstly, an attention mask branch (with an attention…

2nd Place Solution in Google AI Open Images Object Detection Track 2019

Ruoyu Guo, Cheng Cui, Yuning Du +6 more

cs.CV · · cited 2× · arXiv ↗ · more like this ·

We present an object detection framework based on PaddlePaddle. We put all the strategies together (multi-scale training, FPN, Cascade, Dcnv2, Non-local, libra loss) based on ResNet200-vd backbone. Our model score on public leaderboard comes to 0.6269 with single scale test. We proposed a new voting method called top-k voting-nms, based on the SoftNMS detection results. The voting method helps us merge all the models' results more easily and achieve 2nd place in the Google AI Open Images Object Detection Track 2019.