Author

Tan Yu

170 papers · 1983–2026

1983peak 22 in 20222026

Frequent co-authors

Papers

showing 1–25 of 170 RSS

Multi-view Harmonized Bilinear Network for 3D Object Recognition

Tan Yu, Jingjing Meng, Junsong Yuan

Computer Vision and Pattern Recognition · June 2018 · cited 329× · DOI ↗ · more like this ·

View-based methods have achieved considerable success in 3D object recognition tasks. Different from existing view-based methods pooling the view-wise features, we tackle this problem from the perspective of patches-to-patches similarity measurement. By exploiting the relationship between polynomial kernel and bilinear pooling, we obtain an effective 3D object representation by aggregating local convolutional features through bilinear pooling. Meanwhile, we harmonize different components inherited in the bilinear feature to obtain a more discriminative representation. To achieve an end-to-end trainable framework, we incorporate the harmonized bilinear pooling as a layer of a network, constituting the proposed Multi-view Harmonized Bilinear Network (MHBN). Systematic experiments conducted on two public benchmark datasets demonstrate the efficacy of the proposed methods in 3D object…

S$^2$-MLP: Spatial-Shift MLP Architecture for Vision

Tan Yu, Xu Li, Yunfeng Cai +2 more

cs.CV · 23 Jun 2021 · cited 239× · arXiv ↗ · more like this ·

Recently, visual Transformer (ViT) and its following works abandon the convolution and exploit the self-attention operation, attaining a comparable or even higher accuracy than CNNs. More recently, MLP-Mixer abandons both the convolution and the self-attention operation, proposing an architecture containing only MLP layers. To achieve cross-patch communications, it devises an additional token-mixing MLP besides the channel-mixing MLP. It achieves promising results when training on an extremely large-scale dataset. But it cannot achieve as outstanding performance as its CNN and ViT counterparts when training on medium-scale datasets such as ImageNet1K and ImageNet21K. The performance drop of MLP-Mixer motivates us to rethink the token-mixing MLP. We discover that the token-mixing MLP is a variant of the depthwise convolution with a global reception field and spatial-specific…

S<sup>2</sup>-MLP: Spatial-Shift MLP Architecture for Vision

Tan Yu, Xu Li, Yunfeng Cai +2 more

Computer Vision and Pattern Recognition · 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · 2022 · cited 187× · DOI ↗ · more like this ·

Recently, visual Transformer (ViT) and its following works abandon the convolution and exploit the self-attention operation, attaining a comparable or even higher accuracy than CNN. More recently, MLP-mixer abandons both the convolution and the self-attention operation, proposing an architecture containing only MLP layers. To achieve cross-patch communications, it devises an additional token-mixing MLP besides the channel-mixing MLP. It achieves promising results when training on an extremely large-scale dataset such as JFT-300M. But it cannot achieve as outstanding performance as its CNN and ViT counterparts when training on medium-scale datasets such as ImageNet-1K. The performance drop of MLP-mixer motivates us to rethink the token-mixing MLP. We discover that token-mixing operation in MLP-mixer is a variant of depthwise convolution with a global reception field and spatial-specific…

An Efficient and Anonymous Buyer-Seller Watermarking Protocol

Chin‐Laung Lei, Pao-Tan Yu, Pei-I Tsai +1 more

Computer Vision and Pattern Recognition · IEEE Transactions on Image Processing · 15 Nov 2004 · cited 176× · DOI ↗ · more like this ·

For the purpose of deterring unauthorized duplication and distribution of multimedia contents, a seller may insert a unique digital watermark into each copy of the multimedia contents to be sold. When an illegal replica is found in the market sometime later, the seller can determine the responsible distributor by examining the watermark embedded. However, the accusation against the charged distributor, who was the buyer in some earlier transaction, is objectionable because the seller also has access to the watermarked copies and, hence, is able to release such a replica on her own. In this paper, a watermarking protocol is proposed to avoid such difficulties, known as the customer's right problem, in the phase of arbitration. The proposed watermarking protocol also provides a fix to Memon and Wong's scheme by solving the unbinding problem. In addition, the buyer is no longer required to…

Temporal Structure Mining for Weakly Supervised Action Detection

Tan Yu, Zhou Ren, Yuncheng Li +3 more

Computer Vision and Pattern Recognition · October 2019 · cited 104× · DOI ↗ · more like this ·

Different from the fully-supervised action detection problem that is dependent on expensive frame-level annotations, weakly supervised action detection (WSAD) only needs video-level annotations, making it more practical for real-world applications. Existing WSAD methods detect action instances by scoring each video segment (a stack of frames) individually. Most of them fail to model the temporal relations among video segments and cannot effectively characterize action instances possessing latent temporal structure. To alleviate this problem in WSAD, we propose the temporal structure mining (TSM) approach. In TSM, each action instance is modeled as a multi-phase process and phase evolving within an action instance, \emph{i.e.}, the temporal structure, is exploited. Meanwhile, the video background is modeled by a background phase, which separates different action instances in an untrimmed…

S$^2$-MLPv2: Improved Spatial-Shift MLP Architecture for Vision

Tan Yu, Xu Li, Yunfeng Cai +2 more

cs.CV · 2 Aug 2021 · cited 70× · arXiv ↗ · more like this ·

Recently, MLP-based vision backbones emerge. MLP-based vision architectures with less inductive bias achieve competitive performance in image recognition compared with CNNs and vision Transformers. Among them, spatial-shift MLP (S$^2$-MLP), adopting the straightforward spatial-shift operation, achieves better performance than the pioneering works including MLP-mixer and ResMLP. More recently, using smaller patches with a pyramid structure, Vision Permutator (ViP) and Global Filter Network (GFNet) achieve better performance than S$^2$-MLP. In this paper, we improve the S$^2$-MLP vision backbone. We expand the feature map along the channel dimension and split the expanded feature map into several parts. We conduct different spatial-shift operations on split parts. Meanwhile, we exploit the split-attention operation to fuse these split parts. Moreover, like the counterparts, we adopt…

MVT: Multi-view Vision Transformer for 3D Object Recognition

Shuo Chen, Tan Yu, Ping Li

cs.CV · 25 Oct 2021 · cited 69× · arXiv ↗ · more like this ·

Inspired by the great success achieved by CNN in image recognition, view-based methods applied CNNs to model the projected views for 3D object understanding and achieved excellent performance. Nevertheless, multi-view CNN models cannot model the communications between patches from different views, limiting its effectiveness in 3D object recognition. Inspired by the recent success gained by vision Transformer in image recognition, we propose a Multi-view Vision Transformer (MVT) for 3D object recognition. Since each patch feature in a Transformer block has a global reception field, it naturally achieves communications between patches from different views. Meanwhile, it takes much less inductive bias compared with its CNN counterparts. Considering both effectiveness and efficiency, we develop a global-local structure for our MVT. Our experiments on two public benchmarks, ModelNet40 and…

A Dynamic and Cross-Domain Authentication Asymmetric Group Key Agreement in Telemedicine Application

Zhang Qikun, Yong Gan, Zhang Quanxin +2 more

Computer Networks and Communications · IEEE Access · 2018 · cited 66× · DOI ↗ · more like this ·

Telemedicine offers medical services remotely via telecommunications systems and physiological monitoring devices. Group-oriented communication is an important application for telemedicine. However, transmission of information over an insecure channel such as Internet or private data storing generates a security problem. Therefore, authentication, confidentiality, and privacy are important challenges in telemedicine. Therefore, developing suitable encryption communication protocol for group communication is quite important for modern medicine. Group key agreement is one way to ensure the security of group-oriented communication for telemedicine. In this paper, we propose a dynamic and cross-domain authenticated asymmetric group key agreement. The protocol adopts cross-domain authentication mechanism to avoid the security risks of key escrow and the complexity of certificate management.…

Heterogeneous Attention Network for Effective and Efficient Cross-modal Retrieval

Tan Yu, Yi Yang, Yi Li +3 more

Computer Vision and Pattern Recognition · 11 Jul 2021 · cited 51× · DOI ↗ · more like this ·

Traditionally, the task of cross-modal retrieval is tackled through joint embedding. However, the global matching used in joint embedding methods often fails to effectively describe matchings between local regions of the image and words in the text. Hence they may not be effective in capturing the relevance between the text and the image. In this work, we propose a heterogeneous attention network (HAN) for effective and efficient cross-modal retrieval. The proposed HAN represents an image by a set of bounding box features and a sentence by a set of word features. The relevance between the image and the sentence is determined by the set-to-set matching between the set of word features and the set of bounding box features. To enhance the matching effectiveness, we exploit the proposed heterogeneous attention layer to provide the cross-modal context for word features as well as bounding…

Multi‐scale attention encoder for street‐to‐aerial image geo‐localization

Songlian Li, Zhigang Tu, Yujin Chen +1 more

Computer Vision and Pattern Recognition · CAAI Transactions on Intelligence Technology · 8 Jan 2022 · cited 48× · DOI ↗ · more like this ·

Abstract The goal of street‐to‐aerial cross‐view image geo‐localization is to determine the location of the query street‐view image by retrieving the aerial‐view image from the same place. The drastic viewpoint and appearance gap between the aerial‐view and the street‐view images brings a huge challenge against this task. In this paper, we propose a novel multiscale attention encoder to capture the multiscale contextual information of the aerial/street‐view images. To bridge the domain gap between these two view images, we first use an inverse polar transform to make the street‐view images approximately aligned with the aerial‐view images. Then, the explored multiscale attention encoder is applied to convert the image into feature representation with the guidance of the learnt multiscale information. Finally, we propose a novel global mining strategy to enable the network to pay more…

FACTS About Building Retrieval Augmented Generation-based Chatbots

Rama Akkiraju, Anbang Xu, Deepak Bora +35 more

cs.LG · 10 Jul 2024 · cited 37× · arXiv ↗ · more like this ·

Enterprise chatbots, powered by generative AI, are emerging as key applications to enhance employee productivity. Retrieval Augmented Generation (RAG), Large Language Models (LLMs), and orchestration frameworks like Langchain and Llamaindex are crucial for building these chatbots. However, creating effective enterprise chatbots is challenging and requires meticulous RAG pipeline engineering. This includes fine-tuning embeddings and LLMs, extracting documents from vector databases, rephrasing queries, reranking results, designing prompts, honoring document access controls, providing concise responses, including references, safeguarding personal information, and building orchestration agents. We present a framework for building RAG-based chatbots based on our experience with three NVIDIA chatbots: for IT/HR benefits, financial earnings, and general content. Our contributions are…

BOAT: Bilateral Local Attention Vision Transformer

Tan Yu, Gangming Zhao, Ping Li +1 more

cs.CV · January 2022 · cited 35× · arXiv ↗ · more like this ·

Vision Transformers achieved outstanding performance in many computer vision tasks. Early Vision Transformers such as ViT and DeiT adopt global self-attention, which is computationally expensive when the number of patches is large. To improve efficiency, recent Vision Transformers adopt local self-attention mechanisms, where self-attention is computed within local windows. Despite the fact that window-based local self-attention significantly boosts efficiency, it fails to capture the relationships between distant but similar patches in the image plane. To overcome this limitation of image-space local attention, in this paper, we further exploit the locality of patches in the feature space. We group the patches into multiple clusters using their features, and self-attention is computed within every cluster. Such feature-space local attention effectively captures the connections between…

Detecting text based image with optical character recognition for English translation and speech using Android

Sathiapriya Ramiah, Tan Yu Liong, Manoj Jayabalan

Computer Vision and Pattern Recognition · December 2015 · cited 35× · DOI ↗ · more like this ·

Smartphones have been known as most commonly used electronic devices in daily life today. As hardware embedded in smartphones can perform much more task than traditional phones, the smartphones are no longer just a communication device but also considered as a powerful computing device which able to capture images, record videos, surf the internet and etc. With advancement of technology, it is possible to apply some techniques to perform text detection and translation. Therefore, an application that allows smartphones to capture an image and extract the text from it to translate into English and speech it out is no longer a dream. In this study, an Android application is developed by integrating Tesseract OCR engine, Bing translator and phones' built-in speech out technology. Final deliverable is tested by various type of target end user from a different language background and…

Combo-Attention Network for Baidu Video Advertising

Tan Yu, Yi Yang, Yi Li +3 more

Computer Vision and Pattern Recognition · 20 Aug 2020 · cited 32× · DOI ↗ · more like this ·

With the progress of communication technology and the popularity of the smart phone, videos grow to be the largest medium. Since videos can grab a customer's attention quickly and leave a big impression, video ads can gain more trust than traditional ads. Thus advertisers start to pour more resources into making creative video ads to built the connections with potential customers. Baidu, as the leading search engine company in China, receives billions of search queries per day. In this paper, we introduce a technique used in Baidu video advertising for feeding relevant video ads according to the user's query. Note that, retrieving relevant videos using the text query is a cross-modal problem. Due to the modal gap, the text-to-video search is more challenging than well exploited text-to-text search and image-to-image search. To tackle this challenge, we propose a Combo-Attention Network…

Algorithm of laser spot detection based on circle fitting

Tan Yu

Computer Vision and Pattern Recognition · Infrared and Laser Engineering · 2002 · cited 32× · OpenAlex ↗ · more like this ·

Laser spot detection is the key technique used in optical measurement. The precision and speed of the detection algorithm affect those of the optical measurement system directly. The traditional algorithms such as gravity model, Hough transform are unsatisfactory under some conditions. The laser spot detection algorithm referred in this paper is based on the LSM (least square method), and a circle is adopted to approximate the edge of the laser spot. Not only the center and radius can be detected simultaneously, but also the orientation precision is in the order of sub\|pixels, and the speed is fast. The algorithm is suitable for real\|time optical measurement.

Efficient Object Instance Search Using Fuzzy Objects Matching

Tan Yu, Yuwei Wu, Sreyasee Das Bhattacharjee +1 more

Computer Vision and Pattern Recognition · Proceedings of the AAAI Conference on Artificial Intelligence · 12 Feb 2017 · cited 28× · DOI ↗ · more like this ·

Recently, global features aggregated from local convolutional features of the convolutional neural network have shown to be much more effective in comparison with hand-crafted features for image retrieval. However, the global feature might not effectively capture the relevance between the query object and reference images in the object instance search task, especially when the query object is relatively small and there exist multiple types of objects in reference images. Moreover, the object instance search requires to localize the object in the reference image, which may not be achieved through global representations. In this paper, we propose a Fuzzy Objects Matching (FOM) framework to effectively and efficiently capture the relevance between the query object and reference images in the dataset. In the proposed FOM scheme, object proposals are utilized to detect the potential regions…

UAV aerial image target detection based on BLUR-YOLO

Tongyuan Huang, JinJiang Zhu, Yao Liu +1 more

Computer Vision and Pattern Recognition · Remote Sensing Letters · February 2023 · cited 23× · DOI ↗ · more like this ·

With the development of unmanned aerial vehicle (UAV) remote sensing technology, target detection based on UAV images has increasingly become a hot spot for research. Aiming at the problems of many small target instances, complex backgrounds and difficult feature extraction in UAV images, we propose a UAV aerial image target detection algorithm called BLUR-YOLO. First, the h-swish activation function is used in the backbone network and the neck network to increase the expressiveness of the model. Second, an attention mechanism (CoordAttention) is added to the bottleneck layer of the backbone network, thereby increasing the weight of valid information and suppressing background noise interference. Finally, by removing redundant nodes of the path aggregation network (PANet), adding additional connections, and using BlurPool instead of the downsampling method, a feature pyramid network…

Second order optimality conditions for mathematical prograramming with set functions

Jyh‐Horng Chou, Wei-Shen Hsia, Tan-Yu Lee

Numerical Analysis · The Journal of the Australian Mathematical Society Series B Applied Mathematics · 1985 · cited 22× · DOI ↗ · more like this ·

Abstract Second order necessary and sufficient conditions are given for a class of optimization problems involving optimal selection of a measurable subset from a given measure subspace subject to set function inequalities. Relations between twice-differentiability at Ω and local convexity at Ω are also discussed.

Cross-lingual Cross-modal Pretraining for Multimodal Retrieval

Hongliang Fei, Tan Yu, Ping Li

Computer Vision and Pattern Recognition · 2021 · cited 20× · DOI ↗ · more like this ·

Recent pretrained vision-language models have achieved impressive performance on cross-modal retrieval tasks in English. Their success, however, heavily depends on the availability of many annotated image-caption datasets for pretraining, where the texts are not necessarily in English. Although we can utilize machine translation (MT) tools to translate non-English text to English, the performance still largely relies on MT's quality and may suffer from high latency problems in realworld applications. This paper proposes a new approach to learn cross-lingual cross-modal representations for matching images and their relevant captions in multiple languages. We seamlessly combine cross-lingual pretraining objectives and cross-modal pretraining objectives in a unified framework to learn image and text in a joint embedding space from available English image-caption data, monolingual and…

Compressive Quantization for Fast Object Instance Search in Videos

Tan Yu, Zhenzhen Wang, Junsong Yuan

Computer Vision and Pattern Recognition · October 2017 · cited 20× · DOI ↗ · more like this ·

Most of current visual search systems focus on image-to-image (point-to-point) search such as image and object retrieval. Nevertheless, fast image-to-video (point-to-set) search is much less exploited. This paper tackles object instance search in videos, where efficient point-to-set matching is essential. Through jointly optimizing vector quantization and hashing, we propose compressive quantization method to compressM object proposals extracted from each video into only k binary codes, where k ≪ M. Then the similarity between the query object and the whole video can be determined by the Hamming distance between the queryfs binary code and the videofs best-matched binary code. Our compressive quantization not only enables fast search but also significantly reduces the memory cost of storing the video features. Despite the high compression ratio, our proposed compressive quantization…

Semi-RAID: A reliable energy-aware RAID data layout for sequential data access

Li Xiao, Tan Yu-An, Sun Zhizhuo

Computer Networks and Communications · May 2011 · cited 19× · DOI ↗ · more like this ·

In this paper, we propose an alternative RAID data layout, Semi-RAID (S-RAID), for the kind of application that exhibits a sequential data access pattern. The data layout of S-RAID uses a grouping strategy that makes only part of the whole array active and puts the rest of the array into standby mode. In this way S-RAID can greatly reduce the power consumption and improve the reliability while still satisfying the I/O requirements of the application. We have shown that transforming a traditional RAID 5 into S-RAID 5 makes average mean time to failure (MTTF) 8 times as long as before, and reduces the power consumption to 74% in a 5-disk configuration. In a typical video surveillance system that has 25 disks, S-RAID may reduce the power consumption to nearly 15% in the best case. The performance of S-RAID is evaluated and proved to be sufficient for the workloads it is optimized for.…

3D Object Representation Learning: A Set-to-Set Matching Perspective

Tan Yu, Jingjing Meng, Ming Yang +1 more

Computer Vision and Pattern Recognition · IEEE Transactions on Image Processing · 2021 · cited 17× · DOI ↗ · more like this ·

In this paper, we tackle the 3D object representation learning from the perspective of set-to-set matching. Given two 3D objects, calculating their similarity is formulated as the problem of set-to-set similarity measurement between two set of local patches. As local convolutional features from convolutional feature maps are natural representations of local patches, the set-to-set matching between sets of local patches is further converted into a local features pooling problem. To highlight good matchings and suppress the bad ones, we exploit two pooling methods: 1) bilinear pooling and 2) VLAD pooling. We analyze their effectiveness in enhancing the set-to-set matching and meanwhile establish their connection. Moreover, to balance different components inherent in a bilinear-pooled feature, we propose the harmonized bilinear pooling operation, which follows the spirits of…

Efficient Compact Bilinear Pooling via Kronecker Product

Tan Yu, Yunfeng Cai, Ping Li

Computer Vision and Pattern Recognition · Proceedings of the AAAI Conference on Artificial Intelligence · 28 Jun 2022 · cited 14× · DOI ↗ · more like this ·

Bilinear pooling has achieved excellent performance in fine-grained recognition tasks. Nevertheless, high-dimensional bilinear features suffer from over-fitting and inefficiency. To alleviate these issues, compact bilinear pooling (CBP) methods were developed to generate low-dimensional features. Although the low-dimensional features from existing CBP methods enable high efficiency in subsequent classification, CBP methods themselves are inefficient. Thus, the inefficiency issue of the bilinear pooling is still unsolved. In this work, we propose an efficient compact bilinear pooling method to solve the inefficiency problem inherited in bilinear pooling thoroughly. It decomposes the huge-scale projection matrix into a two-level Kronecker product of several small-scale matrices. By exploiting the ``vec trick'' and the tensor modal product, we can obtain the compact bilinear feature…

Inflate and Shrink:Enriching and Reducing Interactions for Fast Text-Image Retrieval

Haoliang Liu, Tan Yu, Ping Li

Computer Vision and Pattern Recognition · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · 2021 · cited 14× · DOI ↗ · more like this ·

By exploiting the cross-modal attention, cross-BERT methods have achieved state-of-the-art accuracy in cross-modal retrieval. Nevertheless, the heavy text-image interactions in the cross-BERT model are prohibitively slow for large-scale retrieval. Late-interaction methods trade off retrieval accuracy and efficiency by exploiting cross-modal interaction only in the late stage, attaining a satisfactory retrieval speed. In this work, we propose an inflating and shrinking approach to further boost the efficiency and accuracy of late-interaction methods. The inflating operation plugs several codes in the input of the encoder to exploit the text-image interactions more thoroughly for higher retrieval accuracy. Then the shrinking operation gradually reduces the text-image interactions through knowledge distilling for higher efficiency. Through an inflating operation followed by a shrinking…

Multi-modal Dictionary BERT for Cross-modal Video Search in Baidu Advertising

Tan Yu, Yi Yang, Yi Li +3 more

Computer Vision and Pattern Recognition · 26 Oct 2021 · cited 13× · DOI ↗ · more like this ·

Due to their attractiveness, video advertisements are adored by advertisers. Baidu, as one of the leading search advertisement platforms in China, is putting more and more effort into video advertisements for its advertisement customers. Search-based video advertisement display is, in essence, a cross-modal retrieval problem, which is normally tackled through joint embedding methods. Nevertheless, due to the lack of interactions between text features and image features, joint embedding methods cannot achieve as high accuracy as its counterpart based on attention. Inspired by the great success achieved by BERT in NLP tasks, many cross-modal BERT models emerge and achieve excellent performance in cross-modal retrieval. Last year, Baidu also launched a cross-modal BERT, CAN, in video advertisement platform, and achieved considerably better performance than the previous joint-embedding…