Author

Ke Yu

276 papers · 2000–2026

2000peak 26 in 20242026

Frequent co-authors

Papers

showing 1–25 of 276 RSS

ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks

Xintao Wang, Ke Yu, Shixiang Wu +6 more

cs.CV · 17 Sep 2018 · cited 4,915× · arXiv ↗ · more like this ·

The Super-Resolution Generative Adversarial Network (SRGAN) is a seminal work that is capable of generating realistic textures during single image super-resolution. However, the hallucinated details are often accompanied with unpleasant artifacts. To further enhance the visual quality, we thoroughly study three key components of SRGAN - network architecture, adversarial loss and perceptual loss, and improve each of them to derive an Enhanced SRGAN (ESRGAN). In particular, we introduce the Residual-in-Residual Dense Block (RRDB) without batch normalization as the basic network building unit. Moreover, we borrow the idea from relativistic GAN to let the discriminator predict relative realness instead of the absolute value. Finally, we improve the perceptual loss by using the features before activation, which could provide stronger supervision for brightness consistency and texture…

EDVR: Video Restoration with Enhanced Deformable Convolutional Networks

Xintao Wang, Kelvin C. K. Chan, Ke Yu +2 more

cs.CV · 7 May 2019 · cited 1,336× · arXiv ↗ · more like this ·

Video restoration tasks, including super-resolution, deblurring, etc, are drawing increasing attention in the computer vision community. A challenging benchmark named REDS is released in the NTIRE19 Challenge. This new benchmark challenges existing methods from two aspects: (1) how to align multiple frames given large motions, and (2) how to effectively fuse different frames with diverse motion and blur. In this work, we propose a novel Video Restoration framework with Enhanced Deformable networks, termed EDVR, to address these challenges. First, to handle large motions, we devise a Pyramid, Cascading and Deformable (PCD) alignment module, in which frame alignment is done at the feature level using deformable convolutions in a coarse-to-fine manner. Second, we propose a Temporal and Spatial Attention (TSA) fusion module, in which attention is applied both temporally and spatially, so as…

Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform

Xintao Wang, Ke Yu, Chao Dong +1 more

cs.CV · 9 Apr 2018 · cited 1,217× · arXiv ↗ · more like this ·

Despite that convolutional neural networks (CNN) have recently demonstrated high-quality reconstruction for single-image super-resolution (SR), recovering natural and realistic texture remains a challenging problem. In this paper, we show that it is possible to recover textures faithful to semantic classes. In particular, we only need to modulate features of a few intermediate layers in a single network conditioned on semantic segmentation probability maps. This is made possible through a novel Spatial Feature Transform (SFT) layer that generates affine transformation parameters for spatial-wise feature modulation. SFT layers can be trained end-to-end together with the SR network using the same loss function. During testing, it accepts an input image of arbitrary size and generates a high-resolution image with just a single forward pass conditioned on the categorical priors. Our final…

BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond

Kelvin C. K. Chan, Xintao Wang, Ke Yu +2 more

cs.CV · December 2020 · cited 693× · arXiv ↗ · more like this ·

Video super-resolution (VSR) approaches tend to have more components than the image counterparts as they need to exploit the additional temporal dimension. Complex designs are not uncommon. In this study, we wish to untangle the knots and reconsider some most essential components for VSR guided by four basic functionalities, i.e., Propagation, Alignment, Aggregation, and Upsampling. By reusing some existing components added with minimal redesigns, we show a succinct pipeline, BasicVSR, that achieves appealing improvements in terms of speed and restoration quality in comparison to many state-of-the-art algorithms. We conduct systematic analysis to explain how such gain can be obtained and discuss the pitfalls. We further show the extensibility of BasicVSR by presenting an information-refill mechanism and a coupled propagation scheme to facilitate information aggregation. The BasicVSR and…

Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning

Ke Yu, Chao Dong, Liang Lin +1 more

cs.CV · 10 Apr 2018 · cited 202× · arXiv ↗ · more like this ·

We investigate a novel approach for image restoration by reinforcement learning. Unlike existing studies that mostly train a single large network for a specialized task, we prepare a toolbox consisting of small-scale convolutional networks of different complexities and specialized in different tasks. Our method, RL-Restore, then learns a policy to select appropriate tools from the toolbox to progressively restore the quality of a corrupted image. We formulate a step-wise reward function proportional to how well the image is restored at each step to learn the action policy. We also devise a joint learning scheme to train the agent and tools for better performance in handling uncertainty. In comparison to conventional human-designed networks, RL-Restore is capable of restoring images corrupted with complex and unknown distortions in a more parameter-efficient manner using the dynamically…

Understanding Deformable Alignment in Video Super-Resolution

Kelvin C. K. Chan, Xintao Wang, Ke Yu +2 more

cs.CV · 15 Sep 2020 · cited 195× · arXiv ↗ · more like this ·

Deformable convolution, originally proposed for the adaptation to geometric variations of objects, has recently shown compelling performance in aligning multiple frames and is increasingly adopted for video super-resolution. Despite its remarkable performance, its underlying mechanism for alignment remains unclear. In this study, we carefully investigate the relation between deformable alignment and the classic flow-based alignment. We show that deformable convolution can be decomposed into a combination of spatial warping and convolution. This decomposition reveals the commonality of deformable alignment and flow-based alignment in formulation, but with a key difference in their offset diversity. We further demonstrate through experiments that the increased diversity in deformable alignment yields better-aligned features, and hence significantly improves the quality of video…

Can contrastive learning avoid shortcut solutions?

Joshua Robinson, Li Sun, Ke Yu +3 more

cs.LG · June 2021 · cited 185× · arXiv ↗ · more like this ·

The generalization of representations learned via contrastive learning depends crucially on what features of the data are extracted. However, we observe that the contrastive loss does not always sufficiently guide which features are extracted, a behavior that can negatively impact the performance on downstream tasks via "shortcuts", i.e., by inadvertently suppressing important predictive features. We find that feature extraction is influenced by the difficulty of the so-called instance discrimination task (i.e., the task of discriminating pairs of similar points from pairs of dissimilar ones). Although harder pairs improve the representation of some features, the improvement comes at the cost of suppressing previously well represented features. In response, we propose implicit feature modification (IFM), a method for altering positive and negative samples in order to guide contrastive…

uTrack

Ke-Yu Chen, Kent Lyons, Sean White +1 more

Human-Computer Interaction · 8 Oct 2013 · cited 170× · DOI ↗ · more like this ·

While much progress has been made in wearable computing in recent years, input techniques remain a key challenge. In this paper, we introduce uTrack, a technique to convert the thumb and fingers into a 3D input system using magnetic field (MF) sensing. A user wears a pair of magnetometers on the back of their fingers and a permanent magnet affixed to the back of the thumb. By moving the thumb across the fingers, we obtain a continuous input stream that can be used for 3D pointing. Specifically, our novel algorithm calculates the magnet's 3D position and tilt angle directly from the sensor readings. We evaluated uTrack as an input device, showing an average tracking accuracy of 4.84 mm in 3D space - sufficient for subtle interaction. We also demonstrate a real-time prototype and example applications allowing users to interact with the computer using 3D finger input.

Finexus

Ke-Yu Chen, Shwetak Patel, Sean Keller

Human-Computer Interaction · 5 May 2016 · cited 150× · DOI ↗ · more like this ·

With the resurgence of head-mounted displays for virtual reality, users need new input devices that can accurately track their hands and fingers in motion. We introduce Finexus, a multipoint tracking system using magnetic field sensing. By instrumenting the fingertips with electromagnets, the system can track fine fingertip movements in real time using only four magnetic sensors. To keep the system robust to noise, we operate each electromagnet at a different frequency and leverage bandpass filters to distinguish signals attributed to individual sensing points. We develop a novel algorithm to efficiently calculate the 3D positions of multiple electromagnets from corresponding field strengths. In our evaluation, we report an average accuracy of 1.33 mm, as compared to results from an optical tracker. Our real-time implementation shows Finexus is applicable to a wide variety of human…

Hierarchical Amortized Training for Memory-efficient High Resolution 3D GAN

Li Sun, Junxiang Chen, Yanwu Xu +3 more

eess.IV · August 2020 · cited 137× · arXiv ↗ · more like this ·

Generative Adversarial Networks (GAN) have many potential medical imaging applications, including data augmentation, domain adaptation, and model explanation. Due to the limited memory of Graphical Processing Units (GPUs), most current 3D GAN models are trained on low-resolution medical images, these models either cannot scale to high-resolution or are prone to patchy artifacts. In this work, we propose a novel end-to-end GAN architecture that can generate high-resolution 3D images. We achieve this goal by using different configurations between training and inference. During training, we adopt a hierarchical structure that simultaneously generates a low-resolution version of the image and a randomly selected sub-volume of the high-resolution image. The hierarchical design has two advantages: First, the memory demand for training on high-resolution images is amortized among sub-volumes.…

Stability criteria for uncertain Takagi–Sugeno fuzzy systems with interval time-varying delay

Chang‐Hua Lien, Ke Yu, W.D. Chen +2 more

Computer Networks and Communications · IET Control Theory and Applications · 8 May 2007 · cited 121× · DOI ↗ · more like this ·

The asymptotic stability for Takagi–Sugeno (T–S) fuzzy systems with interval time-varying delay and parameter uncertainties are investigated. Delay-dependent and delay-independent criteria are proposed to guarantee the asymptotic stability for the uncertain fuzzy systems with interval time-varying delay. Numerical examples are provided to show that the obtained results significantly improve the allowed bounds of time-varying delay over some existing ones in the literature.

Path-Restore: Learning Network Path Selection for Image Restoration

Ke Yu, Xintao Wang, Chao Dong +2 more

cs.CV · April 2019 · cited 115× · arXiv ↗ · more like this ·

Very deep Convolutional Neural Networks (CNNs) have greatly improved the performance on various image restoration tasks. However, this comes at a price of increasing computational burden, hence limiting their practical usages. We observe that some corrupted image regions are inherently easier to restore than others since the distortion and content vary within an image. To leverage this, we propose Path-Restore, a multi-path CNN with a pathfinder that can dynamically select an appropriate route for each image region. We train the pathfinder using reinforcement learning with a difficulty-regulated reward. This reward is related to the performance, complexity and "the difficulty of restoring a region". A policy mask is further investigated to jointly process all the image regions. We conduct experiments on denoising and mixed restoration tasks. The results show that our method achieves…

Deep Network Interpolation for Continuous Imagery Effect Transition

Xintao Wang, Ke Yu, Chao Dong +2 more

cs.CV · 26 Nov 2018 · cited 114× · arXiv ↗ · more like this ·

Deep convolutional neural network has demonstrated its capability of learning a deterministic mapping for the desired imagery effect. However, the large variety of user flavors motivates the possibility of continuous transition among different output effects. Unlike existing methods that require a specific design to achieve one particular transition (e.g., style transfer), we propose a simple yet universal approach to attain a smooth control of diverse imagery effects in many low-level vision tasks, including image restoration, image-to-image translation, and style transfer. Specifically, our method, namely Deep Network Interpolation (DNI), applies linear interpolation in the parameter space of two or more correlated networks. A smooth control of imagery effects can be achieved by tweaking the interpolation coefficients. In addition to DNI and its broad applications, we also investigate…

AirLink

Ke-Yu Chen, Daniel Ashbrook, Mayank Goel +2 more

Human-Computer Interaction · 13 Sep 2014 · cited 97× · DOI ↗ · more like this ·

We introduce AirLink, a novel technique for sharing files between multiple devices. By waving a hand from one device towards another, users can directly transfer files between them. The system utilizes the devices' built-in speakers and microphones to enable easy file sharing between phones, tablets and laptops. We evaluate our system in an 11-participant study with 96.8% accuracy, showing the feasibility of using AirLink in a multiple-device environment. We also implemented a real-time system and demonstrate the capability of AirLink in various applications.

Deep Convolution Networks for Compression Artifacts Reduction

Ke Yu, Chao Dong, Chen Change Loy +1 more

cs.CV · 9 Aug 2016 · cited 87× · arXiv ↗ · more like this ·

Lossy compression introduces complex compression artifacts, particularly blocking artifacts, ringing effects and blurring. Existing algorithms either focus on removing blocking artifacts and produce blurred output, or restore sharpened images that are accompanied with ringing effects. Inspired by the success of deep convolutional networks (DCN) on superresolution, we formulate a compact and efficient network for seamless attenuation of different compression artifacts. To meet the speed requirement of real-world applications, we further accelerate the proposed baseline model by layer decomposition and joint use of large-stride convolutional and deconvolutional layers. This also leads to a more general CNN framework that has a close relationship with the conventional Multi-Layer Perceptron (MLP). Finally, the modified network achieves a speed up of 7.5 times with almost no performance…

DPNM: A Differential Private Notary Mechanism for Privacy Preservation in Cross-Chain Transactions

Kai Zhang, Pei‐Wei Tsai, Jiao Tian +6 more

Artificial Intelligence · IEEE Transactions on Information Forensics and Security · 2025 · cited 65× · DOI ↗ · more like this ·

Notary cross-chain transaction technologies have obtained broad affirmation from industry and academia as they can avoid data islands and enhance chain interoperability. However, the increased privacy concern in data sharing makes the participants hesitate to upload sensitive information without the trust foundation of the external network. To address this issue, this paper proposes a differential private notary mechanism (DPNM) to preserve privacy in blockchain interoperations. It establishes a fully trusted notary organization to conduct data perturbation before replying query to the external blockchain network. In addition, the DPNM contains two built-in privacy budget allocation schemes: Efficiency priority scheme (EPS) and Privacy priority scheme (PPS). These schemes unify the privacy preferences among different nodes based on multi-node consensus in the decentralized environment.…

SideSwipe

Chen Zhao, Ke-Yu Chen, Md Tanvir Islam Aumi +2 more

Human-Computer Interaction · October 2014 · cited 63× · DOI ↗ · more like this ·

Current smartphone inputs are limited to physical buttons, touchscreens, cameras or built-in sensors. These approaches either require a dedicated surface or line-of-sight for interaction. We introduce SideSwipe, a novel system that enables in-air gestures both above and around a mobile device. Our system leverages the actual GSM signal to detect hand gestures around the device. We developed an algorithm to convert the discrete and bursty GSM pulses to a continuous wave that can be used for gesture recognition. Specifically, when a user waves their hand near the phone, the hand movement disturbs the signal propagation between the phone's transmitter and added receiving antennas. Our system captures this variation and uses it for gesture recognition. To evaluate our system, we conduct a study with 10 participants and present robust gesture recognition with an average accuracy of 87.2%…

ReconfigISP: Reconfigurable Camera Image Processing Pipeline

Ke Yu, Zexian Li, Yue Peng +2 more

eess.IV · 10 Sep 2021 · cited 56× · arXiv ↗ · more like this ·

Image Signal Processor (ISP) is a crucial component in digital cameras that transforms sensor signals into images for us to perceive and understand. Existing ISP designs always adopt a fixed architecture, e.g., several sequential modules connected in a rigid order. Such a fixed ISP architecture may be suboptimal for real-world applications, where camera sensors, scenes and tasks are diverse. In this study, we propose a novel Reconfigurable ISP (ReconfigISP) whose architecture and parameters can be automatically tailored to specific data and tasks. In particular, we implement several ISP modules, and enable backpropagation for each module by training a differentiable proxy, hence allowing us to leverage the popular differentiable neural architecture search and effectively search for the optimal ISP architecture. A proxy tuning mechanism is adopted to maintain the accuracy of proxy…

Analysis and Detection of Health-Related Misinformation on Chinese Social Media

Yue Liu, Ke Yu, Xiaofei Wu +2 more

Sociology and Political Science · IEEE Access · 2019 · cited 51× · DOI ↗ · more like this ·

With the mobile Internet development, e-health has become increasingly connected with people's daily life. However, health information on Internet is severely corrupted by misinformation, especially for the aged. It is necessary to analyze the characteristics of health-related misinformation on Internet and to design automated detection tools. In this study, we focus on analyzing common characteristics of reliable and unreliable health-related information on Chinese online social media, and exploring possible detection method using machine learning algorithms. We first collect a dataset containing both reliable and unreliable health-related articles from multiple Chinese online social media sites, with 2,296 reliable and 2,085 unreliable included. Then we analyze their differences with respect to writing style, text topic and feature distribution by both intuitive and statistical…

Monitoring ICU Mortality Risk with A Long Short-Term Memory Recurrent Neural Network

Ke Yu, Mingda Zhang, Tianyi Cui +1 more

Artificial Intelligence · PubMed · 27 Nov 2019 · cited 44× · DOI ↗ · more like this ·

In intensive care units (ICU), mortality prediction is a critical factor not only for effective medical intervention but also for allocation of clinical resources. Structured electronic health records (EHR) contain valuable information for assessing mortality risk in ICU patients, but current mortality prediction models usually require laborious human-engineered features. Furthermore, substantial missing data in EHR is a common problem for both the construction and implementation of a prediction model.Inspired by language-related models, we design a new framework for dynamic monitoring of patients' mortality risk. Our framework uses the bag-of-words representation for all relevant medical events based on most recent history as inputs. By design, it is robust to missing data in EHR and can be easily implemented as an instant scoring system to monitor the medical development of all ICU…

Concept-Guided Prompt Learning for Generalization in Vision-Language Models

Yi Zhang, Ce Zhang, Ke Yu +2 more

cs.CV · 15 Jan 2024 · cited 43× · arXiv ↗ · more like this ·

Contrastive Language-Image Pretraining (CLIP) model has exhibited remarkable efficacy in establishing cross-modal connections between texts and images, yielding impressive performance across a broad spectrum of downstream applications through fine-tuning. However, for generalization tasks, the current fine-tuning methods for CLIP, such as CoOp and CoCoOp, demonstrate relatively low performance on some fine-grained datasets. We recognize the underlying reason is that these previous methods only projected global features into the prompt, neglecting the various visual concepts, such as colors, shapes, and sizes, which are naturally transferable across domains and play a crucial role in generalization tasks. To address this issue, in this work, we propose Concept-Guided Prompt Learning (CPL) for vision-language models. Specifically, we leverage the well-learned knowledge of CLIP to create a…

NTIRE 2019 Challenge on Video Deblurring: Methods and Results

Seungjun Nah, Radu Timofte, Sungyong Baik +9 more

Computer Vision and Pattern Recognition · Scholarworks@UNIST (Ulsan National Institute of Science and Technology) · June 2019 · cited 41× · DOI ↗ · more like this ·

This paper reviews the first NTIRE challenge on video deblurring (restoration of rich details and high frequency components from blurred video frames) with focus on the proposed solutions and results. A new REalistic and Diverse Scenes dataset (REDS) was employed. The challenge was divided into 2 tracks. Track 1 employed dynamic motion blurs while Track 2 had additional MPEG video compression artifacts. Each competition had 109 and 93 registered participants. Total 13 teams competed in the final testing phase. They gauge the state-of-the-art in video deblurring problem.

Data Fusion Oriented Graph Convolution Network Model for Rumor Detection

Ke Yu, Hao Jiang, Ting Li +2 more

Sociology and Political Science · IEEE Transactions on Network and Service Management · 27 Oct 2020 · cited 40× · DOI ↗ · more like this ·

With the rapid development of Mobile Internet, social media platforms explosively grow and greatly facilitate people to obtain and exchange information. Since any users can post arbitrary information on social media, some people with ulterior motives try to create and spread various misinformation for their own benefits, which might damage citizen rights and disturb social order. In this article, we propose a novel rumor detection model based on Graph Convolution Networks (GCNs). Based on the real dataset from social media, we consider not only the static features such as users's basic information and text contents, but also the dynamic features such as rumor propagation relations. The GCN-based model is used to represent the spreading structure of rumors, with graph convolution operator for node vector updating. We also optimize the feature fusion module and pooling module to make our…

Semi-Supervised Hierarchical Drug Embedding in Hyperbolic Space

Ke Yu, Shyam Visweswaran, Kayhan Batmanghelich

cs.LG · June 2020 · cited 39× · arXiv ↗ · more like this ·

Learning accurate drug representation is essential for tasks such as computational drug repositioning and prediction of drug side-effects. A drug hierarchy is a valuable source that encodes human knowledge of drug relations in a tree-like structure where drugs that act on the same organs, treat the same disease, or bind to the same biological target are grouped together. However, its utility in learning drug representations has not yet been explored, and currently described drug representations cannot place novel molecules in a drug hierarchy. Here, we develop a semi-supervised drug embedding that incorporates two sources of information: (1) underlying chemical grammar that is inferred from molecular structures of drugs and drug-like molecules (unsupervised), and (2) hierarchical relations that are encoded in an expert-crafted hierarchy of approved drugs (supervised). We use the…

NTIRE 2019 Challenge on Video Super-Resolution: Methods and Results

Seungjun Nah, Radu Timofte, Shuhang Gu +9 more

Computer Vision and Pattern Recognition · Digital Collections portal (Koç University) · June 2019 · cited 39× · DOI ↗ · more like this ·

This paper reviews the first NTIRE challenge on video super-resolution (restoration of rich details in low-resolution video frames) with focus on proposed solutions and results. A new REalistic and Diverse Scenes dataset (REDS) was employed. The challenge was divided into 2 tracks. Track 1 employed standard bicubic downscaling setup while Track 2 had realistic dynamic motion blurs. Each competition had 124 and 104 registered participants. There were total 14 teams in the final testing phase. They gauge the state-of-the-art in video super-resolution.