arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Chong Mou

31 papers · 2016–2026

2016: 22017: 02018: 02019: 02020: 12021: 72022: 42023: 62024: 72025: 32026: 1
2016peak 7 in 20212026

Frequent co-authors

Papers

showing 1–25 of 31 RSS

T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models

Chong Mou, Xintao Wang, Liangbin Xie +5 more

cs.CV · · cited 1,892× · arXiv ↗ · more like this ·

The incredible generative ability of large-scale text-to-image (T2I) models has demonstrated strong power of learning complex structures and meaningful semantics. However, relying solely on text prompts cannot fully take advantage of the knowledge learned by the model, especially when flexible and accurate controlling (e.g., color and structure) is needed. In this paper, we aim to ``dig out" the capabilities that T2I models have implicitly learned, and then explicitly use them to control the generation more granularly. Specifically, we propose to learn simple and lightweight T2I-Adapters to align internal knowledge in T2I models with external control signals, while freezing the original large T2I models. In this way, we can train various adapters according to different conditions, achieving rich control and editing effects in the color and structure of the generation results. Further…

Deep Generalized Unfolding Networks for Image Restoration

Chong Mou, Qian Wang, Jian Zhang

cs.CV · · cited 334× · arXiv ↗ · more like this ·

Deep neural networks (DNN) have achieved great success in image restoration. However, most DNN methods are designed as a black box, lacking transparency and interpretability. Although some methods are proposed to combine traditional optimization algorithms with DNN, they usually demand pre-defined degradation processes or handcrafted assumptions, making it difficult to deal with complex and real-world applications. In this paper, we propose a Deep Generalized Unfolding Network (DGUNet) for image restoration. Concretely, without loss of interpretability, we integrate a gradient estimation strategy into the gradient descent step of the Proximal Gradient Descent (PGD) algorithm, driving it to deal with complex and real-world image degradation. In addition, we design inter-stage information pathways across proximal mapping in different PGD iterations to rectify the intrinsic information…

DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Models

Chong Mou, Xintao Wang, Jiechong Song +2 more

cs.CV · · cited 241× · arXiv ↗ · more like this ·

Despite the ability of existing large-scale text-to-image (T2I) models to generate high-quality images from detailed textual descriptions, they often lack the ability to precisely edit the generated or real images. In this paper, we propose a novel image editing method, DragonDiffusion, enabling Drag-style manipulation on Diffusion models. Specifically, we construct classifier guidance based on the strong correspondence of intermediate features in the diffusion model. It can transform the editing signals into gradients via feature correspondence loss to modify the intermediate representation of the diffusion model. Based on this guidance strategy, we also build a multi-scale guidance to consider both semantic and geometric alignment. Moreover, a cross-branch self-attention is added to maintain the consistency between the original image and the editing result. Our method, through an…

Robust Invertible Image Steganography

Youmin Xu, Chong Mou, Yujie Hu +2 more

Computer Vision and Pattern Recognition · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · · cited 162× · DOI ↗ · more like this ·

Image steganography aims to hide secret images into a container image, where the secret is hidden from human vision and can be restored when necessary. Previous image steganography methods are limited in hiding capacity and robustness, commonly vulnerable to distortion on container images such as Gaussian noise, Poisson noise, and lossy compression. This paper presents a novelflow-basedframe-work for robust invertible image steganography, dubbed as RIIS. A conditional normalizing flow is introduced to model the distribution of the redundant high-frequency component with the condition of the container image. Moreover, a well-designed container enhancement module (CEM) also contributes to the robust reconstruction. To regulate the net-work parameters for different distortion levels, a distortion-guided modulation (DGM) is implemented over flow-based blocks to make it a one-size-fits-all…

DiffEditor: Boosting Accuracy and Flexibility on Diffusion-based Image Editing

Chong Mou, Xintao Wang, Jiechong Song +2 more

cs.CV · · cited 112× · arXiv ↗ · more like this ·

Large-scale Text-to-Image (T2I) diffusion models have revolutionized image generation over the last few years. Although owning diverse and high-quality generation capabilities, translating these abilities to fine-grained image editing remains challenging. In this paper, we propose DiffEditor to rectify two weaknesses in existing diffusion-based image editing: (1) in complex scenarios, editing results often lack editing accuracy and exhibit unexpected artifacts; (2) lack of flexibility to harmonize editing operations, e.g., imagine new content. In our solution, we introduce image prompts in fine-grained image editing, cooperating with the text prompt to better describe the editing content. To increase the flexibility while maintaining content consistency, we locally combine stochastic differential equation (SDE) into the ordinary differential equation (ODE) sampling. In addition, we…

COLA-Net: Collaborative Attention Network for Image Restoration

Chong Mou, Jian Zhang, Xiaopeng Fan +2 more

cs.CV · · cited 96× · arXiv ↗ · more like this ·

Local and non-local attention-based methods have been well studied in various image restoration tasks while leading to promising performance. However, most of the existing methods solely focus on one type of attention mechanism (local or non-local). Furthermore, by exploiting the self-similarity of natural images, existing pixel-wise non-local attention operations tend to give rise to deviations in the process of characterizing long-range dependence due to image degeneration. To overcome these problems, in this paper we propose a novel collaborative attention network (COLA-Net) for image restoration, as the first attempt to combine local and non-local attention mechanisms to restore image content in the areas with complex textures and with highly repetitive details respectively. In addition, an effective and robust patch-wise non-local attention model is developed to capture long-range…

360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Model

Qian Wang, Weiqi Li, Chong Mou +2 more

cs.CV · · cited 76× · arXiv ↗ · more like this ·

Panorama video recently attracts more interest in both study and application, courtesy of its immersive experience. Due to the expensive cost of capturing 360-degree panoramic videos, generating desirable panorama videos by prompts is urgently required. Lately, the emerging text-to-video (T2V) diffusion methods demonstrate notable effectiveness in standard video generation. However, due to the significant gap in content and motion patterns between panoramic and standard videos, these methods encounter challenges in yielding satisfactory 360-degree panoramic videos. In this paper, we propose a pipeline named 360-Degree Video Diffusion model (360DVD) for generating 360-degree panoramic videos based on the given prompts and motion conditions. Specifically, we introduce a lightweight 360-Adapter accompanied by 360 Enhancement Techniques to transform pre-trained T2V models for panorama video…

Large-capacity and Flexible Video Steganography via Invertible Neural Network

Chong Mou, Youmin Xu, Jiechong Song +3 more

cs.CV · · cited 72× · arXiv ↗ · more like this ·

Video steganography is the art of unobtrusively concealing secret data in a cover video and then recovering the secret data through a decoding protocol at the receiver end. Although several attempts have been made, most of them are limited to low-capacity and fixed steganography. To rectify these weaknesses, we propose a Large-capacity and Flexible Video Steganography Network (LF-VSN) in this paper. For large-capacity, we present a reversible pipeline to perform multiple videos hiding and recovering through a single invertible neural network (INN). Our method can hide/recover 7 secret videos in/from 1 cover video with promising performance. For flexibility, we propose a key-controllable scheme, enabling different receivers to recover particular secret videos from the same cover video through specific keys. Moreover, we further improve the flexibility by proposing a scalable strategy in…

NTIRE 2021 Challenge on Quality Enhancement of Compressed Video: Methods and Results

Ren Yang, Radu Timofte, Jing Liu +69 more

eess.IV · · cited 43× · arXiv ↗ · more like this ·

This paper reviews the first NTIRE challenge on quality enhancement of compressed video, with a focus on the proposed methods and results. In this challenge, the new Large-scale Diverse Video (LDV) dataset is employed. The challenge has three tracks. Tracks 1 and 2 aim at enhancing the videos compressed by HEVC at a fixed QP, while Track 3 is designed for enhancing the videos compressed by x265 at a fixed bit-rate. Besides, the quality enhancement of Tracks 1 and 3 targets at improving the fidelity (PSNR), and Track 2 targets at enhancing the perceptual quality. The three tracks totally attract 482 registrations. In the test phase, 12 teams, 8 teams and 11 teams submitted the final results of Tracks 1, 2 and 3, respectively. The proposed methods and solutions gauge the state-of-the-art of video quality enhancement. The homepage of the challenge…

NTIRE 2023 Challenge on 360° Omnidirectional Image and Video Super-Resolution: Datasets, Methods and Results

Mingdeng Cao, Chong Mou, Fanghua Yu +9 more

Computer Vision and Pattern Recognition · · cited 32× · DOI ↗ · more like this ·

This report introduces two high-quality datasets Flickr360 and ODV360 for omnidirectional image and video super-resolution, respectively, and reports the NTIRE 2023 challenge on 360° omnidirectional image and video super-resolution. Unlike ordinary 2D images/videos with a narrow field of view, omnidirectional images/videos can represent the whole scene from all directions in one shot. There exists a large gap between omnidirectional image/video and ordinary 2D image/video in both the degradation and restoration processes. The challenge is held to facilitate the development of omnidirectional image/video super-resolution by considering their special characteristics. In this challenge, two tracks are provided: one is the omnidirectional image super-resolution and the other is the omnidirectional video super-resolution. The task of the challenge is to super-resolve an input omnidirectional…

Graph Attention Neural Network for Image Restoration

Chong Mou, Jian Zhang

Computer Vision and Pattern Recognition · · cited 11× · DOI ↗ · more like this ·

Self-similarity underpins modern non-local attention mechanism, which has been verified to be an effective prior for image restoration. However, most existing non-local attention restorers are implemented based on pixels, which tend to be biased due to image degeneration. Furthermore, most non-local methods for image restoration are restricted to construct fully-connected correlations in a regular Euclidean space so that all features within the search region have to participate in the feature aggregation process, no matter how similar the key feature is to the query feature. To rectify these weaknesses, in this paper, we propose a novel graph attention network for image restoration, dubbed GATIR, which establishes the non-local attention based on feature patches and utilizes the graph convolution to perform feature aggregation selectively in a non-Euclidean space. Experimental results…

Dynamic Attentive Graph Learning for Image Restoration

Chong Mou, Jian Zhang, Zhuoyuan Wu

cs.CV · · cited 6× · arXiv ↗ · more like this ·

Non-local self-similarity in natural images has been verified to be an effective prior for image restoration. However, most existing deep non-local methods assign a fixed number of neighbors for each query item, neglecting the dynamics of non-local correlations. Moreover, the non-local correlations are usually based on pixels, prone to be biased due to image degradation. To rectify these weaknesses, in this paper, we propose a dynamic attentive graph learning model (DAGL) to explore the dynamic non-local property on patch level for image restoration. Specifically, we propose an improved graph model to perform patch-wise graph convolution with a dynamic and adaptive number of neighbors for each node. In this way, image content can adaptively balance over-smooth and over-sharp artifacts through the number of its connected neighbors, and the patch-wise non-local correlations can enhance…

Image Denoising Based on Correlation Adaptive Sparse Modeling

Hangfan Liu, Jian Zhang, Chong Mou

Computer Vision and Pattern Recognition · · cited 5× · DOI ↗ · more like this ·

Image restoration techniques generally use intrinsic correlations of image signals to reduce the uncertainty of the unknown signal and estimate the latent ground truth. Local and non-local correlation are the two major kinds of correlations utilized. They are different sources of correlations reflecting connections between different image data, but such a difference is not taken into consideration in most of the existing schemes. Typically, sparse representation based works use the same image data to exploit both local and non-local correlation in shared regularization. This paper aims to fully exploit local and non-local correlation of image contents separately so that near-optimal sparse representations are achieved and thus the uncertainty of signals is minimized. The proposed scheme adaptively selects different image data to exploit local and non-local correlations respectively. In…

Attention Based Dual Branches Fingertip Detection Network and Virtual Key System

Chong Mou, Xin Zhang

Human-Computer Interaction · · cited 4× · DOI ↗ · more like this ·

Gesture and fingertip are becoming more and more important mediums for human-computer interaction (HCI). Therefore, algorithms of gesture recognition and fingertip detection have been extensively investigated. However, problems mainly remain in how to achieve a win-win situation between speed and accuracy, and how to deal with complex interaction environment. To rectify these problems, this paper proposes an attention-based dual branches network that can efficiently fulfill both fingertip detection and gesture recognition tasks. In order to deal with complex interaction environment, we combine both channel-wise attention and spatial-wise attention into the fingertip detection model. The extensive experiments demonstrate that our novel model is both effective and efficient. In the experiment, our proposed model achieves the average fingertip detection error at around 2.8 pixels in…

Empowering Real-World Image Super-Resolution With Flexible Interactive Modulation

Chong Mou, Xintao Wang, Yanze Wu +2 more

Computer Vision and Pattern Recognition · IEEE Transactions on Pattern Analysis and Machine Intelligence · · cited 2× · DOI ↗ · more like this ·

Interactive image restoration aims to construct an interactive pathway between users and restoration networks, which empowers users to modulate the restoration results according to their own demands. However, existing methods are primarily limited to training their networks with predefined and simplistic synthetic degradations. Consequently, these methods often encounter significant performance degradation when confronted with real-world degradations that deviate from their assumptions. Furthermore, existing interactive image restoration approaches solely support global modulation, wherein a single modulation factor governs the reconstruction process for the entire image. In this paper, we propose a novel method to perform real-world and intricate image super-resolution in an interactive manner. Specifically, we propose a metric-learning-based degradation estimation strategy to estimate…

Synergic Feature Attention for Image Restoration

Chong Mou, Jian Zhang

Computer Vision and Pattern Recognition · · cited 2× · DOI ↗ · more like this ·

Local and non-local attentions are both effective methods in the domain of image restoration (IR). However, most existing image restoration methods use these two strategies indiscriminately, and how to make a trade-off between local and non-local attention operations has hardly been studied. Furthermore, the commonly used pixel-wise non-local operation tends to be biased during image restoration due to the image degeneration. To overcome these problems, in this paper, we propose a novel Synergic Attention Network (SAT-Net) for image restoration as an inventive attempt to combine local and non-local attention mechanisms to restore complex textures and highly repetitive details distinguishingly. We also propose an effective patch-wise non-local attention method to establish more reliable long-range dependences based on 3D patches. Experimental results on synthetic image denoising, real…

ReVideo: Remake a Video with Motion and Content Control

Chong Mou, Mingdeng Cao, Xintao Wang +3 more

cs.CV · · cited 1× · arXiv ↗ · more like this ·

Despite significant advancements in video generation and editing using diffusion models, achieving accurate and localized video editing remains a substantial challenge. Additionally, most existing video editing methods primarily focus on altering visual content, with limited research dedicated to motion editing. In this paper, we present a novel attempt to Remake a Video (ReVideo) which stands out from existing methods by allowing precise video editing in specific areas through the specification of both content and motion. Content editing is facilitated by modifying the first frame, while the trajectory-based motion control offers an intuitive user interaction experience. ReVideo addresses a new task involving the coupling and training imbalance between content and motion control. To tackle this, we develop a three-stage training strategy that progressively decouples these two aspects…

Diffusion-Based Hierarchical Image Steganography

Youmin Xu, Xuanyu Zhang, Jiwen Yu +3 more

cs.CV · · cited 1× · arXiv ↗ · more like this ·

This paper introduces Hierarchical Image Steganography, a novel method that enhances the security and capacity of embedding multiple images into a single container using diffusion models. HIS assigns varying levels of robustness to images based on their importance, ensuring enhanced protection against manipulation. It adaptively exploits the robustness of the Diffusion Model alongside the reversibility of the Flow Model. The integration of Embed-Flow and Enhance-Flow improves embedding efficiency and image recovery quality, respectively, setting HIS apart from conventional multi-image steganography techniques. This innovative structure can autonomously generate a container image, thereby securely and efficiently concealing multiple images and text. Rigorous subjective and objective evaluations underscore our advantage in analytical resistance, robustness, and capacity, illustrating its…

MM-RealSR: Metric Learning based Interactive Modulation for Real-World Super-Resolution

Chong Mou, Yanze Wu, Xintao Wang +3 more

cs.CV · · cited 1× · arXiv ↗ · more like this ·

Interactive image restoration aims to restore images by adjusting several controlling coefficients, which determine the restoration strength. Existing methods are restricted in learning the controllable functions under the supervision of known degradation types and levels. They usually suffer from a severe performance drop when the real degradation is different from their assumptions. Such a limitation is due to the complexity of real-world degradations, which can not provide explicit supervision to the interactive modulation during training. However, how to realize the interactive modulation in real-world super-resolution has not yet been studied. In this work, we present a Metric Learning based Interactive Modulation for Real-World Super-Resolution (MM-RealSR). Specifically, we propose an unsupervised degradation estimation strategy to estimate the degradation level in real-world…

InstructX: Towards Unified Visual Editing with MLLM Guidance

Chong Mou, Qichao Sun, Yanze Wu +5 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

With recent advances in Multimodal Large Language Models (MLLMs) showing strong visual understanding and reasoning, interest is growing in using them to improve the editing performance of diffusion models. Despite rapid progress, most studies lack an in-depth analysis of MLLM design choices. Moreover, the integration of MLLMs and diffusion models remains an open challenge in some difficult tasks, such as video editing. In this paper, we present InstructX, a unified framework for image and video editing. Specifically, we conduct a comprehensive study on integrating MLLMs and diffusion models for instruction-driven editing across diverse tasks. Building on this study, we analyze the cooperation and distinction between images and videos in unified modeling. (1) We show that training on image data can lead to emergent video editing capabilities without explicit supervision, thereby…

DreamO: A Unified Framework for Image Customization

Chong Mou, Yanze Wu, Wenxu Wu +15 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

Recently, extensive research on image customization (e.g., identity, subject, style, background, etc.) demonstrates strong customization capabilities in large-scale generative models. However, most approaches are designed for specific tasks, restricting their generalizability to combine different types of condition. Developing a unified framework for image customization remains an open challenge. In this paper, we present DreamO, an image customization framework designed to support a wide range of tasks while facilitating seamless integration of multiple conditions. Specifically, DreamO utilizes a diffusion transformer (DiT) framework to uniformly process input of different types. During training, we construct a large-scale training dataset that includes various customization tasks, and we introduce a feature routing constraint to facilitate the precise querying of relevant information…

Consistent Human Image and Video Generation with Spatially Conditioned Diffusion

Mingdeng Cao, Chong Mou, Ziyang Yuan +4 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

Consistent human-centric image and video synthesis aims to generate images or videos with new poses while preserving appearance consistency with a given reference image, which is crucial for low-cost visual content creation. Recent advances based on diffusion models typically rely on separate networks for reference appearance feature extraction and target visual generation, leading to inconsistent domain gaps between references and targets. In this paper, we frame the task as a spatially-conditioned inpainting problem, where the target image is inpainted to maintain appearance consistency with the reference. This approach enables the reference features to guide the generation of pose-compliant targets within a unified denoising network, thereby mitigating domain gaps. Additionally, to better maintain the reference appearance information, we impose a causal feature interaction framework…

OmniDrag: Enabling Motion Control for Omnidirectional Image-to-Video Generation

Weiqi Li, Shijie Zhao, Chong Mou +6 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

As virtual reality gains popularity, the demand for controllable creation of immersive and dynamic omnidirectional videos (ODVs) is increasing. While previous text-to-ODV generation methods achieve impressive results, they struggle with content inaccuracies and inconsistencies due to reliance solely on textual inputs. Although recent motion control techniques provide fine-grained control for video generation, directly applying these methods to ODVs often results in spatial distortion and unsatisfactory performance, especially with complex spherical motions. To tackle these challenges, we propose OmniDrag, the first approach enabling both scene- and object-level motion control for accurate, high-quality omnidirectional image-to-video generation. Building on pretrained video diffusion models, we introduce an omnidirectional control module, which is jointly fine-tuned with temporal…

Neural Video Fields Editing

Shuzhou Yang, Chong Mou, Jiwen Yu +3 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

Diffusion models have revolutionized text-driven video editing. However, applying these methods to real-world editing encounters two significant challenges: (1) the rapid increase in GPU memory demand as the number of frames grows, and (2) the inter-frame inconsistency in edited videos. To this end, we propose NVEdit, a novel text-driven video editing framework designed to mitigate memory overhead and improve consistent editing for real-world long videos. Specifically, we construct a neural video field, powered by tri-plane and sparse grid, to enable encoding long videos with hundreds of frames in a memory-efficient manner. Next, we update the video field through off-the-shelf Text-to-Image (T2I) models to impart text-driven editing effects. A progressive optimization strategy is developed to preserve original temporal priors. Importantly, both the neural video field and T2I model are…

Energetic Probing for the Electron Transfer Reactions Sensitized by\n9,10-Dicyanoanthracene and 9-Cyanoanthracene and Their Modified Zeolite Particles

Yu Chen Chang (3051468), Pei Wen Chang (3051471), Chong Mou Wang (1662331)

Education · Figshare · · cited 0× · DOI ↗ · more like this ·

The photooxidation of diphenylamine (DPA) and triphenylphosphine (P(C<sub>6</sub>H<sub>5</sub>)<sub>3</sub>) sensitized by 9,10-dicyanoanthracene (DCA) and 9-cyanoanthracene (CA) was investigated in this work. Theoretically and\nevidently, DPA could quench the excited DCA and CA (denoted DCA* and CA*) via an electron transfer\npathway. The Stern−Volmer quenching rate constants were calculated to be ca. 4 × 10<sup>4</sup> equiv<sup>-1</sup> s<sup>-1</sup> for both\nDCA* and CA*. In contrast to DPA, P(C<sub>6</sub>H<sub>5</sub>)<sub>3</sub> showed no quenching effects on DCA* and CA* and only\nP(C<sub>6</sub>H<sub>5</sub>)<sub>3</sub> was excited along with the sensitizers upon the exposure to UV light (260 nm). The formal potentials\nof…