arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Yurui Ren

13 papers · 2017–2025

2017: 12018: 12019: 12020: 32021: 22022: 32023: 12024: 02025: 1
2017peak 3 in 20202025

Frequent co-authors

Papers

showing 1–13 of 13 RSS

StructureFlow: Image Inpainting via Structure-aware Appearance Flow

Yurui Ren, Xiaoming Yu, Ruonan Zhang +3 more

cs.CV · · cited 370× · arXiv ↗ · more like this ·

Image inpainting techniques have shown significant improvements by using deep neural networks recently. However, most of them may either fail to reconstruct reasonable structures or restore fine-grained textures. In order to solve this problem, in this paper, we propose a two-stage model which splits the inpainting task into two parts: structure reconstruction and texture generation. In the first stage, edge-preserved smooth images are employed to train a structure reconstructor which completes the missing structures of the inputs. In the second stage, based on the reconstructed structures, a texture generator using appearance flow is designed to yield image details. Experiments on multiple publicly available datasets show the superior performance of the proposed network.

A New Low-Light Image Enhancement Algorithm Using Camera Response Model

Zhenqiang Ying, Ge Li, Yurui Ren +2 more

Computer Vision and Pattern Recognition · · cited 320× · DOI ↗ · more like this ·

Low-light images are not conducive to human observation and computer vision algorithms due to their low visibility. To solve this problem, many image enhancement techniques have been proposed. However, existing techniques inevitably introduce color and lightness distortion when increasing visibility. To lower the distortion, we propose a novel enhancement method using the response characteristics of cameras. First, we investigate the relationship between two images with different exposures to obtain an accurate camera response model. Then we borrow the illumination estimation techniques to estimate the exposure ratio map. Finally, we use our camera response model to adjust each pixel to its desired exposure according to the estimated exposure ratio map. Experiments show that our method can obtain enhancement results with less color and lightness distortion compared to several…

PIRenderer: Controllable Portrait Image Generation via Semantic Neural Rendering

Yurui Ren, Ge Li, Yuanqi Chen +2 more

cs.CV · · cited 299× · arXiv ↗ · more like this ·

Generating portrait images by controlling the motions of existing faces is an important task of great consequence to social media industries. For easy use and intuitive control, semantically meaningful and fully disentangled parameters should be used as modifications. However, many existing techniques do not provide such fine-grained controls or use indirect editing methods i.e. mimic motions of other individuals. In this paper, a Portrait Image Neural Renderer (PIRenderer) is proposed to control the face motions with the parameters of three-dimensional morphable face models (3DMMs). The proposed model can generate photo-realistic portrait images with accurate movements according to intuitive modifications. Experiments on both direct and indirect editing tasks demonstrate the superiority of this model. Meanwhile, we further extend this model to tackle the audio-driven facial reenactment…

LECARM: Low-Light Image Enhancement Using the Camera Response Model

Yurui Ren, Zhenqiang Ying, Thomas H. Li +1 more

Computer Vision and Pattern Recognition · IEEE Transactions on Circuits and Systems for Video Technology · · cited 234× · DOI ↗ · more like this ·

Low-light image enhancement algorithms can improve the visual quality of low-light images and support the extraction of valuable information for some computer vision techniques. However, existing techniques inevitably introduce color and lightness distortions when enhancing the images. To lower the distortions, we propose a novel enhancement framework using the response characteristics of cameras. First, we discuss how to determine a reasonable camera response model and its parameters. Then, we use the illumination estimation techniques to estimate the exposure ratio for each pixel. Finally, the selected camera response model is used to adjust each pixel to the desired exposure according to the estimated exposure ratio map. Experiments show that our method can obtain enhancement results with fewer color and lightness distortions compared with the several state-of-the-art methods.

Deep Image Spatial Transformation for Person Image Generation

Yurui Ren, Xiaoming Yu, Junming Chen +2 more

cs.CV · · cited 205× · arXiv ↗ · more like this ·

Pose-guided person image generation is to transform a source person image to a target pose. This task requires spatial manipulations of source data. However, Convolutional Neural Networks are limited by the lack of ability to spatially transform the inputs. In this paper, we propose a differentiable global-flow local-attention framework to reassemble the inputs at the feature level. Specifically, our model first calculates the global correlations between sources and targets to predict flow fields. Then, the flowed local patch pairs are extracted from the feature maps to calculate the local attention coefficients. Finally, we warp the source features using a content-aware sampling method with the obtained local attention coefficients. The results of both subjective and objective experiments demonstrate the superiority of our model. Besides, additional results in video animation and view…

Neural Texture Extraction and Distribution for Controllable Person Image Synthesis

Yurui Ren, Xiaoqing Fan, Ge Li +2 more

cs.CV · · cited 100× · arXiv ↗ · more like this ·

We deal with the controllable person image synthesis task which aims to re-render a human from a reference image with explicit control over body pose and appearance. Observing that person images are highly structured, we propose to generate desired images by extracting and distributing semantic entities of reference images. To achieve this goal, a neural texture extraction and distribution operation based on double attention is described. This operation first extracts semantic neural textures from reference feature maps. Then, it distributes the extracted neural textures according to the spatial distributions learned from target poses. Our model is trained to predict human images in arbitrary poses, which encourages it to extract disentangled and expressive neural textures representing the appearance of different semantic entities. The disentangled representation further enables…

Deep Spatial Transformation for Pose-Guided Person Image Generation and Animation

Yurui Ren, Ge Li, Shan Liu +1 more

cs.CV · · cited 82× · arXiv ↗ · more like this ·

Pose-guided person image generation and animation aim to transform a source person image to target poses. These tasks require spatial manipulation of source data. However, Convolutional Neural Networks are limited by the lack of ability to spatially transform the inputs. In this paper, we propose a differentiable global-flow local-attention framework to reassemble the inputs at the feature level. This framework first estimates global flow fields between sources and targets. Then, corresponding local source feature patches are sampled with content-aware local attention coefficients. We show that our framework can spatially transform the inputs in an efficient manner. Meanwhile, we further model the temporal consistency for the person image animation task to generate coherent videos. The experiment results of both image generation and animation tasks demonstrate the superiority of our…

Context-aware Attention Network for Predicting Image Aesthetic Subjectivity

Munan Xu, Jia-Xing Zhong, Yurui Ren +2 more

Computer Vision and Pattern Recognition · · cited 22× · DOI ↗ · more like this ·

Image aesthetic assessment involves both fine-grained details and the holistic layout of images. However, most of current approaches learn the local and the holistic information separately, which has a potential loss of contextual information. Additionally, learning-based methods mainly cast image aesthetic assessment as a binary classification or a regression problem, which cannot sufficiently delineate the potential diversity of human aesthetic experience. To address these limitations, we attempt to render the contextual information and model the varieties of aesthetic experience. Specifically, we explore a context-aware attention module in two dimensions: hierarchical and spatial. The hierarchical context is introduced to present the concern of multi-level aesthetic details while the spatial context is served to yield the long-range perception of images. Based on the attention model…

Combining Attention with Flow for Person Image Synthesis

Yurui Ren, Yubo Wu, Thomas H. Li +2 more

cs.CV · · cited 20× · arXiv ↗ · more like this ·

Pose-guided person image synthesis aims to synthesize person images by transforming reference images into target poses. In this paper, we observe that the commonly used spatial transformation blocks have complementary advantages. We propose a novel model by combining the attention operation with the flow-based operation. Our model not only takes the advantage of the attention operation to generate accurate target structures but also uses the flow-based operation to sample realistic source textures. Both objective and subjective experiments demonstrate the superiority of our model. Meanwhile, comprehensive ablation studies verify our hypotheses and show the efficacy of the proposed modules. Besides, additional experiments on the portrait image editing task demonstrate the versatility of the proposed combination.

Seed1.5-VL Technical Report

Dong Guo, Faming Wu, Feida Zhu +194 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

We present Seed1.5-VL, a vision-language foundation model designed to advance general-purpose multimodal understanding and reasoning. Seed1.5-VL is composed with a 532M-parameter vision encoder and a Mixture-of-Experts (MoE) LLM of 20B active parameters. Despite its relatively compact architecture, it delivers strong performance across a wide spectrum of public VLM benchmarks and internal evaluation suites, achieving the state-of-the-art performance on 38 out of 60 public benchmarks. Moreover, in agent-centric tasks such as GUI control and gameplay, Seed1.5-VL outperforms leading multimodal systems, including OpenAI CUA and Claude 3.7. Beyond visual and video understanding, it also demonstrates strong reasoning abilities, making it particularly effective for multimodal reasoning challenges such as visual puzzles. We believe these capabilities will empower broader applications across…

Flow-Guided Attention Deformation for Person Image Generation

Yubo Wu, Yurui Ren, Yuanqi Chen +1 more

Computer Vision and Pattern Recognition · · cited 0× · DOI ↗ · more like this ·

Pose-guided person image generation aims to transfer reference images to target poses while preserving the source appearance. Recent approaches achieve considerable improvement by using spatial transformation modules such as attention operation. However, the commonly used vanilla attention tends to generate a dense correlation matrix which means that the value of a target position is the weighted sum of many source positions, resulting in blurry appearance. In this paper, we propose a novel model named Flow-guided Attention Deformation (FAD) to perform the spatial transformation. Our model first establishes the correlation between sources and targets with a flow-guided attention operation. Then, with the obtained correlation matrix, we perform an accurate deformation for source features to generate the predicted image. Extensive results demonstrate the superiority of the proposed…

Context-Aware Hierarchical Transformer for Fine-Grained Video-Text Retrieval

Mingliang Chen, Weimin Zhang, Yurui Ren +1 more

Computer Vision and Pattern Recognition · 2022 IEEE International Conference on Image Processing (ICIP) · · cited 0× · DOI ↗ · more like this ·

Video-Text Retrieval aims to perform accurate retrieval process that adopts texts to retrieve the corresponding videos, and vice versa. Typically, mainstream methods solve this problem by learning a common joint embedding space, and then measure the similarities between videos and texts. However, these methods lack the ability to represent detailed semantic information. Therefore, we first utilize three pre-trained models to construct the video embeddings of different semantic levels, and then propose a Context-aware Hierarchical Transformer (CHT) model to encode the context information between these levels. More specifically, our model builds fine-grained hierarchical video embeddings of three semantic levels: global, objects, and actions. Attention-based contextual transformers are utilized to establish the context interactions between different semantic levels. Experimental results…

Deep Geometry Post-Processing for Decompressed Point Clouds

Xiaoqing Fan, Ge Li, Dingquan Li +3 more

cs.CV · · arXiv ↗ · more like this ·

Point cloud compression plays a crucial role in reducing the huge cost of data storage and transmission. However, distortions can be introduced into the decompressed point clouds due to quantization. In this paper, we propose a novel learning-based post-processing method to enhance the decompressed point clouds. Specifically, a voxelized point cloud is first divided into small cubes. Then, a 3D convolutional network is proposed to predict the occupancy probability for each location of a cube. We leverage both local and global contexts by generating multi-scale probabilities. These probabilities are progressively summed to predict the results in a coarse-to-fine manner. Finally, we obtain the geometry-refined point clouds based on the predicted probabilities. Different from previous methods, we deal with decompressed point clouds with huge variety of distortions using a single model.…