Author

Guo Lu

169 papers · 2000–2026

2000peak 23 in 20242026

Frequent co-authors

Papers

showing 1–25 of 169 RSS

DVC: An End-to-end Deep Video Compression Framework

Guo Lu, Wanli Ouyang, Dong Xu +3 more

eess.IV · December 2018 · cited 890× · arXiv ↗ · more like this ·

Conventional video compression approaches use the predictive coding architecture and encode the corresponding motion information and residual information. In this paper, taking advantage of both classical architecture in the conventional video compression method and the powerful non-linear representation ability of neural networks, we propose the first end-to-end video compression deep model that jointly optimizes all the components for video compression. Specifically, learning based optical flow estimation is utilized to obtain the motion information and reconstruct the current frames. Then we employ two auto-encoder style neural networks to compress the corresponding motion and residual information. All the modules are jointly learned through a single loss function, in which they collaborate with each other by considering the trade-off between reducing the number of compression bits…

FVC: A New Framework towards Deep Video Compression in Feature Space

Zhihao Hu, Guo Lu, Dong Xu

eess.IV · May 2021 · cited 351× · arXiv ↗ · more like this ·

Learning based video compression attracts increasing attention in the past few years. The previous hybrid coding approaches rely on pixel space operations to reduce spatial and temporal redundancy, which may suffer from inaccurate motion estimation or less effective motion compensation. In this work, we propose a feature-space video coding network (FVC) by performing all major operations (i.e., motion estimation, motion compression, motion compensation and residual compression) in the feature space. Specifically, in the proposed deformable compensation module, we first apply motion estimation in the feature space to produce motion information (i.e., the offset maps), which will be compressed by using the auto-encoder style network. Then we perform motion compensation by using deformable convolution and generate the predicted feature. After that, we compress the residual feature between…

An End-to-End Learning Framework for Video Compression

Guo Lu, Xiaoyun Zhang, Wanli Ouyang +3 more

Computer Vision and Pattern Recognition · IEEE Transactions on Pattern Analysis and Machine Intelligence · 20 Apr 2020 · cited 212× · DOI ↗ · more like this ·

Traditional video compression approaches build upon the hybrid coding framework with motion-compensated prediction and residual transform coding. In this paper, we propose the first end-to-end deep video compression framework to take advantage of both the classical compression architecture and the powerful non-linear representation ability of neural networks. Our framework employs pixel-wise motion information, which is learned from an optical flow network and further compressed by an auto-encoder network to save bits. The other compression components are also implemented by the well-designed networks for high efficiency. All the modules are jointly optimized by using the rate-distortion trade-off and can collaborate with each other. More importantly, the proposed deep video compression framework is very flexible and can be easily extended by using lightweight or advanced networks for…

Content Adaptive and Error Propagation Aware Deep Video Compression

Guo Lu, Chunlei Cai, Xiaoyun Zhang +4 more

eess.IV · 25 Mar 2020 · cited 176× · arXiv ↗ · more like this ·

Recently, learning based video compression methods attract increasing attention. However, the previous works suffer from error propagation due to the accumulation of reconstructed error in inter predictive coding. Meanwhile, the previous learning based video codecs are also not adaptive to different video contents. To address these two problems, we propose a content adaptive and error propagation aware video compression system. Specifically, our method employs a joint training strategy by considering the compression performance of multiple consecutive frames instead of a single frame. Based on the learned long-term temporal information, our approach effectively alleviates error propagation in reconstructed frames. More importantly, instead of using the hand-crafted coding modes in the traditional compression systems, we design an online encoder updating scheme in our system. The…

A Unified End-to-End Framework for Efficient Deep Image Compression

Jiaheng Liu, Guo Lu, Zhihao Hu +1 more

eess.IV · February 2020 · cited 141× · arXiv ↗ · more like this ·

Image compression is a widely used technique to reduce the spatial redundancy in images. Recently, learning based image compression has achieved significant progress by using the powerful representation ability from neural networks. However, the current state-of-the-art learning based image compression methods suffer from the huge computational cost, which limits their capacity for practical applications. In this paper, we propose a unified framework called Efficient Deep Image Compression (EDIC) based on three new technologies, including a channel attention module, a Gaussian mixture model and a decoder-side enhancement module. Specifically, we design an auto-encoder style network for learning based image compression. To improve the coding efficiency, we exploit the channel relationship between latent representations by using the channel attention module. Besides, the Gaussian mixture…

Coarse-to-fine Deep Video Coding with Hyperprior-guided Mode Prediction

Zhihao Hu, Guo Lu, Jinyang Guo +3 more

cs.CV · 15 Jun 2022 · cited 128× · arXiv ↗ · more like this ·

The previous deep video compression approaches only use the single scale motion compensation strategy and rarely adopt the mode prediction technique from the traditional standards like H.264/H.265 for both motion and residual compression. In this work, we first propose a coarse-to-fine (C2F) deep video compression framework for better motion compensation, in which we perform motion estimation, compression and compensation twice in a coarse to fine manner. Our C2F framework can achieve better motion compensation results without significantly increasing bit costs. Observing hyperprior information (i.e., the mean and variance values) from the hyperprior networks contains discriminant statistical information of different patches, we also propose two efficient hyperprior-guided mode prediction methods. Specifically, using hyperprior information as the input, we propose two mode prediction…

Learning 3D Human Shape and Pose from Dense Body Parts

Hongwen Zhang, Jie Cao, Guo Lu +2 more

cs.CV · December 2019 · cited 78× · arXiv ↗ · more like this ·

Reconstructing 3D human shape and pose from monocular images is challenging despite the promising results achieved by the most recent learning-based methods. The commonly occurred misalignment comes from the facts that the mapping from images to the model space is highly non-linear and the rotation-based pose representation of body models is prone to result in the drift of joint positions. In this work, we investigate learning 3D human shape and pose from dense correspondences of body parts and propose a Decompose-and-aggregate Network (DaNet) to address these issues. DaNet adopts the dense correspondence maps, which densely build a bridge between 2D pixels and 3D vertices, as intermediate representations to facilitate the learning of 2D-to-3D mapping. The prediction modules of DaNet are decomposed into one global stream and multiple local streams to enable global and fine-grained…

Formation Control of Mobile Robots Using Distributed Controller With Sampled-Data and Communication Delays

Zhe Liu, Weidong Chen, Jun‐Guo Lu +2 more

Computer Networks and Communications · IEEE Transactions on Control Systems Technology · 3 Feb 2016 · cited 67× · DOI ↗ · more like this ·

This brief presents a synchronous formation control approach using distributed controller with sampled-data and communication delay. The local information of each robot is sampled first, and then transmitted to its coupling robots through a network with communication delays, i.e., the information exchanged between coupling robots is the sampled-data with communication delays. The distributed formation controller is designed by using the synchronization approach, which enables mobile robots to maintain time-varying formations while performing a group task in a synchronous manner. A sufficient condition for formation control that can guarantee the exponential convergence of both the position errors and the synchronization errors is obtained by using the Lyapunov approach. Based on the proposed sufficient condition, an upper bound of the sampling period and the communication delay that…

Finite-Time Synchronization of Fractional-Order Delayed Fuzzy Cellular Neural Networks With Parameter Uncertainties

Feifei Du, Jun‐Guo Lu

Computer Networks and Communications · IEEE Transactions on Fuzzy Systems · 12 Oct 2022 · cited 63× · DOI ↗ · more like this ·

The finite-time synchronization (FTS) is studied for a class of fractional-order delayed fuzzy cellular neural networks (FODFCNNs) with parameter uncertainties. A linear fractional-order finite time inequality (FOFTI) <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${}^{c}_{t_{0}}D^{p}_{t} V(t)\leq -a V(t)-b$</tex-math></inline-formula> is extended to the nonlinear case <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${}^{c}_{t_{0}}D^{p}_{t} V(t)\leq -a V^{-\eta }(t)-b$</tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\eta \geq 1$</tex-math></inline-formula> , which plays a vital role in the FTS of…

New Criteria on Finite-Time Stability of Fractional-Order Hopfield Neural Networks With Time Delays

Feifei Du, Jun‐Guo Lu

Computer Networks and Communications · IEEE Transactions on Neural Networks and Learning Systems · 21 Aug 2020 · cited 62× · DOI ↗ · more like this ·

In this article, the finite-time stability (FTS) of fractional-order Hopfield neural networks with time delays (FHNNTDs) is studied. A widely used inequality in investigating the stability of the fractional-order neural networks is fractional-order Gronwall inequality related to the Mittag-Leffler function, which cannot be directly used to study the stability of the factional-order neural networks with time delays. In the existing works related to fractional-order Gronwall inequality with time delays, the order was divided into two cases: λ ∈ (0,0.5] and λ ∈ (0.5,+∞) . In this article, a new fractional-order Gronwall integral inequality with time delay and the unified form for all the fractional order is developed, which can be widely applied to investigate FTS of various fractional-order systems with time delays. Based on this new inequality, a new criterion for the FTS of FHNNTDs is…

Preprocessing Enhanced Image Compression for Machine Vision

Guo Lu, Xingtong Ge, Tianxiong Zhong +2 more

eess.IV · 12 Jun 2022 · cited 61× · arXiv ↗ · more like this ·

Recently, more and more images are compressed and sent to the back-end devices for the machine analysis tasks~(\textit{e.g.,} object detection) instead of being purely watched by humans. However, most traditional or learned image codecs are designed to minimize the distortion of the human visual system without considering the increased demand from machine vision systems. In this work, we propose a preprocessing enhanced image compression method for machine vision tasks to address this challenge. Instead of relying on the learned image codecs for end-to-end optimization, our framework is built upon the traditional non-differential codecs, which means it is standard compatible and can be easily deployed in practical applications. Specifically, we propose a neural preprocessing module before the encoder to maintain the useful semantic information for the downstream tasks and suppress the…

Exploiting Intra-Slice and Inter-Slice Redundancy for Learning-Based Lossless Volumetric Image Compression

Zhenghao Chen, Shuhang Gu, Guo Lu +1 more

Computer Vision and Pattern Recognition · IEEE Transactions on Image Processing · 2022 · cited 52× · DOI ↗ · more like this ·

3D volumetric image processing has attracted increasing attention in the last decades, in which one major research area is to develop efficient lossless volumetric image compression techniques to better store and transmit such images with massive amount of information. In this work, we propose the first end-to-end optimized learning framework for losslessly compressing 3D volumetric data. Our approach builds upon a hierarchical compression scheme by additionally introducing the intra-slice auxiliary features and estimating the entropy model based on both intra-slice and inter-slice latent priors. Specifically, we first extract the hierarchical intra-slice auxiliary features through multi-scale feature extraction modules. Then, an Intra-slice and Inter-slice Conditional Entropy Coding module is proposed to fuse the intra-slice and inter-slice information from different scales as the…

A Coding Framework and Benchmark towards Low-Bitrate Video Understanding

Yuan Tian, Guo Lu, Yichao Yan +3 more

eess.IV · February 2022 · cited 49× · arXiv ↗ · more like this ·

Video compression is indispensable to most video analysis systems. Despite saving transportation bandwidth, it also deteriorates downstream video understanding tasks, especially at low-bitrate settings. To systematically investigate this problem, we first thoroughly review the previous methods, revealing that three principles, i.e., task-decoupled, label-free, and data-emerged semantic prior, are critical to a machine-friendly coding framework but are not fully satisfied so far. In this paper, we propose a traditional-neural mixed coding framework that simultaneously fulfills all these principles, by taking advantage of both traditional codecs and neural networks (NNs). On one hand, the traditional codecs can efficiently encode the pixel signal of videos but may distort the semantic information. On the other hand, highly non-linear NNs are proficient in condensing video semantics into a…

Deep Non-Local Kalman Network for Video Compression Artifact Reduction

Guo Lu, Xiaoyun Zhang, Wanli Ouyang +3 more

Computer Vision and Pattern Recognition · IEEE Transactions on Image Processing · October 2019 · cited 49× · DOI ↗ · more like this ·

Video compression algorithms are widely used to reduce the huge size of video data, but they also introduce unpleasant visual artifacts due to the lossy compression. In order to improve the quality of the compressed videos, we proposed a deep non-local Kalman network for compression artifact reduction. Specifically, the video restoration is modeled as a Kalman filtering procedure and the decoded frames can be restored from the proposed deep Kalman model. Instead of using the noisy previous decoded frames as temporal information, the less noisy previous restored frame is employed in a recursive way, which provides the potential to generate high quality restored frames. In the proposed framework, several deep neural networks are utilized to estimate the corresponding states in the Kalman filter and integrated together in the deep Kalman filtering network. More importantly, we also exploit…

MISC: Ultra-low Bitrate Image Semantic Compression Driven by Large Multimodal Model

Chunyi Li, Guo Lu, Donghui Feng +6 more

cs.CV · February 2024 · cited 46× · arXiv ↗ · more like this ·

With the evolution of storage and communication protocols, ultra-low bitrate image compression has become a highly demanding topic. However, existing compression algorithms must sacrifice either consistency with the ground truth or perceptual quality at ultra-low bitrate. In recent years, the rapid development of the Large Multimodal Model (LMM) has made it possible to balance these two goals. To solve this problem, this paper proposes a method called Multimodal Image Semantic Compression (MISC), which consists of an LMM encoder for extracting the semantic information of the image, a map encoder to locate the region corresponding to the semantic, an image encoder generates an extremely compressed bitstream, and a decoder reconstructs the image based on the above information. Experimental results show that our proposed MISC is suitable for compressing both traditional Natural Sense…

TSA-SCC: Text Semantic-Aware Screen Content Coding With Ultra Low Bitrate

Tong Tang, Ling Li, Xiaoyu Wu +4 more

Computer Vision and Pattern Recognition · IEEE Transactions on Image Processing · 2022 · cited 45× · DOI ↗ · more like this ·

Due to the rapid growth of web conferences, remote screen sharing, and online games, screen content has become an important type of internet media information and over 90% of online media interactions are screen based. Meanwhile, as the main component in the screen content, textual information averagely takes up over 40% of the whole image on various commonly used screen content datasets. However, it is difficult to compress the textual information by using the traditional coding schemes as HEVC, which assumes strong spatial and temporal correlations within the image/video. State-of-the-art screen content coding (SCC) standard as HEVC-SCC still adopts a block-based coding framework and does not consider the text semantics for compression, thus inevitably blurring texts at a lower bitrate. In this paper, we propose a general text semantic-aware screen content coding scheme (TSA-SCC) for…

VARFVV: View-Adaptive Real-Time Interactive Free-View Video Streaming with Edge Computing

Qiang Hu, Qihan He, Houqiang Zhong +4 more

cs.MM · 23 Jan 2025 · cited 44× · arXiv ↗ · more like this ·

Free-view video (FVV) allows users to explore immersive video content from multiple views. However, delivering FVV poses significant challenges due to the uncertainty in view switching, combined with the substantial bandwidth and computational resources required to transmit and decode multiple video streams, which may result in frequent playback interruptions. Existing approaches, either client-based or cloud-based, struggle to meet high Quality of Experience (QoE) requirements under limited bandwidth and computational resources. To address these issues, we propose VARFVV, a bandwidth- and computationally-efficient system that enables real-time interactive FVV streaming with high QoE and low switching delay. Specifically, VARFVV introduces a low-complexity FVV generation scheme that reassembles multiview video frames at the edge server based on user-selected view tracks, eliminating the…

Self-Conditioned Probabilistic Learning of Video Rescaling

Yuan Tian, Guo Lu, Xiongkuo Min +4 more

cs.CV · July 2021 · cited 43× · arXiv ↗ · more like this ·

Bicubic downscaling is a prevalent technique used to reduce the video storage burden or to accelerate the downstream processing speed. However, the inverse upscaling step is non-trivial, and the downscaled video may also deteriorate the performance of downstream tasks. In this paper, we propose a self-conditioned probabilistic framework for video rescaling to learn the paired downscaling and upscaling procedures simultaneously. During the training, we decrease the entropy of the information lost in the downscaling by maximizing its probability conditioned on the strong spatial-temporal prior information within the downscaled video. After optimization, the downscaled video by our framework preserves more meaningful information, which is beneficial for both the upscaling step and the downstream tasks, e.g., video action recognition task. We further extend the framework to a lossy video…

LSVC: A Learning-based Stereo Video Compression Framework

Zhenghao Chen, Guo Lu, Zhihao Hu +3 more

Computer Vision and Pattern Recognition · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · June 2022 · cited 40× · DOI ↗ · more like this ·

In this work, we propose the first end-to-end optimized framework for compressing automotive stereo videos (i.e., stereo videos from autonomous driving applications) from both left and right views. Specifically, when compressing the current frame from each view, our framework reduces temporal redundancy by performing motion compensation using the reconstructed intra-view adjacent frame and at the same time exploits binocular redundancy by conducting disparity compensation using the latest reconstructed cross-view frame. Moreover, to effectively compress the introduced motion and disparity offsets for better compensation, we further propose two novel schemes called motion residual compression and disparity residual compression to respectively generate the predicted motion offset and disparity offset from the previously compressed motion offset and disparity offset, such that we can more…

DaNet

Hongwen Zhang, Jie Cao, Guo Lu +2 more

Computer Vision and Pattern Recognition · 15 Oct 2019 · cited 38× · DOI ↗ · more like this ·

Reconstructing 3D human shape and pose from a monocular image is challenging despite the promising results achieved by most recent learning based methods. The commonly occurred misalignment comes from the facts that the mapping from image to model space is highly non-linear and the rotation-based pose representation of the body model is prone to result in drift of joint positions. In this work, we present the Decompose-and-aggregate Network (DaNet) to address these issues. DaNet includes three new designs, namely UVI guided learning, decomposition for fine-grained perception, and aggregation for robust prediction. First, we adopt the UVI maps, which densely build a bridge between 2D pixels and 3D vertexes, as an intermediate representation to facilitate the learning of image-to-model mapping. Second, we decompose the prediction task into one global stream and multiple local streams so…

Novel Integration of Frame Rate Up Conversion and HEVC Coding Based on Rate-Distortion Optimization

Guo Lu, Xiaoyun Zhang, Li Chen +1 more

Computer Vision and Pattern Recognition · IEEE Transactions on Image Processing · 30 Oct 2017 · cited 28× · DOI ↗ · more like this ·

Frame rate up conversion (FRUC) can improve the visual quality by interpolating new intermediate frames. However, high frame rate videos by FRUC are confronted with more bitrate consumption or annoying artifacts of interpolated frames. In this paper, a novel integration framework of FRUC and high efficiency video coding (HEVC) is proposed based on rate-distortion optimization, and the interpolated frames can be reconstructed at encoder side with low bitrate cost and high visual quality. First, joint motion estimation (JME) algorithm is proposed to obtain robust motion vectors, which are shared between FRUC and video coding. What's more, JME is embedded into the coding loop and employs the original motion search strategy in HEVC coding. Then, the frame interpolation is formulated as a rate-distortion optimization problem, where both the coding bitrate consumption and visual quality are…

CBANet: Toward Complexity and Bitrate Adaptive Deep Image Compression Using a Single Network

Jinyang Guo, Dong Xu, Guo Lu

Computer Vision and Pattern Recognition · IEEE Transactions on Image Processing · 2023 · cited 27× · DOI ↗ · more like this ·

In this work, we propose a new deep image compression framework called Complexity and Bitrate Adaptive Network (CBANet) that aims to learn one single network to support variable bitrate coding under various computational complexity levels. In contrast to the existing state-of-the-art learning-based image compression frameworks that only consider the rate-distortion trade-off without introducing any constraint related to the computational complexity, our CBANet considers the complex rate-distortion-complexity trade-off when learning a single network to support multiple computational complexity levels and variable bitrates. Since it is a non-trivial task to solve such a rate-distortion-complexity related optimization problem, we propose a two-step approach to decouple this complex optimization task into a complexity-distortion optimization sub-task and a rate-distortion optimization…

Non-Semantics Suppressed Mask Learning for Unsupervised Video Semantic Compression

Yuan Tian, Guo Lu, Guangtao Zhai +1 more

Computer Vision and Pattern Recognition · October 2023 · cited 26× · DOI ↗ · more like this ·

Most video compression methods aim to improve the decoded video visual quality, instead of particularly guaranteeing the semantic-completeness, which deteriorates downstream video analysis tasks, e.g., action recognition. In this paper, we focus on a novel unsupervised video semantic compression problem, where video semantics is compressed in a downstream task-agnostic manner. To tackle this problem, we first propose a Semantic-Mining-then-Compensation (SMC) framework to enhance the plain video codec with powerful semantic coding capability. Then, we optimize the framework with only unlabeled video data, by masking out a proportion of the compressed video and reconstructing the masked regions of the original video, which is inspired by recent masked image modeling (MIM) methods. Although the MIM scheme learns generalizable semantic features, its inner generative learning paradigm may…

Learning based Multi-modality Image and Video Compression

Guo Lu, Tianxiong Zhong, Jing Geng +2 more

Computer Vision and Pattern Recognition · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · June 2022 · cited 26× · DOI ↗ · more like this ·

Multi-modality (i.e., multi-sensor) data is widely used in various vision tasks for more accurate or robust perception. However, the increased data modalities bring new challenges for data storage and transmission. The existing data compression approaches usually adopt individual codecs for each modality without considering the correlation between different modalities. This work proposes a multi-modality compression framework for infrared and visible image pairs by exploiting the cross-modality redun-dancy. Specifically, given the image in the reference modality (e.g., the infrared image), we use the channel-wise alignment module to produce the aligned features based on the affine transform. Then the aligned feature is used as the context information for compressing the image in the current modality (e.g., the visible image), and the corresponding affine coefficients are losslessly…

Time Domain Solution Analysis and Novel Admissibility Conditions of Singular Fractional-Order Systems

Qing‐Hao Zhang, Jun‐Guo Lu, Yingdong Ma +1 more

Modeling and Simulation · IEEE Transactions on Circuits and Systems I Regular Papers · 13 Nov 2020 · cited 26× · DOI ↗ · more like this ·

This paper investigates the regularity, nonimpulsiveness, stability and admissibility of the singular fractional-order systems with the fractional-order α ∈ (0, 1). Firstly, the structure, existence and uniqueness of the time domain solutions of singular fractional-order systems are analyzed based on the Kronecker equivalent standard form. The necessary and sufficient condition for the regularity of singular fractional-order systems is proposed on the basis of the above analysis. Secondly, the necessary and sufficient conditions of non-impulsiveness as well as stability are obtained based on the proposed time domain solutions of singular fractional-order systems, respectively. Thirdly, two novel sufficient and necessary conditions for the admissibility of singular fractional-order systems are derived including the non-strict linear matrix inequality form and the linear matrix inequality…