arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Anni Tang

16 papers · 2021–2025

2021: 22022: 32023: 12024: 72025: 3
2021peak 7 in 20242025

Frequent co-authors

Papers

showing 1–16 of 16 RSS

Generative Compression for Face Video: A Hybrid Scheme

Anni Tang, Yan Huang, Jun Ling +4 more

eess.IV · · cited 20× · arXiv ↗ · more like this ·

As the latest video coding standard, versatile video coding (VVC) has shown its ability in retaining pixel quality. To excavate more compression potential for video conference scenarios under ultra-low bitrate, this paper proposes a bitrate adjustable hybrid compression scheme for face video. This hybrid scheme combines the pixel-level precise recovery capability of traditional coding with the generation capability of deep learning based on abridged information, where Pixel wise Bi-Prediction, Low-Bitrate-FOM and Lossless Keypoint Encoder collaborate to achieve PSNR up to 36.23 dB at a low bitrate of 1.47 KB/s. Without introducing any additional bitrate, our method has a clear advantage over VVC under a completely fair comparative experiment, which proves the effectiveness of our proposed scheme. Moreover, our scheme can adapt to any existing encoder / configuration to deal with…

A Generative Compression Framework For Low Bandwidth Video Conference

Dahu Feng, Yan Huang, Yiwei Zhang +3 more

Computer Vision and Pattern Recognition · · cited 19× · DOI ↗ · more like this ·

Video conferences introduce a new scenario for video transmission, which focuses on keeping the fidelity of faces even in the low bandwidth network environment. In this work, we propose VSBNet, one of the frameworks to utilize face landmarks in video compression. Our method utilizes the adversarial learning to reconstruct origin frames from the landmarks. To recover more details and keep the consistency of identity, we propose the concept of visual sensitivity to separate the contour of the face from the fast-moving parts, such as eyes and mouth. Experimental results demonstrate the superiority of our framework with a low bit rate of around 1KB/s.

High-Fidelity Face Reenactment Via Identity-Matched Correspondence Learning

Han Xue, Jun Ling, Anni Tang +3 more

Computer Vision and Pattern Recognition · ACM Transactions on Multimedia Computing Communications and Applications · · cited 13× · DOI ↗ · more like this ·

Face reenactment aims to generate an animation of a source face using the poses and expressions from a target face. Although recent methods have made remarkable progress by exploiting generative adversarial networks, they are limited in generating high-fidelity and identity-preserving results due to the inappropriate driving information and insufficiently effective animating strategies. In this work, we propose a novel face reenactment framework that achieves both high-fidelity generation and identity preservation. Instead of sparse face representations (e.g., facial landmarks and keypoints), we utilize the Projected Normalized Coordinate Code (PNCC) to better preserve facial details. We propose to reconstruct the PNCC with the source identity parameters and the target pose and expression parameters estimated by 3D face reconstruction to factor out the target identity. By adopting the…

Spatial Pattern, Classification, and Influencing Factors of High-Quality County Development in China

Zhe Cheng, Min Wang, Yun Li +2 more

Urban Studies · Journal of Urban Planning and Development · · cited 7× · DOI ↗ · more like this ·

The county is the basic territory unit of the national governance system and sustainable development. This study aimed to assess and categorize the high-quality county development in China based on the nation’s imbalanced and inadequate development conditions. By constructing a comprehensive evaluation system for high-quality county development, employing spatial analysis and the natural breakpoint approach, the multiscale spatial pattern of high-quality county development was identified and classified. In addition, the grey relational analysis method was used to measure the influencing factors. According to the evaluation scores for the high-quality county development in this study, the counties could be classified into three categories: (1) rising developed counties; (2) steady developing counties; and (3) underdeveloped counties. Significant spatial differences and agglomeration…

Memories are One-to-Many Mapping Alleviators in Talking Face Generation

Anni Tang, Tianyu He, Xu Tan +2 more

cs.CV · · cited 7× · arXiv ↗ · more like this ·

Talking face generation aims at generating photo-realistic video portraits of a target person driven by input audio. Due to its nature of one-to-many mapping from the input audio to the output video (e.g., one speech content may have multiple feasible visual appearances), learning a deterministic mapping like previous works brings ambiguity during training, and thus causes inferior visual results. Although this one-to-many mapping could be alleviated in part by a two-stage framework (i.e., an audio-to-expression model followed by a neural-rendering model), it is still insufficient since the prediction is produced without enough information (e.g., emotions, wrinkles, etc.). In this paper, we propose MemFace to complement the missing information with an implicit memory and an explicit memory that follow the sense of the two stages respectively. More specifically, the implicit memory is…

Efficient Dynamic-NeRF Based Volumetric Video Coding with Rate Distortion Optimization

Zhiyu Zhang, Guo Lu, Huanxiong Liang +3 more

cs.CV · · cited 4× · arXiv ↗ · more like this ·

Volumetric videos, benefiting from immersive 3D realism and interactivity, hold vast potential for various applications, while the tremendous data volume poses significant challenges for compression. Recently, NeRF has demonstrated remarkable potential in volumetric video compression thanks to its simple representation and powerful 3D modeling capabilities, where a notable work is ReRF. However, ReRF separates the modeling from compression process, resulting in suboptimal compression efficiency. In contrast, in this paper, we propose a volumetric video compression method based on dynamic NeRF in a more compact manner. Specifically, we decompose the NeRF representation into the coefficient fields and the basis fields, incrementally updating the basis fields in the temporal domain to achieve dynamic modeling. Additionally, we perform end-to-end joint optimization on the modeling and…

High-Fidelity Free-View Talking Head Synthesis for Low-Bandwidth Video Conference

Zhiyu Zhang, Anni Tang, Chen Zhu +3 more

Computer Vision and Pattern Recognition · · cited 3× · DOI ↗ · more like this ·

As video conferencing becomes an indispensable part of human’s daliy life, how to achieve a high-fidelity calling experience under low bandwidth has been a popular and challenging issue. Deep generative models have great potential in low-bandwidth facial video compression due to the excellent generation capability based on abridged information. Nevertheless, exsiting deep generation-based compression methods tend to handle motion information in pure 2D or pseudo 3D space, causing facial distortion when large head poses are encountered. In this paper, we propose a 3D-aware high-fidelity facial video conferencing system based on a parameterized NeRF-based face model. Through the compression of the parameterized face model and the transmisstion of extracted facial parameters, we implement high-fidelity talking head synthesis for video conferencing at an ultra-low bitrate. Additionally, the…

Rate-aware Compression for NeRF-based Volumetric Video

Zhiyu Zhang, Guo Lu, Huanxiong Liang +3 more

cs.MM · · cited 2× · arXiv ↗ · more like this ·

The neural radiance fields (NeRF) have advanced the development of 3D volumetric video technology, but the large data volumes they involve pose significant challenges for storage and transmission. To address these problems, the existing solutions typically compress these NeRF representations after the training stage, leading to a separation between representation training and compression. In this paper, we try to directly learn a compact NeRF representation for volumetric video in the training stage based on the proposed rate-aware compression framework. Specifically, for volumetric video, we use a simple yet effective modeling strategy to reduce temporal redundancy for the NeRF representation. Then, during the training phase, an implicit entropy model is utilized to estimate the bitrate of the NeRF representation. This entropy model is then encoded into the bitstream to assist in the…

SingAvatar: High-fidelity Audio-driven Singing Avatar Synthesis

Wentao Ma, Anni Tang, Jun Ling +4 more

Computer Vision and Pattern Recognition · · cited 2× · DOI ↗ · more like this ·

Generating photo-realistic avatars from audio plays an important role in extended reality (XR) and metaverse. In this paper, we lift the input audio from speech to singing, which has been rarely studied. The significant distinction between singing and talking poses great challenges for adapting talking face generation methods to the singing regime. To address this, we propose a high-fidelity singing avatar synthesis method called SingAvatar. Besides the audio, we incorporate vocal conditions involving phonemes and variance to alleviate the ambiguity of learning the singing-to-face mapping. Concretely, we tailor a two-stage pipeline: singing voice synthesis and portrait generation from the synthesized audio and auxiliary vocal conditions. Further, we curate a fine-grained singing head dataset containing singing videos with synchronized audio and accurate vocal conditions. In experiments…

ViCoFace: Learning Disentangled Latent Motion Representations for Visual-Consistent Face Reenactment

Jun Ling, Han Xue, Anni Tang +2 more

Computer Vision and Pattern Recognition · ACM Transactions on Multimedia Computing Communications and Applications · · cited 1× · DOI ↗ · more like this ·

Unsupervised face reenactment aims to animate a source image to imitate the motions of a target image while retaining the source portrait’s attributes like facial geometry, identity, hair texture, and background. While prior methods can extract the motion from the target image via compact representations (e.g., keypoints or latent motion bases [ 50 ]), they are not robust in predicting motions that are disentangled with portrait attributes, thus failing to preserve portrait attributes in the cross-subject reenactment. In this work, we propose an effective and cost-efficient face reenactment approach to address this issue. Our approach is highlighted by two major strengths. First, based on the theory of latent motion bases, we disentangle the full-head motion into two parts: the transferable motion and preservable motion and then compose the full motion representation using latent…

Compositional 3D-aware Video Generation with LLM Director

Hanxin Zhu, Tianyu He, Anni Tang +3 more

cs.CV · · cited 1× · arXiv ↗ · more like this ·

Significant progress has been made in text-to-video generation through the use of powerful generative models and large-scale internet data. However, substantial challenges remain in precisely controlling individual concepts within the generated video, such as the motion and appearance of specific characters and the movement of viewpoints. In this work, we propose a novel paradigm that generates each concept in 3D representation separately and then composes them with priors from Large Language Models (LLM) and 2D diffusion models. Specifically, given an input textual prompt, our scheme consists of three stages: 1) We leverage LLM as the director to first decompose the complex query into several sub-prompts that indicate individual concepts within the video~(\textit{e.g.}, scene, objects, motions), then we let LLM to invoke pre-trained expert models to obtain corresponding 3D…

Dense 3D Coordinate Code Prior Guidance for High-Fidelity Face Swapping and Face Reenactment

Anni Tang, Han Xue, Jun Ling +2 more

Computer Vision and Pattern Recognition · 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021) · · cited 1× · DOI ↗ · more like this ·

In face synthesis tasks, commonly used 2D face representations (e.g. 2D landmarks, segmentation maps, etc.) are usually sparse and discontinuous. To combat these shortcomings, we utilize a dense and continuous representation, named Projected Normalized Coordinate Code (PNCC), as the guidance and develop a PNCC-Spatio-Normalization (PSN) method to achieve face synthesis regarding arbitrary head poses and expressions. Based on PSN, we provide an effective framework for face reenactment and face swapping task. To ensure a harmonious and seamless face swapping, a simple yet effective Appearance-Blending Module (ABM) is proposed to fit the synthesized face to the target face. Our method is subject-agnostic and can be applied to any pair of faces without extra fine-tuning. Both qualitative and quantitative experiments are conducted to demonstrate the superiority of the proposed method in…

A Hybrid Scheme for Face Video Compression

Anni Tang, Z.M. Zhang, Chen Zhu +3 more

Computer Vision and Pattern Recognition · ACM Transactions on Multimedia Computing Communications and Applications · · cited 0× · DOI ↗ · more like this ·

With the rapid development of social media, the amount of face video data has grown rapidly, making face video compression a hot research topic. Traditional video coding techniques do not discriminate video content and compress all videos in the same way, while talking head video compression should have more potential. Existing generative compression methods mostly adopt static reference frames, resulting in a decrease in fidelity caused by dynamic background or large pose change. In this article, we propose a hybrid compression scheme for face videos which combines traditional coding with generative compression. On the one hand, we sample and encode key frames with traditional codecs to provide dynamic reference frames which contain real-time background and motion information. On the other hand, we devise a deep video generation model to synthesize smooth video frames according to the…

Anti-corruption Efforts, Corruption Perception and Public Evaluation of Local Governments in China

Zhou Zhou, Yangxuan Zhou, Meng Yuan +1 more

Sociology and Political Science · The China Quarterly · · cited 0× · DOI ↗ · more like this ·

Abstract China has taken significant steps to combat corruption since the 18th National Congress of the Chinese Communist Party (CCP). However, whether and how the anti-corruption efforts influence the public's evaluation of local government performance remain understudied. Using multiple data sources, including panel survey data taken from the China Family Panel Studies from 2010 to 2018, this research examines how anti-corruption efforts improve evaluations of local government performance by reducing public perception of existing corruption. Additional analysis reveals that anti-corruption efforts reduce perceived corruption primarily when the public trusts officials or has had positive experiences with them. The positive impact on local government evaluations has been more pronounced in provinces with high levels of pre-existing corruption and since the 18th CCP National Congress in…

VidTok: A Versatile and Open-Source Video Tokenizer

Anni Tang, Tianyu He, Junliang Guo +3 more

cs.CV · · cited 0× · arXiv ↗ · more like this ·

Encoding video content into compact latent tokens has become a fundamental step in video generation and understanding, driven by the need to address the inherent redundancy in pixel-level representations. Consequently, there is a growing demand for high-performance, open-source video tokenizers as video-centric research gains prominence. We introduce VidTok, a versatile video tokenizer that delivers state-of-the-art performance in both continuous and discrete tokenizations. VidTok incorporates several key advancements over existing approaches: 1) model architecture such as convolutional layers and up/downsampling modules; 2) to address the training instability and codebook collapse commonly associated with conventional Vector Quantization (VQ), we integrate Finite Scalar Quantization (FSQ) into discrete video tokenization; 3) improved training strategies, including a two-stage training…

OmniVTLA: Vision-Tactile-Language-Action Models with Semantic-Aligned Tactile Sensing

Zhengxue Cheng, Yiqian Zhang, Anni Tang +5 more

cs.RO · · arXiv ↗ · more like this ·

Recent vision-language-action (VLA) models build upon vision-language foundations, and have achieved promising results and exhibit the possibility of task generalization in robot manipulation. However, due to the heterogeneity of tactile sensors and the difficulty of acquiring tactile data, current VLA models significantly overlook the importance of tactile perception and fail in contact-rich tasks. To address this issue, this paper proposes OmniVTLA, a novel architecture involving tactile sensing. Specifically, our contributions are threefold. First, our OmniVTLA features a dual-path tactile encoder framework. This framework enhances tactile perception across diverse vision-based and force-based tactile sensors by using a pretrained vision transformer (ViT) and a semantically-aligned tactile ViT (SA-ViT). Second, we introduce ObjTac, a comprehensive force-based tactile dataset…