Author

Reji Mathew

54 papers · 1997–2024

1997peak 7 in 20152024

Frequent co-authors

Papers

showing 1–25 of 54 RSS

Scalable Coding of Depth Maps With R-D Optimized Embedding

Reji Mathew, David Taubman, Pietro Zanuttigh

Computer Vision and Pattern Recognition · IEEE Transactions on Image Processing · 14 Jan 2013 · cited 46× · DOI ↗ · more like this ·

Recent work on depth map compression has revealed the importance of incorporating a description of discontinuity boundary geometry into the compression scheme. We propose a novel compression strategy for depth maps that incorporates geometry information while achieving the goals of scalability and embedded representation. Our scheme involves two separate image pyramid structures, one for breakpoints and the other for sub-band samples produced by a breakpoint-adaptive transform. Breakpoints capture geometric attributes, and are amenable to scalable coding. We develop a rate-distortion optimization framework for determining the presence and precision of breakpoints in the pyramid representation. We employ a variation of the EBCOT scheme to produce embedded bit-streams for both the breakpoint and sub-band data. Compared to JPEG 2000, our proposed scheme enables the same the scalability…

Quad-Tree Motion Modeling With Leaf Merging

Reji Mathew, David Taubman

Signal Processing · IEEE Transactions on Circuits and Systems for Video Technology · 22 Sep 2010 · cited 38× · DOI ↗ · more like this ·

In this paper, we are concerned with the modeling of motion between frames of a video sequence. Typically, it is not possible to represent the motion between frames by a single model and therefore a quad-tree structure is often employed where smaller, variable size regions or blocks are allowed to take on separate motion models. Previous work into quad-tree representations has demonstrated the sub-optimal performance of quad-trees where the dependency between neighboring leaf nodes with different parents is not exploited. Leaf merging has been proposed to rectify this performance loss as it allows joint coding and optimization of related nodes. In this paper, we describe how the merging step can be incorporated into quad-tree motion representations for a range of motion modeling contexts. In particular, we study the impact of rate-distortion optimized merging for two motion coding…

Detecting New Stable Objects In Surveillance Video

Reji Mathew, Zhenghua Yu, Jian Zhang

Computer Vision and Pattern Recognition · October 2005 · cited 31× · DOI ↗ · more like this ·

We describe a novel method to detect new stable objects in video. This includes detecting new objects that appear in a scene and remain stationary for a period of time. Examples include detecting a dropped bag or a parked car. Our method utilizes the state transition history (or a record of the "life cycle") of individual Gaussian distributions in a Gaussian Mixture Model (GMM) used to model the background. In typical implementations of the GMM, this state transition information is ignored however we show that by observing and retaining the history of state transitions of individual distributions, it is possible to detect long term changes in a scene. In particular we identify changes to the most probable background distribution and impose certain conditions on the characteristics and temporal behavior of this distribution. Results presented in this paper illustrate the success of the…

Layered coding using bitstream decomposition with drift correction

Reji Mathew, J.F. Arnold

Signal Processing · IEEE Transactions on Circuits and Systems for Video Technology · 1997 · cited 24× · DOI ↗ · more like this ·

It is well known that layered video coding is useful both for service interworking and as an aid to error resilience. A major drawback of layered coding is that it invariably results in a reduction in overall coding efficiency for the high quality service. An attractive approach would be one in which the high quality service is coded, and then the resulting bitstream is decomposed in such a way that the lower resolution services can be reconstructed using only a subset of the total generated data. This would mean that there would be no impact on the coding efficiency of the highest quality service. In this paper we demonstrate that the quality of the lower layer when this approach is used is fundamentally limited by drift. It is shown that even a coarsely quantized, low rate correction signal provides a major benefit to the quality of this lower layer service while still not impacting…

Base-Anchored Model for Highly Scalable and Accessible Compression of Multiview Imagery

Dominic Ruefenacht, Aous Thabit Naman, Reji Mathew +1 more

Computer Vision and Pattern Recognition · IEEE Transactions on Image Processing · 24 Jan 2019 · cited 17× · DOI ↗ · more like this ·

We present a compression scheme for multiview imagery that facilitates high scalability and accessibility of the compressed content. Our scheme relies upon constructing at a single base view, a disparity model for a group of views, and then utilizing this base-anchored model to infer disparity at all views belonging to the group. We employ a hierarchical disparity-compensated inter-view transform where the corresponding analysis and synthesis filters are applied along the geometric flows defined by the base-anchored disparity model. The output of this inter-view transform along with the disparity information is subjected to spatial wavelet transforms and embedded block-based coding. Rate-distortion results reveal superior performance to the x.265 anchor chosen by the JPEG Pleno standards activity for the coding of multiview imagery captured by high-density camera arrays.

Temporal Frame Interpolation With Motion-Divergence-Guided Occlusion Handling

Dominic Rüfenacht, Reji Mathew, David Taubman

Computer Vision and Pattern Recognition · IEEE Transactions on Circuits and Systems for Video Technology · February 2018 · cited 15× · DOI ↗ · more like this ·

We present a high-quality temporal frame interpolation (TFI) method that employs piecewise-smooth motion and handles disoccluded regions using the observation that motion discontinuities travel with the foreground object. We derive a “motion discontinuity” likelihood map from the divergence of a motion field between the input frames. Motion which is modeled at the reference frame is mapped to the target frame using a cellular-affine mapping strategy - a process during which regions of disocclusion are readily observed. This information is then used to guide the occlusion-aware, bidirectional FI process. Furthermore, we propose two computationally inexpensive texture optimizations that selectively improve the quality of the interpolated frames in regions around moving objects. The scheme produces very high-quality interpolated frames and outperforms current high-quality state-of-the-art…

Scalable Modeling of Motion and Boundary Geometry With Quad-Tree Node Merging

Reji Mathew, David Taubman

Computer Vision and Pattern Recognition · IEEE Transactions on Circuits and Systems for Video Technology · 17 Jan 2011 · cited 15× · DOI ↗ · more like this ·

Quad-tree structures are often used to model motion between frames of a video sequence. However, a fundamental limitation of the quad-tree structure is that it can only capture horizontal and vertical edge discontinuities at dyadically related locations. To address this limitation, recent work has focused on the introduction of geometry information to nodes of tree structured motion representations. In this paper we explore modeling boundary geometry and motion with separate quad-tree structures; thereby enabling each attribute to be refined separately. Recent work into quad-tree representations has also highlighted the benefits of leaf merging. We extend the leaf merging paradigm to incorporate both geometry and motion attributes. We also explore resolution scalability of the merged dual tree representation and present experimental results which demonstrate both rate-distortion and…

Highly Scalable Coding of Depth Maps with Arc Breakpoints

Reji Mathew, Pietro Zanuttigh, David Taubman

Computer Vision and Pattern Recognition · Padua Research Archive (University of Padova) · April 2012 · cited 14× · DOI ↗ · more like this ·

Recent work highlights the importance of incorporating geometry information into the compression of depth maps. For many applications, features such as resolution scalability and embedded coding are also highly desirable. JPEG 2000 offers these scalability features but suffers from poor compression performance in the vicinity of strong discontinuities. We propose a novel compression strategy for depth maps that incorporates geometry information while retaining the highly scalable coding properties of JPEG 2000. Our scheme involves two separate image pyramid structures, one for arc breakpoints and other for sub-band samples produced by a breakpoint-adaptive transform. Breakpoints capture geometric attributes and are also amenable to scalable coding. We develop an R-D optimization framework for the breakpoint data. We also use a variation of the EBCOT scheme to produce embedded…

Illumination Estimation and Compensation of Low Frame Rate Video Sequences for Wavelet-Based Video Compression

Maryam Haghighat, Reji Mathew, Aous Thabit Naman +1 more

Computer Vision and Pattern Recognition · IEEE Transactions on Image Processing · 20 Mar 2019 · cited 12× · DOI ↗ · more like this ·

In this paper, we are interested in the compression of image sets or video with considerable changes in illumination. We develop a framework to decompose frames into illumination fields and texture in order to achieve sparser representations of frames which is beneficial for compression. Illumination variations or contrast ratio factors among frames are described by a full resolution multiplicative field. First, we propose a Lifting-based Illumination Adaptive Transform (LIAT) framework which incorporates illumination compensation to temporal wavelet transforms. We estimate a full resolution illumination field, taking heed of its spatial sparsity by a rate-distortion (R-D) driven framework. An affine mesh model is also developed as a point of comparison. We find the operational coding cost of the subband frames by modeling a typical t + 2D wavelet video coding system. While our general…

Hierarchical and Polynomial Motion Modeling with Quad-Tree Leaf Merging

Reji Mathew, David Taubman

Signal Processing · October 2006 · cited 12× · DOI ↗ · more like this ·

Recent work into quad-tree representations has commented on the sub-optimal performance of quad-trees due to not exploiting the dependency between neighboring leaf nodes with different parents. Leaf merging therefore has been proposed to rectify this performance loss. In the context of quad-tree motion models, the performance of leaf merging, was recently demonstrated in which an optimally pruned H.264 motion model was taken as the starting point for subsequent merging steps. In this paper we explore the performance of leaf merging over a wider range of conditions by starting with a general tree structure where each node is allowed to represent motion using either polynomial models or a single translational vector. We also explore two cases of motion prediction methods, these being spatial prediction and hierarchical prediction. Experimental results show the benefit of merging across…

High Throughput Block Coding in the HTJ2K Compression Standard

David Taubman, Aous Thabit Naman, Reji Mathew

Computer Vision and Pattern Recognition · 26 Aug 2019 · cited 11× · DOI ↗ · more like this ·

This paper describes the block coding algorithm that underpins the new High Throughput JPEG 2000 (HTJ2K) standard. The objective of HTJ2K is to overcome the computational complexity of the original block coding algorithm, by providing a drop-in replacement that preserves as much of the JPEG 2000 feature set as possible, while allowing reversible transcoding to/from the original format. We show how the new standard achieves these goals, with high coding efficiency, and extremely high throughput in software.

A Novel Motion Field Anchoring Paradigm for Highly Scalable Wavelet-Based Video Coding

Dominic Rüfenacht, Reji Mathew, David Taubman

Computer Vision and Pattern Recognition · IEEE Transactions on Image Processing · 30 Oct 2015 · cited 11× · DOI ↗ · more like this ·

Existing video coders anchor motion fields at frames that are to be predicted. In this paper, we demonstrate how changing the anchoring of motion fields to reference frames has some important advantages over conventional anchoring. We work with piecewise-smooth motion fields, and use breakpoints to signal discontinuities at moving object boundaries. We show how discontinuity information can be used to resolve double mappings arising when motion is warped from reference to target frames. We present an analytical model that allows to determine weights for texture, motion, and breakpoints to guide the rate-allocation for scalable encoding. Compared with the conventional way of anchoring motion fields, the proposed scheme requires fewer bits for the coding of motion; furthermore, the reconstructed video frames contain fewer ghosting artefacts. The experimental results show the superior…

FBCOT: a fast block coding option for JPEG 2000

David Taubman, Aous Thabit Naman, Reji Mathew

Computer Vision and Pattern Recognition · 19 Sep 2017 · cited 9× · DOI ↗ · more like this ·

Based on the EBCOT algorithm, JPEG 2000 finds application in many fields, including high performance scientific, geospatial and video coding applications. Beyond digital cinema, JPEG 2000 is also attractive for low-latency video communications. The main obstacle for some of these applications is the relatively high computational complexity of the block coder, especially at high bit-rates. This paper proposes a drop-in replacement for the JPEG 2000 block coding algorithm, achieving much higher encoding and decoding throughputs, with only modest loss in coding efficiency (typically < 0.5dB). The algorithm provides only limited quality/SNR scalability, but offers truly reversible transcoding to/from any standard JPEG 2000 block bit-stream. The proposed FAST block coder can be used with EBCOT's post-compression RD-optimization methodology, allowing a target compressed bit-rate to be…

Efficient action recognition from compressed depth maps

Jie Miao, Xiaoyi Jia, Reji Mathew +3 more

Computer Vision and Pattern Recognition · 17 Aug 2016 · cited 8× · DOI ↗ · more like this ·

We propose an efficient action recognition scheme based solely on compressed depth maps. Each depth map is coded by a recently proposed scalable encoder that employs multi-scale breakpoints and an adaptive discrete wavelet transform (DWT). DWT coefficients describe smooth variations in depth while breakpoints communicate sharp boundaries. Both of these attributes are extracted from the bit-stream and utilized to construct features which are subject to a classification scheme for human action recognition. By extracting features from the compressed bit-stream computational complexity is significantly reduced thereby making the proposed scheme suitable for real-time applications. A L2-regularized collaborative representation classifier is employed for classification. The proposed scheme is computationally more efficient when compared with conventional approaches. Experimental results on…

Joint estimation of motion and arc breakpoints for scalable compression

Sean I. Young, Reji Mathew, David Taubman

Signal Processing · December 2013 · cited 8× · DOI ↗ · more like this ·

An alternative, graphical model for motion estimation (ME) is proposed for video compression using a dense motion field. Unlike other types of optical flow-based motion analysis techniques, the chief goal of our work is jointly to discover a piece-wise smooth motion field with compression-motivated regularization, where the underlying motion compression scheme has resolution and quality scalability attributes. The graphical model formulation is presented and obtained results are discussed.

Scalable Mesh Representation for Depth from Breakpoint-Adaptive Wavelet Coding

Yue Li, Reji Mathew, David Taubman

Computer Vision and Pattern Recognition · 21 Sep 2020 · cited 7× · DOI ↗ · more like this ·

A highly scalable and compact representation of depth data is required in many applications, and it is especially critical for plenoptic multiview image compression frameworks that use depth information for novel view synthesis and interview prediction. Efficiently coding depth data can be difficult as it contains sharp discontinuities. Breakpoint-adaptive discrete wavelet transforms (BPA-DWT) currently being standardized as part of JPEG 2000 Part-17 extensions have been found suitable for coding spatial media with hard discontinuities. In this paper, we explore a modification to the original BPA-DWT by replacing the traditional constant extrapolation strategy with the newly proposed affine extrapolation for reconstructing depth data in the vicinity of discontinuities. We also present a depth reconstruction scheme that can directly decode the BPA-DWT coefficients and breakpoints onto a…

Hierarchical anchoring of motion fields for fully scalable video coding

Dominic Rüfenacht, Reji Mathew, David Taubman

Signal Processing · October 2014 · cited 7× · DOI ↗ · more like this ·

Traditional video codecs anchor motion fields in the frame that is to be predicted, which is natural in a non-scalable context. In this paper, we propose a hierarchical anchoring of motion fields at reference frames, which allows to “reuse” them at finer temporal levels - a very desirable property for temporal scalability. The main challenge using this approach is that the motion fields need to be warped to the target frames, leading to disocclusions and motion folding in the warped motion fields. We show how to resolve motion folding ambiguities that occur in the vicinity of moving object boundaries by using breakpoint fields that have recently been proposed for the scalable coding of motion. During the motion field warping process, we obtain disocclusion and folding maps on-the-fly, which are used to control the temporal update step of the Haar wavelet. Results on synthetic data show…

Bidirectional hierarchical anchoring of motion fields for scalable video coding

Dominic Rüfenacht, Reji Mathew, David Taubman

Computer Vision and Pattern Recognition · September 2014 · cited 7× · DOI ↗ · more like this ·

The ability to predict motion fields at finer temporal scales from coarser ones is a very desirable property for temporal scalability. This is at best very difficult in current state-of-the-art video codecs (i.e., H.264, HEVC), where motion fields are anchored in the frame that is to be predicted (target frame). In this paper, we propose to anchor motion fields in the reference frames. We show how from only one fully coded motion field at the coarsest temporal level as well as breakpoints which signal discontinuities in the motion field, we are able to reliably predict motion fields used at finer temporal levels. This significantly reduces the cost for coding the motion fields. Results on synthetic data show improved rate-distortion (R-D) performance and superior scalability, when compared to the traditional way of anchoring motion fields.

Using inter-frame dependence history to select intra-refresh blocks

Chen Song Zhao, King Ngi Ngan, J. Zhang +2 more

Signal Processing · Electronics Letters · 24 Oct 2002 · cited 7× · DOI ↗ · more like this ·

To prevent error propagation in predictive video coding, intra-refresh methods are often employed. Here, a new algorithm for macroblock intra-refresh that is based on pixel level inter-frame dependence history is proposed. This new method keeps track of the coding mode of every pixel and refreshes those macroblocks that contain pixels with a long history record. Simulation results show that the new algorithm performs better than the existing methods while maintaining the same bit rate.

Rate-Distortion Driven Decomposition of Multiview Imagery to Diffuse and Specular Components

Maryam Haghighat, Reji Mathew, David Taubman

Computer Vision and Pattern Recognition · IEEE Transactions on Image Processing · 2020 · cited 6× · DOI ↗ · more like this ·

In this work, we propose an overcomplete representation of multiview imagery for the purpose of compression. We present a rate-distortion (R-D) driven approach to decompose multiview datasets into two additive parts which can be interpreted as diffuse and specular content. We choose distinct and different sparsifying transforms for the diffuse and specular components and employ an R-D inspired measure as our optimization cost function to drive the decomposition based solely on compressibility. We first describe a framework which performs data separation in a registered domain to avoid the complexity of warping between views. Then a more comprehensive approach is proposed to separate specular data progressively from coordinates of multiple reference views. Experimental results show a coding gain of up to 0.6 dB for synthetic datasets and up to 0.9 dB for real datasets.

High Throughput JPEG 2000 (HTJ2K): New Algorithms and Opportunities

David Taubman, Aous Thabit Naman, Reji Mathew +1 more

Computer Vision and Pattern Recognition · SMPTE Motion Imaging Journal · 24 Sep 2018 · cited 6× · DOI ↗ · more like this ·

This paper describes a drop-in replacement for the Joint Photographic Experts Group (JPEG) 2000 (J2K) block coder that offers exceptionally high throughput, with a small reduction in coding efficiency, while retaining all features of J2K except for quality scalability. Throughputs on the order of 10 times or more are achievable relative to J2K. We coin the term FAST block coding with optimized truncation (FBCOT) for the overall proposed algorithm. Truly reversible transcoding between J2K and FBCOT bitstreams is supported on a block-by-block basis, enabling systems in which the efficiency and scalability of J2K can be combined with the high throughput benefits of FBCOT. The algorithm forms the basis of a new standard that will be J2K Part-15 (also known as high throughput J2K) and is currently at committee draft status within the International Organization for Standardization (ISO)…

Residue boundary histograms for action recognition in the compressed domain

Jie Miao, Xiangmin Xu, Reji Mathew +1 more

Computer Vision and Pattern Recognition · September 2015 · cited 6× · DOI ↗ · more like this ·

Traditional action recognition approaches are too slow for real-time or large-scale applications. This problem has been tackled by replacing optical flow with motion vectors from the compressed domain. Yet further usage of compressed domain information for action recognition is possible. Discrete cosine transform (DCT) coefficients, which correspond to residue data, represent information which the block based motion vectors fail to capture. We propose a set of residue boundary histograms (RBH) features for action recognition, separating each DCT block into four parts to obtain four small residue maps and then encoding each residue map by histogram-based descriptors to obtain local features. Experimental results on three challenging datasets show that proposed RBH features improve upon motion vector based features significantly. While more than 100× faster, the results are highly…

Bidirectional, occlusion-aware temporal frame interpolation in a highly scalable video setting

Dominic Rüfenacht, Reji Mathew, David Taubman

Computer Vision and Pattern Recognition · May 2015 · cited 6× · DOI ↗ · more like this ·

We present a bidirectional, occlusion-aware temporal frame interpolation (BOA-TFI) scheme that builds upon our recently proposed highly scalable video coding scheme. Unlike previous TFI methods, our scheme attempts to put “correct” information in problematic regions around moving objects. From a “parent” motion field between two existing reference frames, we compose motion from both reference frames to the target frame. These motion fields, together with motion discontinuity information, are then warped to the target frame - a process during which we discover valuable information about disocclusions, which we then use to guide the bidirectional prediction of the interpolated frame. The scheme can be used in any state-of-the-art codec, but is most beneficial if used in conjunction with a highly scalable video coder. Evaluation of the method on synthetic data allows us to shine a light on…

Motion modeling with separate quad-tree structures for geometry and motion

Reji Mathew, David Taubman

Signal Processing · October 2008 · cited 6× · DOI ↗ · more like this ·

Quad-tree structures are often used to model motion between frames of a video sequence. However, a fundamental limitation of the quad-tree structure is that it can only capture horizontal and vertical edge discontinuities at dyadically related locations. To address this limitation recent work has focused on the introduction of geometry information to nodes of tree structured motion representations. In this paper we explore modeling boundary geometry and motion with separate quadtree structures. Recent work into quad-tree representations have also highlighted the benefits of leaf merging. We extend the leaf merging paradigm to incorporate separate tree structures for boundary geometry and motion. To achieve an efficient joint representation we introduce polynomial motion models and piecewise linear boundary geometry to our quad-tree structures. Experimental results show that the approach…

Motion Modeling with Geometry and Quad-tree Leaf Merging

Reji Mathew, David Taubman

Computer Vision and Pattern Recognition · 2007 · cited 6× · DOI ↗ · more like this ·

Quad-tree structures are often used to model motion between frames of a video sequence. However, a fundamental limitation of the quadtree structure is that it can only capture horizontal and vertical edge discontinuities at dyadically related locations. In this paper we seek to address this limitation by introducing geometry information into the nodes of a pruned quad-tree. We start with a typical optimally pruned quad-tree where each node is allowed to model motion. Then for each node in the tree, we consider augmenting the node's motion model with a linear geometry model. Experimental results show that the introduction of geometry information improves the performance of quad-trees in representing motion. Recent work into quad-tree representations have highlighted the benefits of leaf merging. In this paper we extend the leaf merging paradigm to incorporate both geometry and motion…