arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Charles Loop

31 papers · 2000–2025

2000: 12001: 02002: 02003: 12004: 02005: 12006: 12007: 02008: 12009: 22010: 32011: 12012: 12013: 42014: 12015: 12016: 22017: 02018: 02019: 12020: 22021: 12022: 32023: 22024: 12025: 1
2000peak 4 in 20132025

Frequent co-authors

Papers

showing 1–25 of 31 RSS

Holoportation

Sergio Orts‐Escolano, Christoph Rhemann, Sean Fanello +9 more

Computer Vision and Pattern Recognition · · cited 649× · DOI ↗ · more like this ·

We present an end-to-end system for augmented and virtual reality telepresence, called Holoportation. Our system demonstrates high-quality, real-time 3D reconstructions of an entire space, including people, furniture and objects, using a set of new depth cameras. These 3D models can also be transmitted in real-time to remote users. This allows users wearing virtual or augmented reality displays to see, hear and interact with remote participants in 3D, almost as if they were present in the same physical space. From an audio-visual perspective, communicating and interacting with remote users edges closer to face-to-face communication. This paper describes the Holoportation technical system in full, its key interactive capabilities, the application scenarios it enables, and an initial qualitative study of using this new communication medium.

Computing rectifying homographies for stereo vision

Charles Loop, Zhengyou Zhang

Computer Vision and Pattern Recognition · · cited 357× · DOI ↗ · more like this ·

Image rectification is the process of applying a pair of 2D projective transforms, or homographies, to a pair of images whose epipolar geometry is known so that epipolar lines in the original images map to horizontally aligned lines in the transformed images. We propose a novel technique for image rectification based on geometrically well defined criteria such that image distortion due to rectification is minimized. This is achieved by decomposing each homography into a specialized projective transform, a similarity transform, followed by a shearing transform. The effect of image distortion at each stage is carefully considered.

Real-time GPU rendering of piecewise algebraic surfaces

Charles Loop, Jim Blinn

Computer Graphics and Computer-Aided Design · ACM Transactions on Graphics · · cited 97× · DOI ↗ · more like this ·

We consider the problem of real-time GPU rendering of algebraic surfaces defined by Bézier tetrahedra. These surfaces are rendered directly in terms of their polynomial representations, as opposed to a collection of approximating triangles, thereby eliminating tessellation artifacts and reducing memory usage. A key step in such algorithms is the computation of univariate polynomial coefficients at each pixel; real roots of this polynomial correspond to possibly visible points on the surface. Our approach leverages the strengths of GPU computation and is highly efficient. Furthermore, we compute these coefficients in Bernstein form to maximize the stability of root finding, and to provide shader instances with an early exit test based on the sign of these coefficients. Solving for roots is done using analytic techniques that map well to a SIMD architecture, but limits us to fourth order…

Resolution independent curve rendering using programmable graphics hardware

Charles Loop, Jim Blinn

Computer Graphics and Computer-Aided Design · ACM Transactions on Graphics · · cited 94× · DOI ↗ · more like this ·

We present a method for resolution independent rendering of paths and bounded regions, defined by quadratic and cubic spline curves, that leverages the parallelism of programmable graphics hardware to achieve high performance. A simple implicit equation for a parametric curve is found in a space that can be thought of as an analog to texture space. The image of a curve's Bézier control points are found in this space and assigned to the control points as texture coordinates. When the triangle(s) corresponding to the Bézier curve control hull are rendered, a pixel shader program evaluates the implicit equation for a pixel's interpolated texture coordinates to determine an inside/outside test for the curve. We extend our technique to handle anti-aliasing of boundaries. We also construct a vector image from mosaics of triangulated Bézier control points and show how to deform such images to…

Fast Ray Sorting and Breadth‐First Packet Traversal for GPU Ray Tracing

Kirill Garanzha, Charles Loop

Computer Graphics and Computer-Aided Design · Computer Graphics Forum · · cited 78× · DOI ↗ · more like this ·

Abstract We present a novel approach to ray tracing execution on commodity graphics hardware using CUDA. We decompose a standard ray tracing algorithm into several data‐parallel stages that are mapped efficiently to the massively parallel architecture of modern GPUs. These stages include: ray sorting into coherent packets, creation of frustums for packets, breadth‐first frustum traversal through a bounding volume hierarchy for the scene, and localized ray‐primitive intersections. We utilize the well known parallel primitives scan and segmented scan in order to process irregular data structures, to remove the need for a stack, and to minimize branch divergence in all stages. Our ray sorting stage is based on applying hash values to individual rays, ray stream compression, sorting and decompression. Our breadth‐first BVH traversal is based on parallel frustum‐bounding box intersection…

Real-time high-resolution sparse voxelization with application to image-based modeling

Charles Loop, Cha Zhang, Zhengyou Zhang

Computer Vision and Pattern Recognition · · cited 72× · DOI ↗ · more like this ·

We present a system for real-time, high-resolution, sparse voxelization of an image-based surface model. Our approach consists of a coarse-to-fine voxel representation and a collection of parallel processing steps. Voxels are stored as a list of unsigned integer triples. An oracle kernel decides, for each voxel in parallel, whether to keep or cull its voxel from the list based on an image consistency criterion of its projection across cameras. After a prefix sum scan, kept voxels are subdivided and the process repeats until projected voxels are pixel size. These voxels are drawn to a render target and shaded as a weighted combination of their projections into a set of calibrated RGB images. We apply this technique to the problem of smooth visual hull reconstruction of human subjects based on a set of live image streams. We demonstrate that human upper body shapes can be reconstructed to…

Analytic displacement mapping using hardware tessellation

Matthias Nießner, Charles Loop

Computer Graphics and Computer-Aided Design · ACM Transactions on Graphics · · cited 41× · DOI ↗ · more like this ·

Displacement mapping is ideal for modern GPUs since it enables high-frequency geometric surface detail on models with low memory I/O. However, problems such as texture seams, normal recomputation, and undersampling artifacts have limited its adoption. We provide a comprehensive solution to these problems by introducing a smooth analytic displacement function. Coefficients are stored in a GPU-friendly tile-based texture format, and a multiresolution mip hierarchy of this function is formed. We propose a novel level-of-detail scheme by computing per-vertex adaptive tessellation factors and select the appropriate prefiltered mip levels of the displacement function. Our method obviates the need for a precomputed normal map since normals are directly derived from the displacements. Thus, we are able to perform authoring and rendering simultaneously without typical displacement map extraction…

RTMV: A Ray-Traced Multi-View Synthetic Dataset for Novel View Synthesis

Jonathan Tremblay, Moustafa Meshry, Alex Evans +9 more

cs.CV · · cited 37× · arXiv ↗ · more like this ·

We present a large-scale synthetic dataset for novel view synthesis consisting of ~300k images rendered from nearly 2000 complex scenes using high-quality ray tracing at high resolution (1600 x 1600 pixels). The dataset is orders of magnitude larger than existing synthetic datasets for novel view synthesis, thus providing a large unified benchmark for both training and evaluation. Using 4 distinct sources of high-quality 3D meshes, the scenes of our dataset exhibit challenging variations in camera views, lighting, shape, materials, and textures. Because our dataset is too large for existing methods to process, we propose Sparse Voxel Light Field (SVLF), an efficient voxel-based light field approach for novel view synthesis that achieves comparable performance to NeRF on synthetic data, while being an order of magnitude faster to train and two orders of magnitude faster to render. SVLF…

Real‐Time Rendering Techniques with Hardware Tessellation

Matthias Nießner, Benjamin Keinert, Matthew Fisher +3 more

Computer Graphics and Computer-Aided Design · Computer Graphics Forum · · cited 32× · DOI ↗ · more like this ·

Abstract Graphics hardware has progressively been optimized to render more triangles with increasingly flexible shading. For highly detailed geometry, interactive applications restricted themselves to performing transforms on fixed geometry, since they could not incur the cost required to generate and transfer smooth or displaced geometry to the GPU at render time. As a result of recent advances in graphics hardware, in particular the GPU tessellation unit, complex geometry can now be generated on the fly within the GPU's rendering pipeline. This has enabled the generation and displacement of smooth parametric surfaces in real‐time applications. However, many well‐established approaches in offline rendering are not directly transferable due to the limited tessellation patterns or the parallel execution model of the tessellation stage. In this survey, we provide an overview of recent…

State of the Art Report on Real-time Rendering with Hardware Tessellation

Henry Schäfer, Matthias Nießner, Benjamin Keinert +2 more

Computer Graphics and Computer-Aided Design · Eurographics · · cited 22× · DOI ↗ · more like this ·

For a long time, GPUs have primarily been optimized to render more and more triangles with increasingly flexible shading. However, scene data itself has typically been generated on the CPU and then uploaded to GPU memory. Therefore, widely used techniques that generate geometry at render time on demand for the rendering of smooth and displaced surfaces were not applicable to interactive applications. As a result of recent advances in graphics hardware, in particular the GPU tessellation unit's ability to overcome this limitation, complex geometry can now be generated within the GPU's rendering pipeline on the fly. GPU hardware tessellation enables the generation of smooth parametric surfaces or application of displacement mapping in real-time applications. However, many well-established approaches in offline rendering are not directly transferable, due to the limited tessellation…

A Closed-Form Bayesian Fusion Equation Using Occupancy Probabilities

Charles Loop, Qin Cai, Sergio Orts‐Escolano +1 more

Computer Vision and Pattern Recognition · · cited 21× · DOI ↗ · more like this ·

We present a new mathematical framework for multi-view surface reconstruction from a set of calibrated color and depth images. We estimate the occupancy probability of points in space along sight rays, and combine these estimates using a normalized product derived from Bayes' rule. The advantage of this approach is that the free space constraint is a natural consequence of the formulation, and not a separate logical operation. We present a single closed form implicit expression for the reconstructed surface in terms of the image data and camera projections, making analytic properties such as surface normals not only easy to compute, but exact. This expression can be efficiently evaluated on the GPU, making it ideal for high performance real-time applications, such as live human body capture for immersive telepresence.

Patch-Based Occlusion Culling for Hardware Tessellation

Matthias Nießner, Charles Loop

Computer Graphics and Computer-Aided Design · · cited 11× · OpenAlex ↗ · more like this ·

We present an occlusion culling algorithm that leverages the unique characteristics of patch prim- itives within the hardware tessellation pipeline. That is, unseen patches are costly to compute, but easy to nd screen space bounds for. Our algorithm uses the well known hierarchical Z-buer approach, but we present a novel variant that uses temporal coherence to maintain lists of visible and occluded patches. Patches may be an- imated and have applied displacement maps. We also allow traditional triangle mesh geometry to serve as oc- cluders. This makes our scheme ideally suited for patch based articulated character models, moving within a polygonal environment.

Sparse Voxels Rasterization: Real-time High-fidelity Radiance Field Rendering

Cheng Sun, Jaesung Choe, Charles Loop +2 more

cs.CV · · cited 9× · arXiv ↗ · more like this ·

We propose an efficient radiance field rendering algorithm that incorporates a rasterization process on adaptive sparse voxels without neural networks or 3D Gaussians. There are two key contributions coupled with the proposed system. The first is to adaptively and explicitly allocate sparse voxels to different levels of detail within scenes, faithfully reproducing scene details with $65536^3$ grid resolution while achieving high rendering frame rates. Second, we customize a rasterizer for efficient adaptive sparse voxels rendering. We render voxels in the correct depth order by using ray direction-dependent Morton ordering, which avoids the well-known popping artifact found in Gaussian splatting. Our method improves the previous neural-free voxel model by over 4db PSNR and more than 10x FPS speedup, achieving state-of-the-art comparable novel-view synthesis results. Additionally, our…

Displacement-Invariant Cost Computation for Stereo Matching

Yiran Zhong, Charles Loop, Wonmin Byeon +7 more

Computer Vision and Pattern Recognition · International Journal of Computer Vision · · cited 9× · DOI ↗ · more like this ·

Abstract Although deep learning-based methods have dominated stereo matching leaderboards by yielding unprecedented disparity accuracy, their inference time is typically slow, i.e. , less than 4 FPS for a pair of 540p images. The main reason is that the leading methods employ time-consuming 3D convolutions applied to a 4D feature volume. A common way to speed up the computation is to downsample the feature volume, but this loses high-frequency details. To overcome these challenges, we propose a displacement-invariant cost computation module to compute the matching costs without needing a 4D feature volume. Rather, costs are computed by applying the same 2D convolution network on each disparity-shifted feature map pair independently. Unlike previous 2D convolution-based methods that simply perform context mapping between inputs and disparity maps, our proposed approach learns to match…

Real-time Collision Detection for Dynamic Hardware Tessellated Objects

Matthias Nießner, Christian Siegl, Henry Schäfer +1 more

Computer Vision and Pattern Recognition · Eurographics · · cited 8× · DOI ↗ · more like this ·

We present a novel method for real-time collision detection of patch based, displacement mapped objects using hardware tessellation. Our method supports fully animated, dynamically tessellated objects and runs entirely on the GPU. In order to determine a collision between two objects, we first find the intersecting volume of the corresponding object oriented bounding boxes. Next, patches of both objects are tested for inclusion within this volume. All possibly colliding patches are then voxelized into a uniform grid of single bit voxels. Finally, the resulting voxelization is used to detect collisions. Testing two moderately complex models containing thousands of patches can be done in less than a millisecond making our approach ideally suited for real-time games.

Managing Adjacency in Triangular Meshes

Charles Loop

Computer Graphics and Computer-Aided Design · · cited 8× · OpenAlex ↗ · more like this ·

The problem of efficiently accessing and maintaining adjacency information for triangulations over general surface domains is addressed. Rapid access to adjacent vertices, edges, and triangles is an important aspect of multiresolution techniques, from subdivision surfaces to mesh simplification. Novel data structures and algorithms for the construction, manipulation, and traversal of triangulations suitable for a dynamic multiresolution framework are presented.

Displacement-Invariant Cost Computation for Efficient Stereo Matching

Yiran Zhong, Charles Loop, Wonmin Byeon +7 more

cs.CV · · cited 7× · arXiv ↗ · more like this ·

Although deep learning-based methods have dominated stereo matching leaderboards by yielding unprecedented disparity accuracy, their inference time is typically slow, on the order of seconds for a pair of 540p images. The main reason is that the leading methods employ time-consuming 3D convolutions applied to a 4D feature volume. A common way to speed up the computation is to downsample the feature volume, but this loses high-frequency details. To overcome these challenges, we propose a \emph{displacement-invariant cost computation module} to compute the matching costs without needing a 4D feature volume. Rather, costs are computed by applying the same 2D convolution network on each disparity-shifted feature map pair independently. Unlike previous 2D convolution-based methods that simply perform context mapping between inputs and disparity maps, our proposed approach learns to match…

Improving Deep Stereo Network Generalization with Geometric Priors

Jialiang Wang, Varun Jampani, Deqing Sun +3 more

cs.CV · · cited 7× · arXiv ↗ · more like this ·

End-to-end deep learning methods have advanced stereo vision in recent years and obtained excellent results when the training and test data are similar. However, large datasets of diverse real-world scenes with dense ground truth are difficult to obtain and currently not publicly available to the research community. As a result, many algorithms rely on small real-world datasets of similar scenes or synthetic datasets, but end-to-end algorithms trained on such datasets often generalize poorly to different images that arise in real-world applications. As a step towards addressing this problem, we propose to incorporate prior knowledge of scene geometry into an end-to-end stereo network to help networks generalize better. For a given network, we explicitly add a gradient-domain smoothness prior and occlusion reasoning into the network training, while the architecture remains unchanged…

Effective Back-Patch Culling for Hardware Tessellation

Charles Loop, Matthias Nießner, Christian Eisenacher

Computer Graphics and Computer-Aided Design · Eurographics · · cited 7× · DOI ↗ · more like this ·

When rendering objects with hardware tessellation, back-facing patches should be culled as early as possible to avoid unnecessary surface evaluations, and setup costs for the tessellator and rasterizer. For dynamic objects the popular cone-of-normals approach is usually approximated using tangent and bitangent cones. This is faster to compute, but less effective. We present a novel approach using the Bézier convex hull of the parametric tangent plane. It is much more accurate, and by operating in clip space we are able to reduce the computational cost significantly. As our algorithm vectorizes well, we observe comparable test times with increased cull-rates.

Data-parallel Micropolygon Rasterization

Christian Eisenacher, Charles Loop

Computer Graphics and Computer-Aided Design · Eurographics · · cited 7× · DOI ↗ · more like this ·

Abstract We implement a tile based sort-middle rasterizer in CUDA and study its performance characteristics when used as a backend for adaptive tessellation down to micropolygons. Tessellation and bucketing map very well to the data-parallel paradigm of CUDA, and the majority of time is spent with rasterization. Despite this, our fastest implementation is able to reach 30-50% of the hardware rasterization performance of an Nvidia GTX 280. Overall we are able to rasterize 4 M textured and Phong shaded microquads into a 1600x1200 framebuffer at 10-12 fps.Abstract We implement a tile based sort-middle rasterizer in CUDA and study its performance characteristics when used as a backend for adaptive tessellation down to micropolygons. Tessellation and bucketing map very well to the data-parallel paradigm of CUDA, and the majority of time is spent with rasterization. Despite this, our fastest…

Real-time Patch-Based Sort-Middle Rendering on Massively Parallel Hardware

Charles Loop, Christian Eisenacher

Computer Graphics and Computer-Aided Design · · cited 7× · OpenAlex ↗ · more like this ·

Recently, sort-middle triangle rasterization, implemented as software on a manycore GPU with vector units (Larabee), has been proposed as an alternative to hardware rasterization. The main reasoning is, that only a fraction of the time per frame is spent sorting and rasterizing triangles. However is this still a valid argument in the context of geometry amplification when the number of primitives increases quickly? A REYES like approach, sorting parametric patches instead, could avoid many of the problems with tiny triangles. To demonstrate that software rasterization with geometry amplification can work in real-time, we implement a tile based sort-middle rasterizer in CUDA and analyze its behavior: First we adaptively subdivide rational bicubic B´ezier patches. Then we sort those into buckets, and for each bucket we dice, grid-shade, and rasterize the micropolygons into the…

RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Control

Zhenggang Tang, Balakumar Sundaralingam, Jonathan Tremblay +6 more

cs.RO · · cited 6× · arXiv ↗ · more like this ·

We present a system for collision-free control of a robot manipulator that uses only RGB views of the world. Perceptual input of a tabletop scene is provided by multiple images of an RGB camera (without depth) that is either handheld or mounted on the robot end effector. A NeRF-like process is used to reconstruct the 3D geometry of the scene, from which the Euclidean full signed distance function (ESDF) is computed. A model predictive control algorithm is then used to control the manipulator to reach a desired pose while avoiding obstacles in the ESDF. We show results on a real dataset collected and annotated in our lab.

Fast Monocular Scene Reconstruction with Global-Sparse Local-Dense Grids

Wei Dong, Chris Choy, Charles Loop +3 more

cs.CV · · cited 1× · arXiv ↗ · more like this ·

Indoor scene reconstruction from monocular images has long been sought after by augmented reality and robotics developers. Recent advances in neural field representations and monocular priors have led to remarkable results in scene-level surface reconstructions. The reliance on Multilayer Perceptrons (MLP), however, significantly limits speed in training and rendering. In this work, we propose to directly use signed distance function (SDF) in sparse voxel block grids for fast and accurate scene reconstruction without MLPs. Our globally sparse and locally dense data structure exploits surfaces' spatial sparsity, enables cache-friendly queries, and allows direct extensions to multi-modal data such as color and semantic labels. To apply this representation to monocular scene reconstruction, we develop a scale calibration algorithm for fast geometric initialization from monocular depth…

NRMVS: Non-Rigid Multi-View Stereo

Matthias Innmann, Kihwan Kim, Jinwei Gu +4 more

cs.CV · · cited 1× · arXiv ↗ · more like this ·

Scene reconstruction from unorganized RGB images is an important task in many computer vision applications. Multi-view Stereo (MVS) is a common solution in photogrammetry applications for the dense reconstruction of a static scene. The static scene assumption, however, limits the general applicability of MVS algorithms, as many day-to-day scenes undergo non-rigid motion, e.g., clothes, faces, or human bodies. In this paper, we open up a new challenging direction: dense 3D reconstruction of scenes with non-rigid changes observed from arbitrary, sparse, and wide-baseline views. We formulate the problem as a joint optimization of deformation and depth estimation, using deformation graphs as the underlying representation. We propose a new sparse 3D to 2D matching technique, together with a dense patch-match evaluation scheme to estimate deformation and depth with photometric consistency. We…

Resolution-independent surface rendering using programmable graphics hardware

Charles T Loop

Computer Graphics and Computer-Aided Design · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · · cited 0× · OpenAlex ↗ · more like this ·

Surfaces defined by a Bezier tetrahedron, and in particular quadric surfaces, are rendered on programmable graphics hardware. Pixels are rendered through triangular sides of the tetrahedra and locations on the shapes, as well as surface normals for lighting evaluations, are computed using pixel shader computations. Additionally, vertex shaders are used to aid interpolation over a small number of values as input to the pixel shaders. Through this, rendering of the surfaces is performed independently of viewing resolution, allowing for advanced level-of-detail management. By individually rendering tetrahedrally-defined surfaces which together form complex shapes, the complex shapes can be rendered in their entirety.