arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Computer Science · Computer Graphics and Computer-Aided Design

The Challenges of Relighting from Multi-View Observations

Siersleben, Daniel, Ortiz-Cayon, Rodrigo, Istenic, Klemen, Tomoto, Yusuke, Schaub-Meyer, Simone

· Eurographics

See on the map

Abstract

High-quality geometry reconstruction from multi-view images with subsequent appearance decomposition into the physical shading components could enable a seamless integration of neural reconstructions into the modern rendering workflow. While 3D reconstruction techniques have steadily improved, the task of inverse rendering by decomposing an appearance into lighting effects and material properties remains fundamentally ill-posed and highly ambiguous. We show that current state-of-the-art inverse rendering approaches fail to accurately recover material properties, significantly impacting relighting quality. Furthermore, we demonstrate that existing evaluation methods, which rely on image-based metrics, do not adequately capture the reconstruction quality in novel lighting conditions. Our findings illustrate the dependence of current systems on simplified setups with predefined illumination, which are necessary to reliably disentangle light and material contributions and to ultimately achieve convincing relighting.

More like this