计算机辅助设计与图形学学报

面向逆渲染的室内场景光源建模

PDF下载

彭思达 ¹⁾ , 陈希 ¹⁾ , 周晨旭 ¹⁾ , 王哲 ²⁾ , 周晓巍 ¹⁾^,^*

计算机辅助设计与图形学学报 | 三维图形计算 2025,37(12): 2052-2060

收起

计算机辅助设计与图形学学报 |三维图形计算 2025 , 37 (12) : 2052 -2060

面向逆渲染的室内场景光源建模

全屏

彭思达¹⁾, 陈希¹⁾, 周晨旭¹⁾, 王哲²⁾, 周晓巍¹⁾^,^*

作者信息

¹⁾浙江大学计算机科学与技术学院　杭州　310058

²⁾阿里巴巴网络技术有限公司　杭州　310058

通讯作者:

（xwzhou@zju.edu.cn）

作者简介:

彭思达（1996—），男，博士，研究员，博士生导师，CCF会员，主要研究方向为计算机图形学、三维计算机视觉

陈希（2000—），男，硕士研究生，主要研究方向为计算机图形学

周晨旭（2002—），男，博士研究生，主要研究方向为三维计算机视觉

王哲（1994—），男，硕士，阿里巴巴研究员，主要研究方向为三维计算机视觉

周晓巍（1986—），男，博士，副教授，博士生导师，CCF会员，论文通信作者，主要研究方向为计算机图形学、三维计算机视觉.

Modeling Emitters in Indoor Scenes for Inverse Rendering

Sida Peng¹⁾, Xi Chen¹⁾, Chenxu Zhou¹⁾, Zhe Wang²⁾, Xiaowei Zhou¹⁾^,^*

Affiliations

¹⁾College of Computer Science and Technology, Zhejiang University, Hangzhou　310058

²⁾Alibaba Group, Hangzhou　310058

出版时间: 2025-12-20 doi: 10.3724/SP.J.1089.2024-00342

文章导航

摘要

收起

近年来的神经逆渲染方法使用神经网络表示物体几何形状和表面材质，通过基于物理的渲染过程从多视图图像学习网络参数. 然而，这些方法通常假设光源位于无限远处，在具有复杂光照条件的室内场景中，这种假设很少成立. 为解决该问题，提出一种基于点云的光照表示方法，用于建模室内场景中的随空间坐标变化的高频光照效果；在输入的多视图图像中检测二维光源掩模，通过三维重建方法得到三维光源点；在基于物理的渲染过程中显式引入所提出的光照表示，提高了渲染模型表达镜面反射效果的能力，有效地减轻逆向渲染过程中的歧义性. 在真实和合成数据集上的实验结果表明，所提方法达到了逆渲染的优异性能，并能够产生逼真的重光照结果.

关键词

逆向渲染 / 重光照 / 神经场景表示

Abstract

收起

Recent neural inverse rendering methods represent object geometry and materials with neural networks and learn network parameters from multi-view images through physically based rendering. However, these methods typically assume that the light source is located at an infinite distance, which seldomly holds in indoor scenarios that exhibit complex illumination. To address this issue, we propose a point cloud-based lighting representation method to model the spatially-varying and high-frequency lighting effects in indoor scenes. Our method first detects 2D light source masks in the input multi-view images, and then obtains a set of emitters through 3D reconstruction algorithm. We explicitly incorporate emitters in the Monte Carlo sampling, which improves the ability to model the specular effects, thus effectively alleviating the ambiguity in the inverse rendering process. Experiments on real and synthetic datasets demonstrate that the proposed method achieves the best performance in inverse rendering and can produce realistic relighting results.

Key words

inverse rendering / relighting / neural scene representation

引用本文

彭思达, 陈希, 周晨旭, 王哲, 周晓巍. 面向逆渲染的室内场景光源建模. 计算机辅助设计与图形学学报, 2025 , 37 (12) : 2052 -2060 . DOI: 10.3724/SP.J.1089.2024-00342

Sida Peng, Xi Chen, Chenxu Zhou, Zhe Wang, Xiaowei Zhou. Modeling Emitters in Indoor Scenes for Inverse Rendering[J]. Journal of Computer-Aided Design & Computer Graphics, 2025 , 37 (12) : 2052 -2060 . DOI: 10.3724/SP.J.1089.2024-00342

基金

收起

国家自然科学基金(62172364)

参考文献

收起

文献

收起

参考文献引证文献

排序方式：

[1]

Debevec P, Hawkins T, Tchou C, et al. Acquiring the reflectance field of a human face[C]//Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM Press, 2000: 145-156

[2]

Guo K W, Lincoln P, Davidson P, et al. The relightables: volumetric performance capture of humans with realistic relighting[J]. ACM Transactions on Graphics, 2019, 38(6): Article No.217

[3]

Nam G, Lee J H, Gutierrez D, et al. Practical SVBRDF acquisition of 3D objects with unstructured flash photography[J]. ACM Transactions on Graphics, 2018, 37(6): Article No.267

[4]

Bi S, Lombardi S, Saito S, et al. Deep relightable appearance models for animatable faces[J]. ACM Transactions on Graphics, 2021, 40(4): Article No.89

[5]

Zhang K, Luan F J, Li Z Q, et al. IRON: inverse rendering by optimizing neural SDFs and materials from photometric images[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2022: 5555-5564

[6]

Zhang X M, Srinivasan P P, Deng B Y, et al. NeRFactor: neural factorization of shape and reflectance under an unknown illumination[J]. ACM Transactions on Graphics, 2021, 40(6): Article No.237

[7]

Zhang K, Luan F J, Wang Q Q, et al. PhySG: inverse rendering with spherical Gaussians for physics-based material editing and relighting[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2021: 5449-5458

[8]

Zhang Y Q, Sun J M, He X Y, et al. Modeling indirect illumination for inverse rendering[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2022: 18622-18631

[9]

Kajiya J T. The rendering equation[C]//Proceedings of the 13th Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM Press, 1986: 143-150

[10]

Munkberg J, Chen W Z, Hasselgren J, et al. Extracting triangular 3D models, materials, and lighting from images[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2022: 8270-8280

[11]

Hasselgren J, Hofmann N, Munkberg J. Shape, light, and material decomposition from images using Monte Carlo rendering and denoising[C]//Proceedings of the 36th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc, 2022: Article No.1661

[12]

Li Z Q, Shafiei M, Ramamoorthi R, et al. Inverse rendering for complex indoor scenes: shape, spatially-varying lighting and SVBRDF from a single image[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2020: 2472-2481

[13]

Gao D, Li X, Dong Y, et al. Deep inverse rendering for high-resolution SVBRDF estimation from an arbitrary number of images[J]. ACM Transactions on Graphics, 2019, 38(4): Article No.134

[14]

Li Z Q, Xu Z X, Ramamoorthi R, et al. Learning to reconstruct shape and spatially-varying reflectance from a single image[J]. ACM Transactions on Graphics, 2018, 37(6): Article No.269

[15]

Guo Y, Smith C, Hašan M, et al. MaterialGAN: reflectance capture using a generative SVBRDF model[J]. ACM Transactions on Graphics, 2020, 39(6): Article No.254

[16]

Ye W C, Chen S, Bao C, et al. IntrinsicNeRF: learning intrinsic neural radiance fields for editable novel view synthesis[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Los Alamitos: IEEE Computer Society Press, 2023: 339-151

[17]

Wang Y F, Holynski A, Zhang X M, et al. SunStage: portrait reconstruction and Relighting using the Sun as a Light Stage[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2023: 20792-20802

[18]

Nimier-David M, Vicini D, Zeltner T, et al. Mitsuba 2: a retargetable forward and inverse renderer[J]. ACM Transactions on Graphics, 2019, 38(6): Article No.203

[19]

Liu S C, Chen W K, Li T Y, et al. Soft rasterizer: a differentiable renderer for image-based 3D reasoning[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Los Alamitos: IEEE Computer Society Press, 2019: 7707-7716

[20]

Chen W Z, Gao J, Ling H, et al. Learning to predict 3D objects with an interpolation-based differentiable renderer[C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc, 2019: Article No.862

[21]

Chen W Z, Litalien J, Gao J, et al. DIB-R++: learning to predict lighting and material with a hybrid differentiable renderer[C]//Proceedings of the 35th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc, 2021: Article No.1749

[22]

Kato H, Ushiku Y, Harada T. Neural 3D mesh renderer[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2018: 3907-3916

[23]

Park J J, Florence P, Straub J, et al. DeepSDF: learning continuous signed distance functions for shape representation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2019: 165-174

[24]

Mildenhall B, Srinivasan P P, Tancik M, et al. NeRF: representing scenes as neural radiance fields for view synthesis[C]//Proceedings of the 16th European Conference on Computer Vision. Heidelberg: Springer, 2020: 405-421

[25]

Barron J T, Mildenhall B, Tancik M, et al. Mip-NeRF: a multiscale representation for anti-aliasing neural radiance fields[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Los Alamitos: IEEE Computer Society Press, 2021: 5835-5844

[26]

Wang P, Liu L J, Liu Y, et al. NeuS: learning neural implicit surfaces by volume rendering for multi-view reconstruction[C]//Proceedings of the 35th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc, 2021: Article No.2081

[27]

Yariv L, Gu J T, Kasten Y, et al. Volume rendering of neural implicit surfaces[C]//Proceedings of the 35th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc, 2021: Article No.367

[28]

Bi S, Xu Z X, Sunkavalli K, et al. Deep reflectance volumes: relightable reconstructions from multi-view photometric images[C]//Proceedings of the 16th European Conference on Computer Vision. Heidelberg: Springer, 2020: 294-311

[29]

Schmitt C, Donné S, Riegler G, et al. On joint estimation of pose, geometry and svBRDF from a handheld scanner[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2020: 3490-3500

[30]

Boss M, Braun R, Jampani V, et al. NeRD: neural reflectance decomposition from image collections[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Los Alamitos: IEEE Computer Society Press, 2021: 12664-12674

[31]

Srinivasan P P, Deng B Y, Zhang X M, et al. NeRV: neural reflectance and visibility fields for relighting and view synthesis[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2021: 7491-7500

[32]

Yao Y, Zhang J Y, Liu J B, et al. NeILF: neural incident light field for physically-based material estimation[C]//Proceedings of the 17th European Conference on Computer Vision. Heidelberg: Springer, 2022: 700-716

[33]

Cook R L, Torrance K E. A reflectance model for computer graphics[J]. ACM Transactions on Graphics, 1982, 1(1): 7-24

[34]

Blender. Blender 4.2 LTS[EB/OL]. [2024-07-02]. https://www.blender.org

[35]

Schönberger J L, Frahm J M. Structure-from-motion revisited[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2016: 4104-4113

[36]

Lowe D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110

[37]

Lepetit V, Moreno-Noguer F, Fua P. EPnP: an accurate O(n) solution to the PnP problem[J]. International Journal of Computer Vision, 2009, 81(2): 155-166

[38]

Laine S, Hellsten J, Karras T, et al. Modular primitives for high-performance differentiable rendering[J]. ACM Transactions on Graphics, 2020, 39(6): Article No.194

[39]

Parker S G, Bigler J, Dietrich A, et al. OptiX: a general purpose ray tracing engine[J]. ACM Transactions on Graphics, 2010, 29(4): Article No.66

[40]

Yu Z H, Peng S Y, Niemeyer M, et al. MonoSDF: exploring monocular geometric cues for neural implicit surface reconstruction[C]//Proceedings of the 36th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc, 2022: Article No.1814

[41]

Zhang R, Isola P, Efros A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2018: 586-595

2025年第37卷第12期

PDF下载

100

引用本文

BibTeX

文章信息

doi: 10.3724/SP.J.1089.2024-00342

接收时间：2024-07-02
首发时间：2026-10-10
出版时间：2025-12-20

补充材料

相关文章

文章信息

作者

出版历史

收稿日期：2024-07-02
修回日期：2024-08-21

基金

国家自然科学基金(62172364)

作者信息

¹⁾浙江大学计算机科学与技术学院　杭州　310058

²⁾阿里巴巴网络技术有限公司　杭州　310058

通讯作者:

（xwzhou@zju.edu.cn）

参考文献

分享链接

https://castjournals.cast.org.cn/joweb/jsjfzsj/CN/10.3724/SP.J.1089.2024-00342

分享至

全文二维码

扫描看全文

引用本文

BibTeX

本文的引用情况

2种不同金属材料的力学参数

科 Family	属数 Number of genus	种数 Number of species	占总种数比例 Percentage of total species (%)	属 Genus	种数 Number of species	占总种数比例 Percentage of total species (%)
鹅膏菌科Amanitaceae	2	11	5.26	鹅膏菌属 Amanita	10	4.78
小菇科 Mycenaceae	2	12	5.74	丝盖伞属 Inocybe	5	2.39
多孔菌科 Polyporaceae	8	14	6.70	蜡蘑属 Laccaria	5	2.39
红菇科 Russulaceae	3	23	11.00	小皮伞属 Marasmius	6	2.87
				小菇属 Mycena	11	5.26
				光柄菇属 Pluteus	5	2.39
				红菇属 Russula	17	8.13
				栓菌属 Trametes	5	2.39

关闭全屏

BibTeX
EndNote
RefWorks
TxT