收藏切换
Deep learning methods for portrait relighting
收藏切换
Lina Lei1, Jiacheng Zhu2, Chunle Guo1, Jiewen Huang2, Jun Luo2, Chongyi Li1, *
Journal of Image and Graphics | 2026, 31(10) : 3482 - 3501
Less
收藏切换
Journal of Image and Graphics | 2026, 31(10): 3482-3501
• Review •
Deep learning methods for portrait relighting
Full
Lina Lei1, Jiacheng Zhu2, Chunle Guo1, Jiewen Huang2, Jun Luo2, Chongyi Li1, *
Affiliations
  • 1 School of Computer Science,Nankai University,Tianjin 300350,China
  • 2 OPPO Guangdong Mobile Communications Co.,LTD.,Dongguan 523860,China
Published: 2026-10-16 doi: 10.11834/jig.250303
Outline
收藏切换

Portrait relighting is an important research direction bridging computer vision and computer graphics. Its core objective is to simulate the lighting effects under different real-world light sources by adjusting the lighting conditions of the person in an image to optimize the visual representation of images,improve the presentation of facial features,enhance the aesthetic effect and meet the needs of artistic creation. This technology has extensive application value,particularly in scenarios such as postprocessing of portrait photography,virtual reality,augmented reality,film special effects production,digital entertainment,human-computer interaction,intelligent monitoring and video conferencing. Relighting can not only restore or enhance the quality of an image but also add a new emotional atmosphere to it,making it more visually impactful and communicative. Therefore,how to relight human image efficiently,accurately,and naturally has been a key issue that researchers at home and abroad have focused on in recent years. The traditional methods for relighting portraits,such as the spherical harmonics illumination model,the Lambertian reflectance model,the Phong model,and the mixed model of ambient light and directional light,are mostly based on physical modeling ideas. These methods attempt to depict the interrelationships among lighting,materials,and geometry by establishing clear mathematical and physical models. However,real-world human faces are highly complex,with variations in skin color,uneven reflection attributes,and fine and irregular geometric structures,which cause traditional physical modeling-based methods encounter challenges such as large modeling errors,high computational complexity,and poor generalization ability when applied to real images. Especially in single-image input scenarios that lack multiperspective or depth information,traditional methods often fail to generate satisfactory relighting effects. With the rapid development of deep learning technology,the image relighting method based on neural networks has gradually emerged and quickly become the mainstream research paradigm. Compared with traditional methods,deep learning methods can automatically learn the complex nonlinear mapping relationship between illumination and facial features in images through large-scale data training,thereby achieving highly realistic and detail-rich relighting effects without explicitly modeling the physical process of illumination. This type of method usually has great flexibility,expressiveness,and scalability and can adapt to various complex lighting conditions and changes in face poses,showing enhanced robustness in practical applications. To sort out the development context and research status of portrait relighting technology comprehensively,this study first starts from the research background,systematically summarizes the development process and key technological breakthroughs in this field,and clarifies its important value in various practical scenarios. This study also collates and compares the current mainstream portrait datasets and illumination datasets,covering synthetic data (such as face data generated on the basis of rendering engines) and real collected data (such as real illumination images obtained using multilight source shooting systems). These datasets provide fundamental support for the training and evaluation of algorithm models. However,problems such as insufficient data scale,imperfect illumination labels,and uneven distribution of races and genders must be addressed urgently. The method review section classifies the current portrait relighting methods based on deep learning into three major categories: 1) the first type is methods combined with physical models,which attempts to establish a connection between the powerful representational ability of neural networks and the physical interpretability of traditional illumination models. For example,by introducing learnable spherical harmonic parameters,normal graphs,and reflection models,it guides the network to learn intermediate representations with physical relevance,making the generated results more realistic and controllable. 2) The second type is methods based on image transformation,which mainly relies on image-to-image mapping networks,such as U-Net structure,generative adversarial network,and self-attention mechanism. It directly learns the mapping relationship between the input image and the target illumination image and has advantages such as end-to-end training,high efficiency,and natural visual effects. However,certain deficiencies in terms of illumination control accuracy and geometric consistency still exist. 3) The third type is methods based on 3D reconstruction,which utilizes face reconstruction technology to first perform 3D reconstruction on the characters in the input image (such as restoring 3DMM parameters,depth maps,or normal maps),then reconstruct the lighting conditions in 3D space,and finally project to generate a 2D image. This approach is usually superior to the pure image conversion method in terms of accuracy and control ability,but it has higher requirements for the quality of the input image and the accuracy of geometric reconstruction,and the computational cost is relatively large. On the basis of a detailed analysis of the above three types of methods,this study also compares and evaluates the advantages and disadvantages of different methods from multiple dimensions such as the realism of the relighting effect,the degree of detail retention,computational complexity,and the generalization ability of the model. Meanwhile,in combination with the requirements of portrait relighting in practical applications,the applicability and robustness of various methods when dealing with complex factors such as different races,postures,the number of light sources and the position of light sources are explored. Finally,considering the current development trends of deep learning and computer graphics,this study looks forward to the future research directions of portrait relighting algorithms. Future research may pay more attention to the lightweight design of the model,real-time processing capabilities,the fusion of multimodal inputs (such as combining semantic information,depth maps,etc.),the improvement of cross-domain transfer capabilities,and more realistic and natural lighting style modeling. Meanwhile,the diversity of the dataset,the accuracy of the labels,and the rationality of the evaluation indicators also become important factors restricting the development of the technology. This study hopes to provide references and inspirations for subsequent research through a systematic review and analysis of portrait relighting technology and promote the further development of this field at the theoretical and application levels. In conclusion,this study aims to provide comprehensive technical references and theoretical support for relevant researchers through a systematic review and in-depth analysis of the existing research methods in the field of portrait relighting and to promote continuous progress in this field in terms of algorithm performance,application breadth,and visual quality.

portrait relighting  /  deep learning  /  physical model  /  image transformation  /  3D reconstruction
雷莉娜, 朱家成, 郭春乐, 黄杰文, 罗俊, 李重仪. 人像重打光深度学习方法研究进展. 中国图象图形学报, 2026 , 31 (10) : 3482 -3501 . DOI: 10.11834/jig.250303
Lina Lei, Jiacheng Zhu, Chunle Guo, Jiewen Huang, Jun Luo, Chongyi Li. Deep learning methods for portrait relighting[J]. Journal of Image and Graphics, 2026 , 31 (10) : 3482 -3501 . DOI: 10.11834/jig.250303
  • National Natural Science Foundation of China(62306153,U23B2011,62176130); Tianjin Municipal Natural Science Foundation(25ZXRGGX00290,24JCJQJC00020)
Year 2026 volume 31 Issue 10
17
7
Cite this Article
BibTeX
Article Info
doi: 10.11834/jig.250303
  • Receive Date:2025-07-02
  • Online Date:2026-10-30
  • Published:2026-10-16
Article Data
Affiliations
History
  • Received:2025-07-02
  • Revised:2026-01-28
Funding
National Natural Science Foundation of China(62306153,U23B2011,62176130); Tianjin Municipal Natural Science Foundation(25ZXRGGX00290,24JCJQJC00020)
Affiliations
    1 School of Computer Science,Nankai University,Tianjin 300350,China
    2 OPPO Guangdong Mobile Communications Co.,LTD.,Dongguan 523860,China
References
Share
https://castjournals.cast.org.cn/joweb/zgtxtxxb/EN/10.11834/jig.250303
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT