While metalenses focus on imaging, metasurface-based holography, known as metaholography, is a field that creates images through that manipulation of the phase and amplitude of light. Traditionally, iterative algorithms such as the Gerchberg–Saxton algorithm are used for phase retrieval, and meta-atoms are designed to fit the required phase map. AI-based methods have also been proposed for phase retrieval, for example using gradient descent
10 and physics-driven DNNs
141. However, with end-to-end optimization, direct meta-atom selection based on material, and geometric parameters can be directly optimized
142, and additional noise and imperfections from light propagation and sensor readout, can be integrated into the process. While the multiplexing nature of metasurfaces naturally allows for multi-channel holography, providing a path towards multi-dimensional holography, the design of such metasurface becomes burdensome. Yin et al.
143 demonstrated an AI-powered pipeline to create metaholography with amplitude, phase, wavelength, and polarization dependencies. Using meta-atoms with only 2 degrees of freedom, they showed holographic displays with up to 12 channels, including 3 wavelengths, 2 polarization states, and two reconstruction planes. The scalar diffraction kernel is general and could be adapted based on the requirements of the system. The only limitation is that the kernel must be fully differentiable, which is easily achieved through machine learning packages such as PyTorch. Furthermore, Liu et al.
144 moved multiplexed metaholography from free-space to on-chip. By exploiting the properties of geometric and detour phase, they use end-to-end optimization to integrate a Si metasurface onto a Si
3N
4 waveguide. Different holograms appear based on the direction that the light is travelling inside the waveguide. This idea has been pushed further to create on-chip metaholography with up to 18 channels. Finally, waveguide-based holographic displays could play a key role in the future of AR devices, allowing for 3D image construction over a large eyebox. Gopakumar et al.
145 used the co-design of a metasurface waveguide and AI-based holography to create a full-color 3D holographic AR display system with AI trained using data obtained from both the physical metasurface waveguide and camera feedback (
Fig. 6(c)). This system was developed further to create mixed reality displays with a large etendue and high image quality, across an eyebox of 98 mm
146, demonstrating that the mismatch between ideal properties in the system, such as the spatial light modulator, aberrations, and coherence of the source, can be effectively understood and corrected by a well-trained AI model. Finally, Jang et al.
147 demonstrated this with a modeling pipeline that includes both a computationally propagated path and a real-world path through the experimentally fabricated waveguide (
Fig. 6(d)).