Liquid lens based holographic camera for real 3D scene hologram acquisition using end-to-end physical model-driven network

Di Wang; Zhao-Song Li; Yi Zheng; You-Ran Zhao; Chao Liu; Jin-Bo Xu; Yi-Wei Zheng; Qian Huang; Chen-Liang Chang; Da-Wei Zhang; Song-Lin Zhuang; Qiong-Hua Wang

doi:10.1038/s41377-024-01410-8

Your Location：

Home >

Browse articles >

Liquid lens based holographic camera for real 3D scene hologram acquisition using end-to-end physical model-driven network

Original Articles

- Liquid lens based holographic camera for real 3D scene hologram acquisition using end-to-end physical model-driven network
- Light: Science & Applications Vol. 13, Issue 3, Pages: 488-497(2024)
- Affiliations：
  
  1.School of Instrumentation and Optoelectronic Engineering, Beihang University, Beijing 100191, China
  2.School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China
- Author bio：
  
  Qiong-Hua Wang (qionghua@buaa.edu.cn)
- Funds：
- DOI：10.1038/s41377-024-01410-8
  CLC：
- Received：23 November 2023，
  
  Revised：30 January 2024，
  
  Accepted：2024-02-12，
  
  Published Online：29 February 2024，
  
  Published：31 March 2024
- Accepted：
Scan QR Code
Wang, D. et al. Liquid lens based holographic camera for real 3D scene hologram acquisition using end-to-end physical model-driven network.,Light: Science & Applications, 13, 488-497 (2024).
DOI：

Wang, D. et al. Liquid lens based holographic camera for real 3D scene hologram acquisition using end-to-end physical model-driven network.,Light: Science & Applications, 13, 488-497 (2024). DOI： 10.1038/s41377-024-01410-8.

摘要

Abstract

With the development of artificial intelligence

neural network provides unique opportunities for holography

such as high fidelity and dynamic calculation. How to obtain real 3D scene and generate high fidelity hologram in real time is an urgent problem. Here

we propose a liquid lens based holographic camera for real 3D scene hologram acquisition using an end-to-end physical model-driven network (EEPMD-Net). As the core component of the liquid camera

the first 10 mm large aperture electrowetting-based liquid lens is proposed by using specially fabricated solution. The design of the liquid camera ensures that the multi-layers of the real 3D scene can be obtained quickly and with great imaging performance. The EEPMD-Net takes the information of real 3D scene as the input

and uses two new structures of encoder and decoder networks to realize low-noise phase generation. By comparing the intensity information between the reconstructed image after depth fusion and the target scene

the composite loss function is constructed for phase optimization

and the high-fidelity training of hologram with true depth of the 3D scene is realized for the first time. The holographic camera achieves the high-fidelity and fast generation of the hologram of the real 3D scene

and the reconstructed experiment proves that the holographic image has the advantage of low noise. The proposed holographic camera is unique and can be used in 3D display

measurement

encryption and other fields.

关键词

Keywords

references

Gu, M., Zhang, Q. M. & Lamon, S. Nanomaterials for optical data storage. Nature Reviews . Materials 1 , 16070 (2016)..

Huang, Z. Z. et al. Dual-plane coupled phase retrieval for non-prior holographic imaging. PhotoniX 3 , 3 (2022)..

Siegel, N. et al. High-magnification super-resolution FINCH microscopy using birefringent crystal lens interferometers. Nat. Photonics 10 , 802–808 (2016)..

Gao, P. & Yuan, C. J. Resolution enhancement of digital holographic microscopy via synthetic aperture: a review. Light. : Adv. Manuf. 3 , 105–120 (2022)..

Ni, Y. B. et al. Computational spectropolarimetry with a tunable liquid crystal metasurface. eLight 2 , 23 (2022)..

Kim, I. et al. Holographic metasurface gas sensors for instantaneous visual alarms. Sci. Adv. 7 , eabe9943 (2021)..

Utadiya, S. et al. Integrated self-referencing single shot digital holographic microscope and optical tweezer. Light. : Adv. Manuf. 3 , 453–465 (2022)..

Dorrah, A. H. et al. Light sheets for continuous-depth holography and three-dimensional volumetric displays. Nat. Photonics 17 , 427–434 (2023)..

Yang, D. et al. Diffraction-engineered holography: beyond the depth representation limit of holographic displays. Nat. Commun. 13 , 6012 (2022)..

Yu, H. et al. Deep learning-based incoherent holographic camera enabling acquisition of real-world holograms for holographic streaming system. Nat. Commun. 14 , 3534 (2023)..

Blinder, D. et al. The state-of-the-art in computer generated holography for 3D display. Light. : Adv. Manuf. 3 , 572–600 (2022)..

Bianco, V. et al. Strategies for reducing speckle noise in digital holography. Light Sci. Appl. 7 , 48 (2018)..

Pi, D. P., Liu, J. & Wang, Y. T. Review of computer-generated hologram algorithms for color dynamic holographic three-dimensional display. Light Sci. Appl. 11 , 231 (2022)..

Sahin, E. et al. Computer-generated holograms for 3D imaging: a survey. ACM Comput. Surv. 53 , 32 (2021)..

Oh, J., Hugonnet, H. & Park, Y. Non-interferometric stand-alone single-shot holographic camera using reciprocal diffractive imaging. Nat. Commun. 14 , 4870 (2023)..

Situ, G. H. Deep holography. Light. Adv. Manuf. 3 , 278–300 (2022)..

Nehmetallah, G. & Banerjee, P. P. Applications of digital and analog holography in three-dimensional imaging. Adv. Opt. Photonics 4 , 472–553 (2012)..

Wang, F. et al. Phase imaging with an untrained neural network. Light Sci. Appl. 9 , 77 (2020)..

Chen, H. L. et al. Fourier Imager Network (FIN): a deep neural network for hologram reconstruction with superior exte rnal generalization. Light Sci. Appl. 11 , 254 (2022)..

Stetson, K. A. The discovery of holographic interferometry, its development, and applications. Light. : Adv. Manuf. 3 ,349–357 (2022)..

Fratz, M. et al. Digital holography in production: an overview. Light. Adv. Manuf. 2 , 283–295 (2021)..

Dong, L. et al. Adaptive liquid microlenses activated by stimuli-responsive hydrogels. Nature 442 , 551–554 (2006)..

Guo, C. L. et al. Miniscope-LFOV: a large-field-of-view, single-cell-resolution, miniature microscope for wired and wire-free imaging of neural dynamics in freely behaving animals. Sci. Adv. 9 , eadg3918 (2023)..

Liu, C. et al. Continuous optical zoom microscope with extended depth of field and 3D reconstruction. PhotoniX 3 , 20 (2022)..

Liu, C. et al. Real scene acquisition and holographic near-eye display system based on a zoom industrial endoscope. Opt. Express 30 , 33170–33181 (2022)..

Zong, W. J. et al. Miniature two-photon microscopy for enlarged field-of-view, multi-plane, and long-term brain imaging. Nat. Methods 18 , 46–49 (2021)..

Wang, D. et al. Holographic capture and projection system of real object based on tunable zoom lens. PhotoniX 1 , 6 (2020)..

Hsiang, E. L. et al. AR/VR light engines: perspectives and challenges. Adv. Opt. Photonics 14 , 783–861 (2022)..

Liu, C. et al. Tunable liquid lenses: emerging technologies and future perspectives. Laser Photonics Rev. 17 , 2300274 (2023)..

Shi, L., Li, B. C. & Matusik, W. End-to-end learning of 3D phase-only holograms for holographic display. Light Sci. Appl. 11 , 247 (2022)..

Chang, C. L. et al. From picture to 3D hologram: end-to-end learning of real-time 3D photorealistic hologram generation from 2D image input. Opt. Lett. 48 , 851–854 (2023)..

Choi, S. et al. Optimizing image quality for holographic near-eye displays with Michelson Holography. Optica 8 , 143–146 (2021)..

Peng, Y. F. et al. Speckle-free holography with partially coherent light sources and camera-in-the-loop calibration. Sci. Adv. 7 , eabg5040 (2021)..

Shi, L. et al. Towards real-time photorealistic 3D holography with deep neural networks. Nature 591 , 234–239 (2021)..

Peng, Y. F. et al. Neural holography with camera-in-the-loop training. ACM Trans. Graph. 39 , 185 (2020)..

Choi, S. et al. Neural 3D holography: learning accurate wave propagation models for 3D holographic virtual and augmented reality displays. ACM Trans. Graph. 40 , 240 (2021)..

Wang, D. et al. High-quality holographic 3D display system based on virtual splicing of spatial light modulator. ACS Photonics 10 , 2297–2307 (2023)..

Liu, K. X. et al. 4K-DMDNet: diffraction model-driven network for 4K computer-generated holography. Opto-Electron. Adv. 6 , 220135 (2023)..

Shui, X. H. et al. Diffraction model-informed neural network for unsupervised layer-based computer-generated holography. Opt. Express 30 , 44814–44826 (2022)..

Zhang, S. J. et al. End-to-end real-time holographic display based on real-time capture of real scenes. Opt. Lett. 48 , 1850–1853 (2023)..

Chang, C. L. et al. Three-dimensional computer holography enabled from a single 2D image. Opt. Lett. 47 , 2202–2205 (2022)..

Dong, J. Q. et al. High-speed real 3D scene acquisition and 3D holographic reconstruction system based on ultrafast optical axial scanning. Opt. Express 31 , 21721–21730 (2023)..

Wang, D. et al. Color liquid crystal grating-based color holographic 3D display system with large viewing angle. Light Sci. Appl. 13 , 16 (2024)..

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰