Learning-based real-time imaging through dynamic scattering media

Haishan Liu; Fei Wang; Ying Jin; Xianzheng Ma; Siteng Li; Yaoming Bian; Guohai Situ

doi:10.1038/s41377-024-01569-0

Go to Nature Website

Your Location：

Home >

Browse articles >

Learning-based real-time imaging through dynamic scattering media

Original Articles

- Learning-based real-time imaging through dynamic scattering media
- Light: Science & Applications Vol. 13, Issue 10, Pages: 2131-2144(2024)
- Affiliations：
  
  1.Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, Shanghai 201800, China
  2.Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, 100049 Beijing, China
  3.Department of Engineering Science, University of Oxford, Oxford, UK
  4.Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou 310024, China
- Author bio：
  
  Guohai Situ (ghsitu@siom.ac.cn)
- Funds：
- DOI：10.1038/s41377-024-01569-0
  CLC：
- Published：31 October 2024，
  
  Published Online：16 August 2024，
  
  Received：28 April 2024，
  
  Revised：01 August 2024，
  
  Accepted：05 August 2024
- Accepted：
Scan QR Code
Liu, H. S. et al. Learning-based real-time imaging through dynamic scattering media. Light: Science & Applications, 13, 2131-2144 (2024).
DOI：

Liu, H. S. et al. Learning-based real-time imaging through dynamic scattering media. Light: Science & Applications, 13, 2131-2144 (2024). DOI： 10.1038/s41377-024-01569-0.

摘要

Abstract

Imaging through dynamic scattering media is one of the most challenging yet fascinating problems in optics

with applications spanning from biological detection to remote sensing. In this study

we propose a comprehensive learning-based technique that facilitates real-time

non-invasive

incoherent imaging of real-world objects through dense and dynamic scattering media. We conduct extensive experiments

demonstrating the capability of our technique to see through turbid water and natural fog. The experimental results indicate that the proposed technique surpasses existing approaches in numerous aspects and holds significant potential for imaging applications across a broad spectrum of disciplines.

关键词

Keywords

references

Ersoy, O. K.Diffraction, Fourier Optics and Imaging(John Wiley&Sons Inc., 2006).

Ishimaru, A.Wave Propagation and Scattering in Random Media(IEEE Press, 1978).

Mitsuo, T. et al. Holographic 3D Imaging through Random Media: Methodologies and Challenges.Light Adv. Manuf.3, 301–313 (2021)..

Yoo, K. M.&Alfano, R. R. Time-resolved coherent and incoherent components of forward light scattering in random media.Opt. Lett.15, 320–322 (1990)..

Wang, L. et al. Ballistic 2-D imaging through scattering walls using an ultrafast optical Kerr gate.Science253, 769–771 (1991)..

Yoo, K. M., Das, B. B.&Alfano, R. R. Imaging of a translucent object hidden in a highly scattering medium from the early portion of the diffuse component of a transmitted ultrafast laser pulse.Opt. Lett.17, 958–960 (1992)..

Meyer, H. J., Mamani, S.&Alfano, R. R. Femtosecond optical kerr gate in tissues.J. Biophotonics16, e202300099 (2023)..

Ho, P. P.&Alfano, R. R. Optical Kerr effect in liquids.Phys. Rev. A20, 2170–2187 (1979)..

Zhang, Y. Z. et al. Application of short-coherence lensless Fourier-transform digital holography in imaging through diffusive medium.Opt. Commun.286, 56–59 (2013)..

Zimnyakov, D. A.et al. Polarization-sensitive speckle spectroscopy of scattering media beyond the diffusion limit.J. Optical Soc. Am. A21, 59–70 (2004)..

Shi, L. Y. et al. Transmission in near‐infrared optical windows for deep brain imaging.J. Biophotonics9, 38–43 (2016)..

Yoo, K. M., Liu, F.&Alfano, R. R. Imaging through a scattering wall using absorption.Opt. Lett.16, 1068–1070 (1991)..

Shi, L. Y.&Alfano, R. R.Deep Imaging in Tissue and Biomedical Materials: Using Linear and Nonlinear Optical Methods(Pan Stanford, 2017).

Bertolotti, J. et al. Non-invasive imaging through opaque scattering layers.Nature491, 232–234 (2012)..

Katz, O., Small, E.&Silberberg, Y. Looking around corners and through thin turbid layers in real time with scattered incoherent light.Nat. Photonics6, 549–553 (2012)..

Badon, A. et al. Smart optical coherence tomography for ultra-deep imaging through highly scattering media.Sci. Adv.2, e1600370 (2016)..

Luo, Y. et al. Computational imaging without a computer: seeing through random diffusers at the speed of light.eLight2, 4 (2022)..

Satat, G. et al. All photons imaging through volumetric scattering.Sci. Rep.6, 33946 (2016)..

Yang, W. Q., Li, G. W.&Situ, G. H. Imaging through scattering media with the auxiliary of a known reference object.Sci. Rep.8, 9614 (2018)..

Gröger, A. et al. Advantages of holographic imaging through fog.Appl. Opt.62, D68–D76 (2023)..

Tahir, W., Wang, H.&Tian, L. Adaptive 3D descattering with a dynamic synthesis network.Light Sci. Appl.11, 42 (2022)..

Horisaki, R., Takagi, R.&Tanida, J. Learning-based imaging through scattering media.Opt. Express24, 13738–13743 (2016)..

Li, F. Q. et al. Compressive ghost imaging through scattering media with deep learning.Opt. Express28, 17395–17408 (2020)..

Li, Y. Z., Xue, Y. J.&Tian, L. Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media.Optica5, 1181–1190 (2018)..

Li, S. et al. Imaging through glass diffusers using densely connected convolutional networks.Optica5, 803–813 (2018)..

Lyu, M. et al. Learning-based lensless imaging through optically thick scattering media.Adv. Photonics1, 036002 (2019)..

Sanghvi, Y., Kalepu, Y.&Khankhoje, U. K. Embedding deep learning in inverse scattering problems.IEEE Trans. Comput. Imaging6, 46–56 (2020)..

Zheng, S. S. et al. Non-line-of-sight imaging under white-light illumination: a two-step deep learning approach.Opt. Express29, 40091–40105 (2021)..

Zheng, S. S. et al. Incoherent imaging through highly nonstatic and optically thick turbid media based on neural network.Photonics Res.9, B220–B228 (2021)..

Tang, Z. W. et al. DeepSCI: scalable speckle correlation imaging using physics-enhanced deep learning.Opt. Lett.48, 2285–2288 (2023)..

Zhu, S. et al. Imaging through unknown scattering media based on physics-informed learning.Photonics Res.9, B210–B219 (2021)..

Sun, Y. W. et al. Image reconstruction through dynamic scattering media based on deep learning.Opt. Express27, 16032–16046 (2019)..

Wang, D. et al. Non-invasive super-resolution imaging through dynamic scattering media.Nat. Commun.12, 3150 (2021)..

Barbastathis, G., Ozcan, A.&Situ, G. H. On the use of deep learning for computational imaging.Optica6, 921–943 (2019)..

Fenza, G. et al. Data set quality in machine learning: consistency measure based on group decision making.Appl. Soft Comput.106, 107366 (2021)..

Wang, F. et al. Phase imaging with an untrained neural network.Light Sci. Appl.9, 77 (2020)..

Wang, F. et al. Far-field super-resolution ghost imaging with a deep neural network constraint.Light Sci. Appl.11, 1 (2022)..

Chen, Z. Y. et al. Physics-driven deep learning enables temporal compressive coherent diffraction imaging.Optica9, 677–680 (2022)..

Lee, C. et al. Deep learning based on parameterized physical forward model for adaptive holographic imaging with unpaired data.Nat. Mach. Intell.5, 35–45 (2023)..

Shi, Z. et al. ZeroScatter: domain transfer for long distance imaging and vision through scattering media. InProceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(IEEE, 2021), pp. 3476–3486.

Sakaridis, C., Dai,D. X.&Van Gool, L. Semantic foggy scene understanding with synthetic data.Int. J. Comput. Vis.126, 973–992 (2018)..

Popoff, S. M. et al. Coherent control of total transmission of light through disordered media.Phys. Rev. Lett.112, 133903 (2014)..

Deng, J. et al. ImageNet: a large-scale hierarchical image database. InProceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition(IEEE, 2009) pp. 248–255.

Ronneberger, O., Fischer, P.&Brox, T. U-Net: convolutional networks for biomedical image segmentation. InProceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention(Springer, 2015) pp. 234–241.

Oktay, O. et al. Attention U-Net: learning where to look for the pancreas. InProceedings of 2018 International Conference on Medical Imaging with Deep Learning,https://openreview.net/forum?id=Skft7cijMhttps://openreview.net/forum?id=Skft7cijM(2018)..

Liang, J. Y. et al. SwinIR: image restoration using swin transformer. InProceedings of 2021 IEEE/CVF International Conference on Computer Vision Workshops(IEEE, 2021) pp. 1833–1844.

He, K. M., Sun, J.&Tang, X. O. Single image haze removal using dark channel prior.IEEE Trans. Pattern Anal. Mach. Intell.33, 2341–2353 (2011)..

Xie, B., Guo, F.&Cai, Z. X. Improved single image dehazing using dark channel prior and multi-scale retinex. InProceedings of 2010 International Conference on Intelligent System Design and Engineering Application(IEEE, 2010) pp. 848–851.

Fu, X. Y. et al. A retinex-based enhancing approach for single underwater image. InProceedings of 2014 IEEE International Conference on Image Processing (ICIP)(IEEE, 2014) pp. 4572–4576.

Zuiderveld, K. Contrast limited adaptive histogram equalization. InGraphics Gems IV(ed. Heckbert, P. S. ) 474–485 (Academic Press Professional, 1994).

Horn, B. K. P. Determining lightness from an image.Comput. Graph. Image Process.3, 277–299 (1974)..

Bian, Y. M. et al. Passive imaging through dense scattering media.Photonics Res.12, 134–140 (2024)..

Heikenfeld, J. et al.Review paper: a critical review of the present and future prospects for electronic paper.J. Soc. Inf. Disp.19, 129–156 (2011)..

Michels, R., Foschum, F.&Kienle, A. Optical properties of fat emulsions.Opt. Express16, 5907–5925 (2008)..

Agustsson, E.&Timofte, R. N. NTIRE 2017 challenge on single image super-resolution: Dataset and study. InProceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops(IEEE, 2017) pp. 126–135.

Mittal, N. Automatic contrast enhancement of low contrast images using MATLAB.Int. J. Adv. Res. Comput. Sci.3, 333–338 (2012)..

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰