Far-field super-resolution ghost imaging with a deep neural network constraint

Fei Wang; Chenglong Wang; Mingliang Chen; Wenlin Gong; Yu Zhang; Shensheng Han; Guohai Situ

doi:10.1038/s41377-021-00680-w

Go to Nature Website

Your Location：

Home >

Browse articles >

Far-field super-resolution ghost imaging with a deep neural network constraint

Articles

- Far-field super-resolution ghost imaging with a deep neural network constraint
- Light: Science & Applications Vol. 11, Issue 1, Pages: 27-37(2022)
- Affiliations：
  
  1.Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, Shanghai 201800, China
  2.Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, Beijing 100049, China
  3.Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou 310024, China
  4.CAS Center for Excellence in Ultra-intense Laser Science, Shanghai 201800, China
- Author bio：
  
  Guohai Situ (ghsitu@siom.ac.cn)
- Funds：
- DOI：10.1038/s41377-021-00680-w
  CLC：
- Published：31 January 2022，
  
  Published Online：01 January 2022，
  
  Received：17 May 2021，
  
  Revised：28 October 2021，
  
  Accepted：14 November 2021
- Accepted：
Scan QR Code
Wang, F. et al. Far-field super-resolution ghost imaging with a deep neural network constraint. Light: Science & Applications, 11, 27-37 (2022).
DOI：

Wang, F. et al. Far-field super-resolution ghost imaging with a deep neural network constraint. Light: Science & Applications, 11, 27-37 (2022). DOI： 10.1038/s41377-021-00680-w.

摘要

Abstract

Ghost imaging (GI) facilitates image acquisition under low-light conditions by single-pixel measurements and thus has great potential in applications in various fields ranging from biomedical imaging to remote sensing. However

GI usually requires a large amount of single-pixel samplings in order to reconstruct a high-resolution image

imposing a practical limit for its applications. Here we propose a far-field super-resolution GI technique that incorporates the physical model for GI image formation into a deep neural network. The resulting hybrid neural network does not need to pre-train on any dataset

and allows the reconstruction of a far-field image with the resolution beyond the diffraction limit. Furthermore

the physical model imposes a constraint to the network output

making it effectively interpretable. We experimentally demonstrate the proposed GI technique by imaging a flying drone

and show that it outperforms some other widespread GI techniques in terms of both spatial resolution and sampling ratio. We believe that this study provides a new framework for GI

and paves a way for its practical applications.

关键词

Keywords

references

Pittman, T. B. et al. Optical imaging by means of two-photon quantum entanglement.Phys. Rev. A52, R3429-R3432 (1995)..

Strekalov, D. V. et al. Observation of two-photon "ghost" interference and diffraction.Phys. Rev. Lett.74, 3600-3603 (1995)..

Gatti, A. et al. Ghost imaging with thermal light: comparing entanglement and classical correlation.Phys. Rev. Lett.93, 093602 (2004)..

Cheng, J.&Han, S. S. Incoherent coincidence imaging and its applicability in X-ray diffraction.Phys. Rev. Lett.92, 093903 (2004)..

Erkmen, B. I.&Shapiro, J. H. Ghost imaging: from quantum to classical to computational.Adv. Opt. Photonics2, 405-450 (2010)..

Moreau, P. A. et al. Ghost imaging using optical correlations.Laser Photonics Rev.12, 1700143 (2018)..

Edgar, M. P., Gibson, G. M.&Padgett, M. J. Principles and prospects for single-pixel imaging.Nat. Photonics13, 13-20 (2019)..

Gibson, G. M., Johnson, S. D.&Padgett, M. J. Single-pixel imaging 12 years on: a review.Opt. Express28, 28190-28208 (2020)..

Katz, O., Bromberg, Y.&Silberberg, Y. Compressive ghost imaging.Appl. Phys. Lett.95, 131110 (2009)..

Zhao, C. Q. et al. Ghost imaging lidar via sparsity constraints.Appl. Phys. Lett.101, 141123 (2012)..

Duarte, M. F. et al. Single-pixel imaging via compressive sampling.IEEE Signal Process. Mag.25, 83-91 (2008)..

Ferri, F. et al. High-resolution ghost image and ghost diffraction experiments with thermal light.Phys. Rev. Lett.94, 183602 (2005)..

Gong, W. L.&Han, S. S. High-resolution far-field ghost imaging via sparsity constraint.Sci. Rep.5, 9280 (2015)..

Li, Z. P. et al. Super-resolution single-photon imaging at 8.2 kilometers.Opt. Express28, 4076-4087 (2020)..

Candés, E. J., Romberg, J. K.&Tao, T. Stable signal recovery from incomplete and inaccurate measurements.Commun. Pure Appl. Math.59, 1207-1223 (2006)..

Donoho, D. L. Compressed sensing.IEEE Trans. Inf. Theory52, 1289-1306 (2006)..

Eldar, Y. C.&Kutyniok, G.Compressed Sensing: Theory and Applications(New York: Cambridge University Press, 2012).

Brady, D. J. et al. Compressive holography.Opt. Express17, 13040-13049 (2009)..

Han, S. S. et al. A review of ghost imaging via sparsity constraints.Appl. Sci.8, 1379 (2018)..

Bian, L. H. et al. Experimental comparison of single-pixel imaging algorithms.J. Optical Soc. Am. A35, 78-87 (2018)..

Gong, W. L.&Han, S. S. Experimental investigation of the quality of lensless super-resolution ghost imaging via sparsity constraints.Phys. Lett. A376, 1519-1522 (2012)..

Li, W. W. et al. Single-frame wide-field nanoscopy based on ghost imaging via sparsity constraints.Optica6, 1515-1523 (2019)..

Amitonova, L. V.&de Boer, J. F. Endo-microscopy beyond the Abbe and Nyquist limits.Light. : Sci. Appl.9, 81 (2020)..

Sun, M. J. et al. Single-pixel three-dimensional imaging with time-based depth resolution.Nat. Commun.7, 12010 (2016)..

Goodfellow, I., Bengio, Y.&Courville, A.Deep Learning(Cambridge: MIT Press, 2016).

LeCun, Y., Bengio, Y.&Hinton, G. Deep learning.Nature521, 436-444 (2015)..

Barbastathis, G., Ozcan, A.&Situ, G. On the use of deep learning for computational imaging.Optica6, 921-943 (2019)..

Lyu, M. et al. Deep-learning-based ghost imaging.Sci. Rep.7, 17865 (2017)..

He, Y. C. et al. Ghost imaging based on deep learning.Sci. Rep.8, 6469 (2018)..

Wang, F. et al. Learning from simulation: an end-to-end deep-learning approach for computational ghost imaging.Opt. Express27, 25560-25572 (2019)..

Higham, C. F. et al. Deep learning for real-time single-pixel video.Sci. Rep.8, 2369 (2018)..

Lempitsky, V., Vedaldi, A.&Ulyanov, D. Deep image prior.Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)(Salt Lake City, UT, USA: IEEE, 2018).

Dittmer, S. et al. Regularization by architecture: a deep prior approach for inverse problems.J. Math. Imaging Vis.62, 456-470 (2020)..

Wang, F. et al. Phase imaging with an untrained neural network.Light. : Sci. Appl.9, 77 (2020)..

Bostan, E. et al. Deep phase decoder: self-calibrating phase microscopy with an untrained deep neural network.Optica7, 559-562 (2020)..

Van Veen, D. et al. Compressed sensing with deep image prior and learned regularization. Preprint at arXiv: 1806.06438 (2018).

Heckel, R.&Soltanolkotabi, M. Compressive sensing with un-trained neural networks: gradient descent finds the smoothest approximation.Proceedings of the 37th International Conference on Machine Learning(eds Ⅲ, Hal, D. and Singh, A. ).119, 4149-4158http://proceedings.mlr.press/v119/heckel20a/heckel20a.pdfhttp://proceedings.mlr.press/v119/heckel20a/heckel20a.pdf(PMLR, 2020)..

Zhou, K. C.&Horstmeyer, R. Diffraction tomography with a deep image prior.Opt. Express28, 12872-12896 (2020)..

Gong, W. L.&Han, S. S. A method to improve the visibility of ghost images obtained by thermal light.Phys. Lett. A374, 1005-1008 (2010)..

Ferri, F. et al. Differential ghost imaging.Phys. Rev. Lett.104, 253603 (2010)..

Wang, C. L. et al. Airborne near infrared three-dimensional ghost imaging LiDAR via sparsity constraint.Remote Sens.10, 732 (2018)..

Bromberg, Y., Katz, O.&Silberberg, Y. Ghost imaging with a single detector.Phys. Rev. A79, 053840 (2009)..

Scully, M. O.&Zubairy, M. S.Quantum Optics(Cambridge University Press, Cambridge, 1997).

Deng, M. et al. On the interplay between physical and content priors in deep learning for computational imaging.Opt. Express28, 24152-24170 (2020)..

Zhang, P. L. et al. Improving resolution by the second-order correlation of light fields.Opt. Lett.34, 1222-1224 (2009)..

Wang, W. et al. Gerchberg-Saxton-like ghost imaging.Opt. Express23, 28416-28422 (2015)..

Mangeat, T. et al. Super-resolved live-cell imaging using random illumination microscopy.Cell Rep. Methods1, 100009 (2021)..

Yariv, A.&Yeh, P.Photonics: Optical Electronics in Modern Communications(Oxford: Oxford University Press, 2006).

Healey, G. E.&Kondepudy, R. Radiometric CCD camera calibration and noise estimation.IEEE Trans. Pattern Anal. Mach. Intell.16, 267-276 (1994)..

Goodman, J. W.Statistical Optics(New York: Wiley-Blackwell, 2000).

Howard, A. G. et al. MobileNets: efficient convolutional neural networks for mobile vision applications. Preprint at arXiv: 1704.04861v1 (2017).

Glorot, X.&Bengio, Y. Understanding the difficulty of training deep feedforward neural networks.J. Mach. Learn. Res.9, 249-256 (2010)..

Ruder, S. An overview of gradient descent optimization algorithms. Preprint at arXiv: 1609.04747v2 (2017).

Ronneberger, O., Fischer, P.&Brox, T. U-Net: convolutional networks for biomedical image segmentation.Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention(Munich, Germany: Springer, 2015).

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰