Phase imaging with an untrained neural network

Fei Wang; Yaoming Bian; Haichao Wang; Meng Lyu; Giancarlo Pedrini; Wolfgang Osten; George Barbastathis; Guohai Situ

doi:10.1038/s41377-020-0302-3

Go to Nature Website

Your Location：

Home >

Browse articles >

Phase imaging with an untrained neural network

Letters

- Phase imaging with an untrained neural network
- Light: Science & Applications Vol. 9, Issue 3, Pages: 499-505(2020)
- Affiliations：
  
  1.Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, 201800 Shanghai, China
  2.Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, 100049 Beijing, China
  3.Institut für Technische Optik, Universität Stuttgart, Pfaffenwaldring 9, 70569 Stuttgart, Germany
  4.Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139-4301, USA
  5.Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, 310024 Hangzhou, China
- Author bio：
  
  Guohai Situ (ghsitu@siom.ac.cn)
- Funds：
- DOI：10.1038/s41377-020-0302-3
  CLC：
- Published：2020，
  
  Published Online：06 May 2020，
  
  Received：15 November 2019，
  
  Revised：17 March 2020，
  
  Accepted：23 March 2020
- Accepted：
Scan QR Code
Wang, F. et al. Phase imaging with an untrained neural network. Light: Science & Applications, 9, 499-505 (2020).
DOI：

Wang, F. et al. Phase imaging with an untrained neural network. Light: Science & Applications, 9, 499-505 (2020). DOI： 10.1038/s41377-020-0302-3.

摘要

Abstract

Most of the neural networks proposed so far for computational imaging (CI) in optics employ a supervised training strategy

and thus need a large training set to optimize their weights and biases. Setting aside the requirements of environmental and system stability during many hours of data acquisition

in many practical applications

it is unlikely to be possible to obtain sufficient numbers of ground-truth images for training. Here

we propose to overcome this limitation by incorporating into a conventional deep neural network a complete physical model that represents the process of image formation. The most significant advantage of the resulting physics-enhanced deep neural network (PhysenNet) is that it can be used without training beforehand

thus eliminating the need for tens of thousands of labeled data. We take single-beam phase imaging as an example for demonstration. We experimentally show that one needs only to feed PhysenNet a single diffraction pattern of a phase object

and it can automatically optimize the network and eventually produce the object phase through the interplay between the neural network and the physical model. This opens up a new paradigm of neural network design

in which the concept of incorporating a physical model into a neural network can be generalized to solve many other CI problems.

关键词

Keywords

references

Barbastathis, G., Ozcan, A.&Situ, G. On the use of deep learning for computational imaging.Optica6, 921–943 (2019)..

Kamilov, U. S. et al. Learning approach to optical tomography.Optica2, 517–522 (2015)..

Lyu, M. et al. Deep-learning-based ghost imaging.Sci. Rep.7, 17865 (2017)..

Wang, F. et al. Learning from simulation: an end-to-end deep-learning approach for computational ghost imaging.Opt. Express27, 25560–25572 (2019)..

Ren, Z. B., Xu, Z. M.&Lam, E. Y. Learning-based nonparametric autofocusing for digital holography.Optica5, 337–344 (2018)..

Wang, H., Lyu, M.&Situ, G. eHoloNet: a learning-based end-to-end approach for in-line digital holographic reconstruction.Opt. Express26, 22603–22614 (2018)..

Rivenson, Y. et al. Phase recovery and holographic image reconstruction using deep learning in neural networks.Light Sci. Appl.7, 17141 (2018)..

Lyu, M. et al. Learning-based lensless imaging through optically thick scattering media.Adv. Photonics1, 036002 (2019)..

Li, Y. Z., Xue, Y. J.&Tian, L. Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media.Optica5, 1181–1190 (2018)..

Li, S. et al. Imaging through glass diffusers using densely connected convolutional networks.Optica5, 803–813 (2018)..

Wu, G. et al. Artificial neural network approaches for fluorescence lifetime imaging techniques.Opt. Lett.41, 2561–2564 (2016)..

Goy, A. et al. Low photon count phase retrieval using deep learning.Phys. Rev. Lett.121, 243902 (2018)..

Sinha, A. et al. Lensless computational imaging through deep learning.Optica4, 1117–1125 (2017)..

Li, X. et al. Quantitative phase imaging via a cGAN network with dual intensity images captured under centrosymmetric illumination.Opt. Lett.44, 2879–2882 (2019)..

Xue, Y. J. et al. Reliable deep-learning-based phase imaging with uncertainty quantification.Optica6, 618–629 (2019)..

Wang, K. Q. et al. One-step robust deep learning phase unwrapping.Opt. Express27, 15100–15115 (2019)..

Feng, S. J. et al. Fringe pattern analysis using deep learning.Adv. Photonics1, 025001 (2019)..

Goodfellow, I., Bengio, Y.&Courville, A.Deep Learning775 (MIT Press, Cambridge, 2016).

Ronneberger, O., Fischer, P.&Brox, T. U-Net: convolutional networks for biomedical image segmentation. InProc. 18th International Conference on Medical Image Computing and Computer-Assisted Intervention234–241 (Springer, Munich, 2015).

Lempitsky, V., Vedaldi, A.&Ulyanov, D. Deep image prior. InProc. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition9446–9454 (IEEE, Salt Lake City, 2018).

Anirudh, R et al. An unsupervised approach to solving inverse problems using generative adversarial networks. Preprint athttps://arxiv.org/pdf/1805.07281.pdfhttps://arxiv.org/pdf/1805.07281.pdf(2018).

Liu, J. M. et al. Image restoration using total variation regularized deep image prior. InICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)7715–7719 (IEEE, Brighton, 2019).

Jagatap, G.&Hegde, C. Phase retrieval using untrained neural network priors. InNeurIPS 2019 Workshop on Solving Inverse Problems with Deep Networks. Vancouver (2019).

Shechtman, Y. et al. Phase retrieval with application to optical imaging: a contemporary overview.IEEE Signal Process. Mag.32, 87–109 (2015)..

Fienup, J. R. Phase retrieval algorithms: a comparison.Appl. Opt.21, 2758–2769 (1982)..

Teague, M. R. Deterministic phase retrieval: a Green's function solution.J. Opt. Soc. Am.73, 1434–1441 (1983)..

Osten, W. et al. Recent advances in digital holography [Invited].Appl. Opt.53, G44–G63 (2014)..

Goodman, J. W.Introduction to Fourier Optics3rd edn (Roberts and Company Publishers, Greenwoood Village, 2005).

Aharon, M., Elad, M.&Bruckstein, A. K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation.IEEE Trans. Signal Process.54, 4311–4322 (2006)..

Rubinstein, R., Bruckstein, A. M.&Elad, M. Dictionaries for sparse representation modeling.Proc. IEEE98, 1045–1057 (2010)..

Kingma, D. P.&Ba, J. Adam: a method for stochastic optimization. Preprint athttps://arxiv.org/abs/1412.6980https://arxiv.org/abs/1412.6980(2014).

Huang, G. B. et al.Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments(University of Massachusetts, 2007).

Zhang, K. et al. Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising.IEEE Trans. Image Process.26, 3142–3155 (2017)..

Mataev, G., Elad, M.&Milanfar, P. DeepRED: deep image prior powered by RED. Preprint athttps://arxiv.org/abs/1903.10176https://arxiv.org/abs/1903.10176(2019).

Zhou, A. et al. Fast and robust misalignment correction of Fourier ptychographic microscopy for full field of view reconstruction.Opt. Express26, 23661–23674 (2018)..

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰