Learning to synthesize: robust phase retrieval at low photon counts

Mo Deng; Shuai Li; Alexandre Goy; Iksung Kang; George Barbastathis

doi:10.1038/s41377-020-0267-2

Go to Nature Website

Your Location：

Home >

Browse articles >

Learning to synthesize: robust phase retrieval at low photon counts

Article

- Learning to synthesize: robust phase retrieval at low photon counts
- Light: Science & Applications Vol. 9, Issue 2, Pages: 343-358(2020)
- Affiliations：
  
  1.Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
  2.Sensebrain Technology Limited LLC, 2550 N 1st Street, Suite 300, San Jose, CA 95131, USA
  3.Omnisens SA, Riond Bosson 3, 1110 Morges, VD, Switzerland
  4.Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
  5.Singapore-MIT Alliance for Research and Technology (SMART) Centre, Singapore 117543, Singapore
- Author bio：
  
  Mo Deng (modeng@mit.edu)
- Funds：
- DOI：10.1038/s41377-020-0267-2
  CLC：
- Published：2020，
  
  Published Online：09 March 2020，
  
  Received：29 August 2019，
  
  Revised：04 February 2020，
  
  Accepted：19 February 2020
- Accepted：
Scan QR Code
Deng, M. et al. Learning to synthesize: robust phase retrieval at low photon counts. Light: Science & Applications, 9, 343-358 (2020).
DOI：

Deng, M. et al. Learning to synthesize: robust phase retrieval at low photon counts. Light: Science & Applications, 9, 343-358 (2020). DOI： 10.1038/s41377-020-0267-2.

摘要

Abstract

The quality of inverse problem solutions obtained through deep learning is limited by the nature of the priors learned from examples presented during the training phase. Particularly in the case of quantitative phase retrieval

spatial frequencies that are underrepresented in the training database

most often at the high band

tend to be suppressed in the reconstruction. Ad hoc solutions have been proposed

such as pre-amplifying the high spatial frequencies in the examples; however

while that strategy improves the resolution

it also leads to high-frequency artefacts

as well as low-frequency distortions in the reconstructions. Here

we present a new approach that learns separately how to handle the two frequency bands

low and high

and learns how to synthesize these two bands into full-band reconstructions. We show that this "learning to synthesize" (LS) method yields phase reconstructions of high spatial resolution and without artefacts and that it is resilient to high-noise conditions

e.g.

in the case of very low photon flux. In addition to the problem of quantitative phase retrieval

the LS method is applicable

in principle

to any inverse problem where the forward operator treats different frequency bands unevenly

i.e.

is ill-posed.

关键词

Keywords

references

Marquet, P. et al. Digital holographic microscopy: a noninvasive contrast imaging technique allowing quantitative visualization of living cells with subwavelength axial accuracy.Opt. Lett.30, 468-470 (2005)..

Popescu, G. et al. Diffraction phase microscopy for quantifying cell structure and dynamics.Opt. Lett.31, 775-777 (2006)..

Mayo, S. C. et al. X-ray phase-contrast microscopy and microtomography.Opt. Express11, 2289-2302 (2003)..

Pfeiffer, F. et al. Phase retrieval and differential phase-contrast imaging with low-brilliance X-ray sources.Nat. Phys.2, 258-261 (2006)..

Pan, A. et al. Contrast enhancement in x-ray phase contrast tomography.Opt. Express22, 18020-18026 (2014)..

Holler, M. et al. High-resolution non-destructive three-dimensional imaging of integrated circuits.Nature543, 402-406 (2017)..

Goodman, J. W.&Lawrence, R. W. Digital image formation from electronically detected holograms.Appl. Phys. Lett.11, 77-79 (1967)..

Creath, K. Phase-shifting speckle interferometry.Appl. Opt.24, 3053-3058 (1985)..

Gerchberg, R. W.&Saxton, W. O. A practical algorithm for the determination of phase from image and diffraction plane pictures.Optik35, 237-246 (1972)..

Fienup, J. R. Reconstruction of an object from the modulus of its Fourier transform.Opt. Lett.3, 27-29 (1978)..

Zheng, G. A., Horstmeyer, R.&Yang, C. Wide-field, high-resolution Fourier ptychographic microscopy.Nat. Photonics7, 739-745 (2013)..

Tian, L. et al. Multiplexed coded illumination for Fourier Ptychography with an LED array microscope.Biomed. Opt. Express5, 2376-2389 (2014)..

Teague, M. R. Deterministic phase retrieval: a Green's function solution.J. Optical Soc. Am.73, 1434-1441 (1983)..

Streibl, N. Phase imaging by the transport equation of intensity.Opt. Commun.49, 6-10 (1984)..

Fienup, J. R. Phase retrieval algorithms: a comparison.Appl. Opt.21, 2758-2769 (1982)..

Bauschke, H. H., Combettes, P. L.&Luke, D. R. Phase retrieval, error reduction algorithm, and fienup variants: a view from convex optimization.J. Optical Soc. Am. A19, 1334-1345 (2002)..

Gerchberg, R. W. The lock problem in the Gerchberg-, Saxton algorithm for phase retrieval.Optik74, 91-93 (1986)..

Fienup, J. R.&Wackerman, C. C. Phase-retrieval stagnation problems and solutions.J. Optical Soc. Am. A3, 1897-1907 (1986)..

Gregor, K.&LeCun, Y. Learning fast approximations of sparse coding. InProceedings of the 27th International Conference on Machine Learning. (Omnipress, Haifa, 2010).

Rubinstein, R., Bruckstein, A. M.&Elad, M. Dictionaries for sparse representation modeling.Proc. IEEE98, 1045-1057 (2010)..

Bao, C. L. et al. Dictionary learning for sparse coding: algorithms and convergence analysis.IEEE Trans. Pattern Anal. Mach. Intell.38, 1356-1369 (2016)..

Mardani, M. et al. Recurrent generative adversarial networks for proximal learning and automated compressive image recovery. Preprint at arXiv.org/abs/1711.10046 (2017).

Yang, C. Y., Ma, C.&Yang, M. H. Single-image super-resolution: a benchmark. InProceedings of the 13th European Conference on Computer Vision. (Springer, Zurich, 2014).

Dong, C. et al. Image super-resolution using deep convolutional networks.IEEE Trans. Pattern Anal. Mach. Intell.38, 295-307 (2016)..

Jin, K. H. et al. Deep convolutional neural network for inverse problems in imaging.IEEE Trans. Image Process.26, 4509-4522 (2017)..

McCann, M. T., Jin, K. H.&Unser, M. Convolutional neural networks for inverse problems in imaging: a review.IEEE Signal Process. Mag.34, 85-95 (2017)..

Sinha, A. et al. Lensless computational imaging through deep learning.Optica4, 1117-1125 (2017)..

Li, S.&Barbastathis, G. Spectral pre-modulation of training examples enhances the spatial resolution of the phase extraction neural network (PhENN).Opt. Express26, 29340-29352 (2018)..

Goy, A. et al. Low photon count phase retrieval using deep learning.Phys. Rev. Lett.121, 243902 (2018)..

Rivenson, Y. et al. Phase recovery and holographic image reconstruction using deep learning in neural networks.Light Sci. Appl.7, 17141 (2018)..

Wu, Y. C. et al. Extended depth-of-field in holographic imaging using deep-learning-based autofocusing and phase recovery.Optica5, 704-710 (2018)..

Nguyen, T. et al. Deep learning approach for Fourier ptychography microscopy.Opt. Express26, 26470-26484 (2018)..

Xue, Y. et al. Reliable deep-learning-based phase imaging with uncertainty quantification.Optica6, 618-629 (2019)..

Kamilov, U. S. et al. Learning approach to optical tomography.Optica2, 517-522 (2015)..

Goy, A. et al. High-resolution limited-angle phase tomography of dense layered objects using deep neural networks.Proc. Natl Acad. Sci. USA116, 19848-19856 (2019)..

Jo, Y. et al. Quantitative phase imaging and artificial intelligence: a review.IEEE J. Sel. Top. Quantum Electron.25, 6800914 (2019)..

Barbastathis, G., Ozcan, A.&Situ, G. On the use of deep learning for computational imaging.Optica6, 921-943 (2019)..

Daubechies, I. Orthonormal bases of compactly supported wavelets.Commun. Pure Appl. Math.41, 909-996 (1988)..

Daubechies, I.Ten Lectures on Wavelets.(Pa: Society for Industrial and Applied Mathematics, Philadelphia, 1992).

Coifman, R. R.&Donoho, D. L. Translation-invariant de-noising. InWavelets and Statistics, Vol. 103 (eds. Antoniadis, A.&Oppenheim, G.) (Springer-Verlag, New York, 1995), 120-150.

Strang, G.&Nguyen, T.Wavelets and Filter Banks. 2nd edn. (Wellesley-Cambridge Press, Wellesley, 1996).

Chan, R. H. et al. Wavelet algorithms for high-resolution image reconstruction.SIAM J. Sci. Comput.24, 1408-1432 (2003)..

Daubechies, I., Defrise, M.&De Mol, C. An iterative thresholding algorithm for linear inverse problems with a sparsity constraint.Commun. Pure Appl. Math.57, 1413-1457 (2004)..

Figueiredo, M. A. T.&Nowak, R. D. An EM algorithm for wavelet-based image restoration.IEEE Trans. Image Process.12, 906-916 (2003)..

Mallat, S.A Wavelet Tour of Signal Processing: The Sparse Way. 3rd edn. (Amsterdam: Academic Press, 2008).

Lim, D., Chu, K. K.&Mertz, J. Wide-field fluorescence sectioning with hybrid speckle and uniform-illumination microscopy.Opt. Lett.33, 1819-1821 (2008)..

Mertz, J. Optical sectioning microscopy with planar or structured illumination.Nat. Methods8, 811-819 (2011)..

Bhattacharya, D. et al. Three dimensional HiLo-based structured illumination for a digital scanned laser sheet microscopy (DSLM) in thick tissue imaging.Opt. Express20, 27337-27347 (2012)..

Zhu, Y. H. et al. Low-noise phase imaging by hybrid uniform and structured illumination transport of intensity equation.Opt. Express22, 26696-26711 (2014)..

Han, Y.&Ye, J. C. Framing U-net via deep convolutional framelets: application to sparse-view CT.IEEE Trans. Med. Imaging37, 1418-1429 (2018)..

Ye, J. C., Han, Y.&Cha, E. Deep convolutional framelets: a general deep learning framework for inverse problems.SIAM J. Imaging Sci.11, 991-1048 (2018)..

Pan, J. S. et al. Learning dual convolutional neural networks for low-level vision. InProceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. (IEEE, Salt Lake City, 2018).

Romano, Y., Elad, M.&Milanfar, P. The little engine that could: regularization by denoising (RED).SIAM J. Imaging Sci.10, 1804-1844 (2017)..

Tikhonov, A. N. On the solution of ill-posed problems and the method of regularization.Dokl. Akademii Nauk SSSR151, 501-504 (1963)..

Candès, E. J.&Tao, T. Decoding by linear programming.IEEE Trans. Inf. Theory51, 4203-4215 (2005)..

Candès, E. J., Romberg, J.&Tao, T. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information.IEEE Trans. Inf. Theory52, 489-509 (2006)..

Donoho, D. L. Compressed sensing.IEEE Trans. Inf. Theory52, 1289-1306 (2006)..

Candès, E. J., Romberg, J. K.&Tao, T. Stable signal recovery from incomplete and inaccurate measurements.Commun. Pure Appl. Math.59, 1207-1223 (2006)..

Eldar, Y. C.&Kutyniok, G.Compressed Sensing: Theory and Applications.(Cambridge University Press, Cambridge, 2012).

Elad, M.&Aharon, M. Image denoising via sparse and redundant representations over learned dictionaries.IEEE Trans. Image Process.15, 3736-3745 (2006)..

Aharon, M., Elad, M.&Bruckstein, A. K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation.IEEE Trans. Signal Process.54, 4311-4322 (2006)..

Olshausen, B. A.&Field, D. J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images.Nature381, 607-609 (1996)..

Olshausen, B. A.&Field, D. J. Natural image statistics and efficient coding.Netw. Comput. Neural Syst.7, 333-339 (1996)..

Van Der Schaaf, A.&Van Hateren, J. H. Modelling the power spectra of natural images: statistics and information.Vis. Res.36, 2759-2770 (1996)..

Lewicki, M. S.&Olshausen, B. A. Probabilistic framework for the adaptation and comparison of image codes.J. Optical Soc. Am. A16, 1587-1601 (1999)..

Lewicki, M. S.&Sejnowski, T. J. Learning overcomplete representations.Neural Comput.12, 337-365 (2000)..

Russakovsky, O.et al. ImageNet large scale visual recognition challenge.Int. J. Computer Vis.115, 211-252 (2015)..

Gupta, P. et al. A modified PSNR metric based on hvs for quality assessment of color images. InProceedings of 2011 International Conference on Communication and Industrial Application. (IEEE, Kolkata, 2011).

Wang, Z. et al. Multiscale structural similarity for image quality assessment. InProceedings of the Thirty-Seventh Asilomar Conference on Signals, Systems & Computers. (IEEE, Pacific Grove, 2003).

Wang, Z. et al. Image quality assessment: from error visibility to structural similarity.IEEE Trans. Image Process.13, 600-612 (2004)..

Pearson, K. Contributions to the mathematical theory of evolution. note on reproductive selection.Proc. R. Soc. Lond.59, 300-305 (1896)..

Lee Rodgers, J.&Nicewander, W. A. Thirteen ways to look at the correlation coefficient.Am. Statistician42, 59-66 (1988)..

Li, S. et al. Imaging through glass diffusers using densely connected convolutional networks.Optica5, 803-813(2018)..

Deng, M., Li, S.&Barbastathis, G. Learning to synthesize: splitting and recombining low and high spatial frequencies for image recovery. Preprint at arXiv.org/abs/1811.07945 (2018).

Ronneberger, O., Fischer, P.&Brox, T. U-Net: convolutional networks for biomedical image segmentation. InProceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. (Springer, Munich, 2015).

Bottou, L. Large-scale machine learning with stochastic gradient descent. InProceedings of the 19th International Conference on Computational Statistics. (Springer, Paris, 2010).

Kingma, D. P.&Ba, J. Adam: A method for stochastic optimization. InProceedings of the 3rd International Conference on Learning Representations. (Conference Track Proceedings, San Diego, 2015).

Hinton, G. E. Learning translation invariant recognition in a massively parallel networks. InProceedings of the International Conference on Parallel Architectures and Languages Europe. (Springer-Verlag, Eindhoven, 1987).

Johnson, J., Alahi, A.&Li, F. F. Perceptual losses for real-time style transfer and super-resolution. InProceedings of the 14th European Conference on Computer Vision. (Springer, Amsterdam, 2016).

Ledig, C. et al. Photo-realistic single image super-resolution using a generative adversarial network. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition. (IEEE, Honolulu, 2017).

Li, S., Barbastathis, G.&Goy, A. Analysis of phase extraction neural network (PhENN) performance for lensless quantitative phase imaging.Proc. SPIE 10887, Quantitative Phase Imaging V, 108870T(4 March 2019).

Goy, A. et al. The importance of physical pre-processors for quantitative phase retrieval under extremely low photon counts.Proc. SPIE 10887, Quantitative Phase Imaging V, 108870S(4 March 2019).

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰