Phase recovery and holographic image reconstruction using deep learning in neural networks

Yair Rivenson; Yibo Zhang; Harun Günaydın; Da Teng; Aydogan Ozcan

doi:10.1038/lsa.2017.141

Go to Nature Website

Your Location：

Home >

Browse articles >

Phase recovery and holographic image reconstruction using deep learning in neural networks

Original Articles

- Phase recovery and holographic image reconstruction using deep learning in neural networks
- Light: Science & Applications Vol. 7, Issue 2, (2018)
- Affiliations：
  
  1.Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095, USA
  2.Bioengineering Department, University of California, Los Angeles, CA 90095, USA
  3.California NanoSystems Institute (CNSI), University of California, Los Angeles, CA 90095, USA
  4.Computer Science Department, University of California, Los Angeles, CA 90095, USA
  5.Department of Surgery, David Geffen School of Medicine, University of California, Los Angeles, CA 90095, USA
- Author bio：
  
  A Ozcan, Email: ozcan@ucla.edu
- Funds：
- DOI：10.1038/lsa.2017.141
  CLC：
- Published：2018，
  
  Published Online：23 February 2018，
  
  Received：21 June 2017，
  
  Revised：05 October 2017，
  
  Accepted：11 October 2017
- Accepted：
Scan QR Code
Rivenson, Y. et al. Phase recovery and holographic image reconstruction using deep learning in neural networks. Light: Science & Applications, 7, 17141 (2018).
DOI：

Rivenson, Y. et al. Phase recovery and holographic image reconstruction using deep learning in neural networks. Light: Science & Applications, 7, 17141 (2018). DOI： 10.1038/lsa.2017.141.

摘要

Abstract

Phase recovery from intensity-only measurements forms the heart of coherent imaging techniques and holography. In this study

we demonstrate that a neural network can learn to perform phase recovery and holographic image reconstruction after appropriate training. This deep learning-based approach provides an entirely new framework to conduct holographic imaging by rapidly eliminating twin-image and self-interference-related spatial artifacts. This neural network-based method is fast to compute and reconstructs phase and amplitude images of the objects using only one hologram

requiring fewer measurements in addition to being computationally faster. We validated this method by reconstructing the phase and amplitude images of various samples

including blood and Pap smears and tissue sections. These results highlight that challenging problems in imaging science can be overcome through machine learning

providing new avenues to design powerful computational imaging systems.

关键词

Keywords

deep learningholographymachine learningneural networksphase recovery

references

Gerchberg RW, Saxton WO. A practical algorithm for the determination of phase from image and diffraction plane pictures.Optik1972;35: 237..

Fienup JR. Reconstruction of an object from the modulus of its Fourier transform.Opt Lett1978;3: 27–29..

Zalevsky Z, Mendlovic D, Dorsch RG. Gerchberg–Saxton algorithm applied in the fractional Fourier or the Fresnel domain.Opt Lett1996;21: 842–844..

Elser V. Solution of the crystallographic phase problem by iterated projections.Acta Crystallogr A2003;59: 201–209..

Luke DR. Relaxed averaged alternating reflections for diffraction imaging.Inverse Probl2005;21: 37–50..

Latychevskaia T, Fink HW. Solution to the twin image problem in holography.Phys Rev Lett2007;98: 233901..

Marchesini S. Invited Article: a unified evaluation of iterative projection algorithms for phase retrieval.Rev Sci Instrum2007;78: 011301..

Quiney HM, Williams GJ, Nugent KA. Non-iterative solution of the phase retrieval problem using a single diffraction measurement.Opt Express2008;16: 6896–6903..

Brady DJ, Choi K, Marks DL, Horisaki R, Lim S. Compressive holography.Opt Express2009;17: 13040–13049..

Szameit A, Shechtman Y, Osherovich E, Bullkich E, Sidorenko Pet al. Sparsity-based single-shot subwavelength coherent diffractive imaging.Nat Mater2012;11: 455–459..

Candès EJ, Eldar YC, Strohmer T, Voroninski V. Phase retrieval via matrix completion.SIAM J Imaging Sci2013;6: 199–225..

Rodriguez JA, Xu R, Chen CC, Zou YF, Miao JW. Oversampling smoothness: an effective algorithm for phase retrieval of noisy diffraction intensities.J Appl Crystallogr2013;46: 312–318..

Rivenson Y, Aviv (Shalev) M, Weiss A, Panet H, Zalevsky Z. Digital resampling diversity sparsity constrained-wavefield reconstruction using single-magnitude image.Opt Lett2015;40: 1842–1845..

Gabor D. A new microscopic principle.Nature1948;161: 777–778..

Faulkner HML, Rodenburg JM. Movable aperture lensless transmission microscopy: a novel phase retrieval algorithm.Phys Rev Lett2004;93: 023903..

Dierolf M, Menzel A, Thibault P, Schneider P, Kewish CMet al. Ptychographic X-ray computed tomography at the nanoscale.Nature2010;467: 436–439..

Zheng GA, Horstmeyer R, Yang C. Wide-field, high-resolution Fourier ptychographic microscopy.Nat Photonics2013;7: 739–745..

Tian L, Waller L. 3D intensity and phase imaging from light field measurements in an LED array microscope.Optica2015;2: 104–111..

Misell DL. An examination of an iterative method for the solution of the phase problem in optics and electron optics: I. Test calculations.J Phys D Appl Phys1973;6: 2200–2216..

Teague MR. Deterministic phase retrieval: a Green's function solution.J Opt Soc Am1983;73: 1434–1441..

Paganin D, Barty A, McMahon PJ, Nugent KA. Quantitative phase-amplitude microscopy. Ⅲ. The effects of noise.J Microsc2004;214: 51–61..

Greenbaum A, Ozcan A. Maskless imaging of dense samples using pixel super-resolution based multi-height lensfree on-chip microscopy.Opt Express2012;20: 3129–3143..

Rivenson Y, Wu Y, Wang H, Zhang Y, Feizi Aet al. Sparsity-based multi-height phase recovery in holographic microscopy.Sci Rep2016;6: 37862..

Wang H, Göröcs Z, Luo W, Zhang Y, Rivenson Yet al. Computational out-of-focus imaging increases the space–bandwidth product in lensbased coherent microscopy.Optica2016;3: 1422–1429..

Ferraro P, Miccio L, Grilli S, Paturzo M, De Nicola Set al. Quantitative Phase Microscopy of microstructures with extended measurementrange and correction of chromatic aberrations by multiwavelength digital holography.Opt Express2007;15: 14591–14600..

Luo W, Zhang YB, Feizi A, Göröcs Z, Ozcan A. Pixel super-resolution using wavelength scanning.Light Sci Appl2016;5: e16060 doi:10.1038/lsa.2016.60..

Gonsalves RA. Phase retrieval and diversity in adaptive optics.Opt Eng1982;21: 215829..

Eisebitt S, Lüning J, Schlotter WF, Lörgen M, Hellwig Oet al. Lensless imaging of magnetic nanostructures by X-ray spectro-holography.Nature2004;432: 885–888..

Rosen J, Brooker G. Non-scanning motionless fluorescence three-dimensional holographic microscopy.Nat Photonics2008;2: 190–195..

Marchesini S, Boutet S, Sakdinawat AE, Bogan MJ, Bajt Set al. Massively parallel X-ray holography.Nat Photonics2008;2: 560–563..

Popescu G, Ikeda T, Dasari RR, Feld MS. Diffraction phase microscopy for quantifying cell structure and dynamics.Opt Lett2006;31: 775–777..

Coppola G, Di Caprio G, Gioffré M, Puglisi R, Balduzzi Det al. Digital self-referencing quantitative phase microscopy by wavefront folding in holographic image reconstruction.Opt Lett2010;35: 3390–3392..

Wang Z, Millet L, Mir M, Ding HF, Unarunotai Set al. Spatial light interference microscopy (SLIM).Opt Express2011;19: 1016–1026..

Rivenson Y, Katz B, Kelner R, Rosen J. Single channel in-line multimodal digital holography.Opt Lett2013;38: 4719–4722..

Shechtman Y, Eldar YC, Cohen O, Chapman HN, Miao JWet al. Phase retrieval with application to optical imaging: a contemporary overview.IEEE Signal Process Mag2015;32: 87–109..

Kelner R, Rosen J. Methods of single-channel digital holography for three-dimensional imaging.IEEE Trans Ind Inform2016;12: 220–230..

Zuo JM, Vartanyants I, Gao M, Zhang R, Nagahara LA. Atomic resolution imaging of a carbon nanotube from diffraction intensities.Science2003;300: 1419–1421..

Song CY, Jiang HD, Mancuso A, Amirbekian B, Peng Let al. Quantitative imaging of single, unstained viruses with coherent X rays.Phys Rev Lett2008;101: 158101..

Miao JW, Ishikawa T, Shen Q, Earnest T. Extending X-ray crystallography to allow the imaging of noncrystalline materials, cells, and single protein complexes.Annu Rev Phys Chem2008;59: 387–410..

Loh ND, Hampton CY, Martin AV, Starodub D, Sierra RGet al. Fractal morphology, imaging and mass spectrometry of single aerosol particles in flight.Nature2012;486: 513–517..

LeCun Y, Bengio Y, Hinton G. Deep learning.Nature2015;521: 436–444..

Schmidhuber J. Deep learning in neural networks: An overview.Neural Netw2015;61: 85–117..

Dong C, Loy CC, He KM, Tang XO. Image super-resolution using deep convolutional networks.IEEE Trans Pattern Anal Mach Intell2016;38: 295–307..

Rivenson Y, Gorocs Z, Gunaydin H, Zhang YB, Wang HDet al. Deep learning microscopy.Optica2017;4: 1437–1443..

Jin KH, McCann MT, Froustey E, Unser M. Deep convolutional neural network for inverse problems in imaging.IEEE Trans Image Process2017;26: 4509–4522..

Wang SS, Su ZH, Ying L, Peng X, Zhu Set al. Accelerating magnetic resonance imaging via deep learning. Proceedings of the 13th International Symposium on Biomedical Imaging (ISBI); 13-16 April 2016; Prague, Czech Republic. IEEE: Prague, Czech Republic 2016 doi:10.1109/ISBI.2016.7493320http://dx.doi.org/10.1109/ISBI.2016.7493320..

Antholzer S, Haltmeier M, Schwab J. Deep learning for photoacoustic tomography from sparse data. arXiv: 1704.04587, 2017.

Jo Y, Park S, Jung J, Yoon J, Joo Het al. Holographic deep learning for rapid optical screening of anthrax spores.Sci Adv2017;3: e1700606..

Sinha A, Lee J, Li S, Barbastathis G. Lensless computational imaging through deep learning. arXiv: 1702.08516, 2017.

Bartels M, Krenkel M, Haber J, Wilke RN, Salditt T. X-ray holographic imaging of hydratedbiological cells in solution.Phys Rev Lett2015;114: 048103..

McNulty I, Kirz J, Jacobsen C, Anderson EH, Howells MRet al. High-resolution imaging by Fourier transform X-ray holography.Science1992;256: 1009–1012..

Greenbaum A, Zhang YB, Feizi A, Chung PL, Luo Wet al. Wide-field computational imaging of pathology slides using lens-free on-chip microscopy.Sci Transl Med2014;6: 267ra175–267ra175..

Nowlan SJ, Hinton GE. Simplifying neural networks by soft weight-sharing.Neural Comput1992;4: 473–493..

Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity.IEEE Trans Image Process2004;13: 600–612..

Lohmann A. Optische einseitenbandübertragung angewandt auf das Gabor-Mikroskop.Opt Acta1956;3: 97–99..

Leith EN, Upatnieks J. Reconstructed wavefronts and communication theory.J Opt Soc Am1962;52: 1123–1130..

Goodman JW.Introduction to Fourier Optics. 3rd edn.Roberts and Company Publishers: Greenwood Village, Colorado. 2005.

cuFFT. NVIDIA Developer 2012. Available athttps://developer.nvidia.com/cuffthttps://developer.nvidia.com/cufft(accessed on 9th April 2017) (The content in the link is not NVIDIA Developer).

Thrust-Parallel Algorithms Library. Available athttps://thrust.github.io/https://thrust.github.io/(accessed and 9th April 2017) (The content in the link is not NVIDIA Developer).

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Label-free 3D computational imaging of spermatozoon locomotion, head spin and flagellum beating over a large volume

Related Author

Mustafa Ugur Daloglu

Wei Luo

Faizan Shabbir

Francis Lin

Kevin Kim

Inje Lee

Jia-Qi Jiang

Wen-Jun Cai

Related Institution

Chemistry and Biochemistry Department, University of California

Department of Physics and Astronomy, University of California

Department of Mathematics, University of California

⁰