PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning

Yair Rivenson; Tairan Liu; Zhensong Wei; Yibo Zhang; Kevin de Haan; Aydogan Ozcan

doi:10.1038/s41377-019-0129-y

Go to Nature Website

Your Location：

Home >

Browse articles >

PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning

Articles

- PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning
- Light: Science & Applications Vol. 8, Issue 1, Pages: 172-182(2019)
- Affiliations：
  
  1.Electrical and Computer Engineering Department, University of California, Los Angeles, CA, 90095, USA
  2.Bioengineering Department, University of California, Los Angeles, CA, 90095, USA
  3.California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, 90095, USA
  4.Department of Surgery, David Geffen School of Medicine, University of California, Los Angeles, CA, 90095, USA
- Author bio：
  
  Aydogan Ozcan (ozcan@ucla.edu)
- Funds：
- DOI：10.1038/s41377-019-0129-y
  CLC：
- Published：2019，
  
  Published Online：06 February 2019，
  
  Received：23 July 2018，
  
  Revised：05 January 2019，
  
  Accepted：11 January 2019
- Accepted：
Scan QR Code
Rivenson, Y. et al. PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning. Light: Science & Applications, 8, 172-182 (2019).
DOI：

Rivenson, Y. et al. PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning. Light: Science & Applications, 8, 172-182 (2019). DOI： 10.1038/s41377-019-0129-y.

摘要

Abstract

Using a deep neural network

we demonstrate a digital staining technique

which we term PhaseStain

to transform the quantitative phase images (QPI) of label-free tissue sections into images that are equivalent to the brightfield microscopy images of the same samples that are histologically stained. Through pairs of image data (QPI and the corresponding brightfield images

acquired after staining)

we train a generative adversarial network and demonstrate the effectiveness of this virtual-staining approach using sections of human skin

kidney

and liver tissue

matching the brightfield microscopy images of the same samples stained with Hematoxylin and Eosin

Jones' stain

and Masson's trichrome stain

respectively. This digital-staining framework may further strengthen various uses of label-free QPI techniques in pathology applications and biomedical research in general

by eliminating the need for histological staining

reducing sample preparation related costs and saving time. Our results provide a powerful example of some of the unique opportunities created by data-driven image transformations enabled by deep learning.

关键词

Keywords

references

Cuche, E., Marquet, P.&Depeursinge, C. Simultaneous amplitude-contrast and quantitative phase-contrast microscopy by numerical reconstruction of Fresnel off-axis holograms.Appl. Opt.38, 6994-7001 (1999)..

Popescu, G.Quantitative Phase Imaging of Cells and Tissues. (McGraw-Hill, New York, 2011).

Shaked, N. T., Rinehart, M. T.&Wax, A. Dual-interference-channel quantitative-phase microscopy of live cell dynamics.Opt. Lett.34, 767-769 (2009)..

Wang, Z. et al.Spatial light interference microscopy (SLIM).Opt. Express19, 1016-1026 (2011)..

Greenbaum, A. et al. Imaging without lenses: achievements and remaining challenges of wide-field on-chip microscopy.Nat. Methods9, 889-895 (2012)..

Zheng, G. A., Horstmeyer, R.&Yang, C. Wide-field, high-resolution Fourier ptychographic microscopy.Nat. Photonics7, 739-745 (2013)..

Tian, L.&Waller, L. Quantitative differential phase contrast imaging in an LED array microscope.Opt. Express23, 11394-11403 (2015)..

Wang, Z., Tangella, K., Balla, A.&Popescu, G. Tissue refractive index as marker of disease.J. Biomed. Opt.16, 116017 (2011)..

Wang, Z., Ding, H. F.&Popescu, G. Scattering-phase theorem.Opt. Lett.36, 1215-1217 (2011)..

Liu Y., et al. Detecting cancer metastases on gigapixel pathology images. ArXiv: 1703.02442 (2017).

Litjens G., et al. A survey on deep learning in medical image analysis.Med. Image Anal.42: 60-88 (2017)..

Tao, Y. K. et al. Assessment of breast pathologies using nonlinear microscopy.Proc. Natl Acad. Sci. USA111, 15304-15309 (2014)..

Giacomelli, M. G. et al. Virtual hematoxylin and eosin transillumination microscopy using epi-fluorescence imaging.PLoS ONE11, e0159337 (2016)..

Orringer, D. A. et al. Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy.Nat. Biomed. Eng.1, 0027 (2017)..

Rivenson, Y. et al.Deep learning microscopy.Optica4, 1437-1443 (2017)..

Rivenson, Y. et al. Deep learning enhanced mobile-phone microscopy.ACS Photon.5, 2354-2364,https://doi.org/10.1021/acsphotonics.8b00146https://doi.org/10.1021/acsphotonics.8b00146(2018)..

Wang H., et al. Deep learning enables cross-modality super-resolution in fluorescence microscopyNat. Methods16, 103-110 (2019)..

Sinha, A., Lee, J., Li, S.&Barbastathis, G. Lensless computational imaging through deep learning.Optica4, 1117-1125 (2017)..

Rivenson, Y., Zhang, Y. B., Günaydin, H., Teng, D.&Ozcan, A. Phase recovery and holographic image reconstruction using deep learning in neural networks.Light Sci. Appl.7, e17141 (2018)..

Wu, Y. C. et al.Extended depth-of-field in holographic imaging using deep-learning-based autofocusing and phase recovery.Optica5, 704-710 (2018)..

Jo Y., et al.Quantitative phase imaging and artificial intelligence: a review. arXiv: 1806.03982 (2018).

Kamilov, U. et al.Learning approach to optical tomography.Optica2, 517-522 (2015)..

Nguyen T., Xue Y. J., Li Y. Z., Tian L., Nehmetallah G. Deep learning approach to Fourier ptychographic microscopy. arXiv: 1805.00334 (2018).

Boyd N., Jonas E., Babcock H. P., Recht B. DeepLoco: fast 3D localization microscopy using neural networks. bioRxiv: 267096, 2018.https://doi.org/10.1101/267096https://doi.org/10.1101/267096.

Nehme, E., Weiss, L. E., Michaeli, T.&Shechtman, Y. Deep-STORM: super-resolution single-molecule microscopy by deep learning.Optica5, 458-464 (2018)..

Ouyang, W., Aristov, A., Lelek, M., Hao, X.&Zimmer, C. Deep learning massively accelerates super-resolution localization microscopy.Nat. Biotechnol.36, 460-468,https://doi.org/10.1038/nbt.4106https://doi.org/10.1038/nbt.4106(2018)..

Zahavy, T. et al.Deep learning reconstruction of ultrashort pulses.Optica5, 666-673 (2018)..

Rivenson Y., et al. Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning.Nat. Biomed. Eng.(in the press).

Goodfellow I. J., et al.Generative adversarial nets. In Proceedings of the 27th International Conference on Neural Information Processing Systems. (MIT Press: Cambridge, MA, 2014) pp. 2672-2680.

Park, H. S., Rinehart, M. T., Walzer, K. A., Chi, J. T. A.&Wax, A. Automated detection ofP. falciparumusing machine learning algorithms with quantitative phase images of unstained cells.PLoS ONE11, e0163045 (2016)..

Chen, C. L. et al. Deep learning in label-free cell classification.Sci. Rep.6, 21471 (2016)..

Roitshtain, D. et al. Quantitative phase microscopy spatial signatures of cancer cells.Cytometry A91, 482-493 (2017)..

Jo, Y. et al. Holographic deep learning for rapid optical screening of anthrax spores.Sci. Adv.3, e1700606,https://doi.org/10.1101/109108https://doi.org/10.1101/109108(2017)..

Javidi, B. et al. Sickle cell disease diagnosis based on spatio-temporal cell dynamics analysis using 3D printed shearing digital holographic microscopy.Opt. Express26, 13614-13627 (2018)..

Tata, A. et al. Wide-field tissue polarimetry allows efficient localized mass spectrometry imaging of biological tissues.Chem. Sci.7, 2162-2169 (2016)..

Cree, I. A. et al. Guidance for laboratories performing molecular pathology for cancer patients.J. Clin. Pathol.67, 923-931 (2014)..

Patel, P. G. et al. Preparation of formalin-fixed paraffin-embedded tissue cores for both RNA and DNA extraction.J. Vis. Exp.2016, e54299,https://doi.org/10.3791/54299https://doi.org/10.3791/54299(2016)..

Ikeda, T., Popescu, G., Dasari, R. R.&Feld, M. S. Hilbert phase microscopy for investigating fast dynamics in transparent systems.Opt. Lett.30, 1165-1167 (2005)..

Shaked, N. T., Zhu, Y. Z., Badie, N., Bursac, N.&Wax, A. Reflective interferometric chamber for quantitative phase imaging of biological sample dynamics.J. Biomed. Opt.15, 030503 (2010)..

Watanabe, E., Hoshiba, T.&Javidi, B. High-precision microscopic phase imaging without phase unwrapping for cancer cell identification.Opt. Lett.38, 1319-1321 (2013)..

Greenbaum, A. et al. Wide-field computational imaging of pathology slides using lens-free on-chip microscopy.Sci. Transl. Med6, 267ra175 (2014)..

Bishara, W., Su, T. W., Coskun, A. F.&Ozcan, A. Lensfree on-chip microscopy over a wide field-of-view using pixel super-resolution.Opt. Express18, 11181-11191 (2010)..

Luo, W., Zhang, Y. B., Göröcs, Z., Feizi, A.&Ozcan, A. Propagation phasor approach for holographic image reconstruction.Sci. Rep.6, 22738 (2016)..

Farsiu, S., Robinson, M. D., Elad, M.&Milanfar, P. Fast and robust multiframe super resolution.IEEE Trans. Image Process.13, 1327-1344 (2014)..

Zhang, Y. B., Wang, H. D., Wu, Y. C., Tamamitsu, M.&Ozcan, A. Edge sparsity criterion for robust holographic autofocusing.Opt. Lett.42, 3824-3827 (2017)..

Greenbaum, A.&Ozcan, A. Maskless imaging of dense samples using pixel super-resolution based multi-height lensfree on-chip microscopy.Opt. Express20, 3129-3143 (2012)..

Goodman, J. W.Introduction to Fourier Optics. (Roberts and Company Publishers, Englewood, 2005).

Canny, J. A computational approach to edge detection.IEEE Trans. Pattern Anal. Mach. Intell.PAMI-8, 679-698 (1986)..

Ronneberger O., Fischer P., Brox T. U-Net: convolutional networks for biomedical image segmentation. In: Navab N., Hornegger J., Wells W. M., Frangi A. F., (eds). Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015. (Springer, Cham, 2015) pp. 234-241https://doi.org/10.1007/978-3-319-24574-4_28https://doi.org/10.1007/978-3-319-24574-4_28.

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰