DEEP-squared: deep learning powered De-scattering with Excitation Patterning

Navodini Wijethilake; Mithunjha Anandakumar; Cheng Zheng; Peter T. C. So; Murat Yildirim; Dushan N. Wadduwage

doi:10.1038/s41377-023-01248-6

Go to Nature Website

Your Location：

Home >

Browse articles >

DEEP-squared: deep learning powered De-scattering with Excitation Patterning

Original Articles

- DEEP-squared: deep learning powered De-scattering with Excitation Patterning
- Light: Science & Applications Vol. 12, Issue 10, Pages: 2199-2214(2023)
- Affiliations：
  
  1.Center for Advanced Imaging, Faculty of Arts and Sciences, Harvard University, Cambridge, MA, USA
  2.Department of Electronic and Telecommunication Engineering, University of Moratuwa, Moratuwa, Sri Lanka
  3.Department of Mechanical Engineering, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA
  4.Laser Biomedical Research Center, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA
  5.Department of Biological Engineering, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA
  6.Picower Institute for Learning and Memory, Massachusetts Institute of Technology, 77 Massachusetts Ave., Cambridge, MA 02139, USA
  7.Department of Neuroscience, Cleveland Clinic Lerner Research Institute, Cleveland, OH 44195, USA
- Author bio：
  
  Dushan N. Wadduwage (wadduwage@fas.harvard.edu)
- Funds：
- DOI：10.1038/s41377-023-01248-6
  CLC：
- Published：31 October 2023，
  
  Published Online：13 September 2023，
  
  Received：01 March 2023，
  
  Revised：21 July 2023，
  
  Accepted：29 July 2023
- Accepted：
Scan QR Code
Wijethilake, N. et al. DEEP-squared: deep learning powered De-scattering with Excitation Patterning. Light: Science & Applications, 12, 2199-2214 (2023).
DOI：

Wijethilake, N. et al. DEEP-squared: deep learning powered De-scattering with Excitation Patterning. Light: Science & Applications, 12, 2199-2214 (2023). DOI： 10.1038/s41377-023-01248-6.

摘要

Abstract

Limited throughput is a key ch

allenge in in vivo deep tissue imaging using nonlinear optical microscopy. Point scanning multiphoton microscopy

the current gold standard

is slow especially compared to the widefield imaging modalities used for optically cleared or thin specimens. We recently introduced "De-scattering with Excitation Patterning" or "DEEP" as a widefield alternative to point-scanning geometries. Using patterned multiphoton excitation

DEEP encodes spatial information inside tissue before scattering. However

to de-scatter at typical depths

hundreds of such patterned excitations were needed. In this work

we present DEEP

a deep learning-based model that can de-scatter images from just tens of patterned excitations instead of hundreds. Consequently

we improve DEEP's throughput by almost an order of magnitude. We demonstrate our method in multiple numerical and experimental imaging studies

including in vivo cortical vasculature imaging up to 4 scattering lengths deep in live mice.

关键词

Keywords

references

Rocheleau, J. V.&Piston, D. W. Two-photon excitation microscopy for the study of living cells and tissues.Curr. Protoc. Cell Biol.20, 4.11.1–4.11.15 (2003)..

Yildirim, M. et al. Functional imaging of visual cortical layers and subplate in awake mice with optimized three-photon microscopy.Nat. Commun.10, 77 (2019)..

Yildirim, M. et al. Label-free three-photon imaging of intact human cerebral organoids for tracking early events in brain development and deficits in Rett syndrome.eLife11, e78079 (2022)..

Benninger, R. K. P.&Piston, D. W. Two-photon excitation microscopy for the study of living cells and tissues.Curr. Protoc. Cell Biol.59, 4.11.1–4.11.24 (2013)..

Oron, D., Tal, E.&Silberberg, Y. Scanningless depth-resolved microscopy.Opt. Express13, 1468–1476 (2005)..

Zhu, G. H. et al. Simultaneous spatial and temporal focusing of femtosecond pulses.Opt. Express13, 2153–2159 (2005)..

Vaziri, A.&Shank, C. V. Ultrafast widefield optical sectioning microscopy by multifocal temporal focusing.Opt. Express18, 19645–19655 (2010)..

Rowlands, C. J. et al. Wide-field three-photon excitation in biological samples.Light Sci. Appl.6, e16255 (2017)..

Escobet-Montalbán, A. et al. Wide-field multiphoton imaging through scattering media without correction.Sci. Adv.4, eaau1338 (2018)..

Zheng, C. et al. De-scattering with excitation patterning enables rapid wide-field imaging through scattering media.Sci. Adv.7, eaay5496 (2021)..

Mertz, J. Optical sectioning microscopy with planar or structured illumination.Nat. Methods8, 811–819 (2011)..

Belthangady, C.&Royer, L. A. Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction.Nat. Methods16, 1215–1225 (2019)..

Weigert, M. et al. Content-aware image restoration: pushing the limits of fluorescence microscopy.Nat. Methods15, 1090–1097 (2018)..

Jin, K. H. et al. Deep convolutional neural network for inverse problems in imaging.IEEE Trans. Image Process.26, 4509–4522 (2017)..

Yanny, K. et al. Deep learning for fast spatially varying deconvolution.Optica9, 96–99 (2022)..

Krizhevsky, A., Sutskever, I.&Hinton, G. E. Imagenet classification with deep convolutional neural networks. In:Proceedings of the 25th International Conference on Neural Information Processing Systems, 1097–1105 (Curran Associates Inc., Lake Tahoe, USA, 2012).

Ziletti, A. et al. Insightful classification of crystal structures using deep learning.Nat. Commun.9, 2775 (2018)..

Wei, Z., Liu, D.&Chen, X. D. Dominant-current deep learning scheme for electrical impedance tomography.IEEE Trans. Biomed. Eng.66, 2546–2555 (2019)..

Liu, S. Q. et al. Early diagnosis of Alzheimer's disease with deep learning. In:Proceedings of 2014 IEEE 11th International Symposium on Biomedical Imaging, 1015–1018 (IEEE, Beijing, China, 2014).

Wang, S. S. et al. Accelerating magnetic resonance imaging via deep learning. In:Proceedings of 2016 IEEE 13th International Symposium on Biomedical Imaging, 514–517 (IEEE, Prague, Czech Republic, 2016).

Girshick, R., Donahue, J., Darrell, T.&Malik, J. Rich feature hierarchies for accurate object detection and semantic segmentation. In:Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition, 580–587 (IEEE, Columbus, OH, USA, 2014).

Nielsen, A. A. K.&Voigt, C. A. Deep learning to predict the lab-of-origin of engineered DNA.Nat. Commun.9, 3135 (2018)..

Eraslan, G. et al. Single-cell RNA-seq denoising using a deep count autoencoder.Nat. Commun.10, 390 (2019)..

Eulenberg, P. et al. Reconstructing cell cycle and disease progression using deep learning.Nat. Commun.8, 463 (2017)..

Wei, Z.&Chen, X. D. Physics-inspired convolutional neural network for solving full-wave inverse scattering problems.IEEE Trans. Antennas Propag.67, 6138–6148 (2019)..

Zhu, B. et al. Image reconstruction by domain-transform manifold learning.Nature555, 487–492 (2018)..

Wei, Z. et al. 3D deep learning enables fast imaging of spines through scattering media by temporal focusing microscopy. Preprint athttps://arxiv.org/abs/2001.00520https://arxiv.org/abs/2001.00520(2019).

Ma, Y. et al. Wide-field optical mapping of neural activity and brain haemodynamics: considerations and novel approaches.Philos. Trans. R. Soc. B Biol. Sci.371, 20150360 (2016)..

Jacques, S. L.&Wang, L. Monte Carlo modeling of light transport in tissues. InOptical-Thermal Response of Laser-Irradiated Tissue(eds. Welch, A. J.&Gemert, M. J. C. ) 73–100 (Springer, Boston, 1995).

Robbins, M. S.&Hadwen, B. J. The noise performance of electron multiplying charge-coupled devices.IEEE Trans. Electron Devices50, 1227–1232 (2003)..

EMCCD Tutorial.https://www.nuvucameras.com/emccd-tutorial/https://www.nuvucameras.com/emccd-tutorial/. Accessed February 24, 2012.

Ronneberger, O., Fischer, P.&Brox, T. U-net: convolutional networks for biomedical image segmentation. In:Proceedings of the 18th International Conference on Medical Image Computing and Computer-assisted Intervention, 234–241 (Springer, Munich, Germany, 2015).

Roy, A. G., Navab, N.&Wachinger, C. Concurrent spatial and channel 'squeeze&excitation' in fully convolutional networks. In:Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention, 421–429 (Springer, Granada, Spain, 2018).

Uhrig, J. et al. Sparsity invariant CNNs. In:Proceedings of 2017 International Conference on 3D Vision, 11–20 (IEEE, Qingdao, China, 2017).

Paszke, A. et al. Automatic differentiation in pytorch. In:Neural Information Processing Systems 2017 Autodiff Workshop(Long Beach, CA, USA, 2017).

Wang, Z. et al. Image quality assessment: from error visibility to structural similarity.IEEE Trans. Image Process.13, 600–612 (2004)..

Yildirim, M. et al. Quantitative third-harmonic generation imaging of mouse visual cortex areas reveals correlations between functional maps and structural substrates.Biomed. Opt. Express11, 5650–5673 (2020)..

Schneider, M. et al. Tissue metabolism driven arterial tree generation.Med. Image Anal.16, 1397–1414 (2012)..

Todorov, M. I. et al. Machine learning analysis of whole mouse brain vasculature.Nat. Methods17, 442–449 (2020)..

Yardeni, Y., Eckhaus, M., Morris, H.D., Huizing, M.&Hoogstraten-Miller, S. Retro-orbital injections in mice.Lab. Anim. (NY)40, 155–160 (2011)..

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰