Experimentally unsupervised deconvolution for light-sheet microscopy with propagation-invariant beams

Philip Wijesinghe; Stella Corsetti; Darren J. X. Chow; Shuzo Sakata; Kylie R. Dunning; Kishan Dholakia

doi:10.1038/s41377-022-00975-6

Go to Nature Website

Your Location：

Home >

Browse articles >

Experimentally unsupervised deconvolution for light-sheet microscopy with propagation-invariant beams

Original Articles

- Experimentally unsupervised deconvolution for light-sheet microscopy with propagation-invariant beams
- Light: Science & Applications Vol. 11, Issue 11, Pages: 2790-2804(2022)
- Affiliations：
  
  1.SUPA, School of Physics and Astronomy, University of St Andrews, North Haugh, St Andrews, Fife KY16 9SS, UK
  2.Robinson Research Institute, Adelaide Medical School, The University of Adelaide, Adelaide, SA, Australia
  3.Australian Research Council Centre of Excellence for Nanoscale BioPhotonics, The University of Adelaide, Adelaide, SA, Australia
  4.Institute for Photonics and Advanced Sensing, The University of Adelaide, Adelaide, SA, Australia
  5.Strathclyde Institute of Pharmacy and Biomedical Sciences, University of Strathclyde, Glasgow G4 0RE, UK
  6.School of Biological Sciences, The University of Adelaide, Adelaide, SA, Australia
  7.Department of Physics, College of Science, Yonsei University, Seoul 03722, South Korea
- Author bio：
  
  Philip Wijesinghe (pw64@st-andrews.ac.uk)
  Kishan Dholakia (kd1@st-andrews.ac.uk)
- Funds：
- DOI：10.1038/s41377-022-00975-6
  CLC：
- Published：30 November 2022，
  
  Published Online：02 November 2022，
  
  Received：09 August 2021，
  
  Revised：26 August 2022，
  
  Accepted：31 August 2022
- Accepted：
Scan QR Code
Wijesinghe, P. et al. Experimentally unsupervised deconvolution for light-sheet microscopy with propagation-invariant beams. Light: Science & Applications, 11, 2790-2804 (2022).
DOI：

Wijesinghe, P. et al. Experimentally unsupervised deconvolution for light-sheet microscopy with propagation-invariant beams. Light: Science & Applications, 11, 2790-2804 (2022). DOI： 10.1038/s41377-022-00975-6.

摘要

Abstract

Deconvolution is a challenging inverse problem

particularly in techniques that employ complex engineered point-spread functions

such as microscopy with propagation-invariant beams. Here

we present a deep-learning method for deconvolution that

in lieu of end-to-end training with ground truths

is trained using known physics of the imaging system. Specifically

we train a generative adversarial network with images generated with the known point-spread function of the system

and combine this with unpaired experimental data that preserve perceptual content. Our method rapidly and robustly deconvolves and super-resolves microscopy images

demonstrating a two-fold improvement in image contrast to conventional deconvolution methods. In contrast to common end-to-end networks that often require 1000–10

000s paired images

our method is experimentally unsupervised and can be trained solely on a few hundred regions of interest. We demonstrate its performance on light-sheet microscopy with propagation-invariant Airy beams in oocytes

preimplantation embryos and excised brain tissue

as well as illustrate its utility for Bessel-beam LSM. This method aims to democratise learned methods for deconvolution

as it does not require data acquisition outwith the conventional imaging protocol.

关键词

Keywords

references

Sibarita, J. B. In Deconvolution Microscopy in Microscopy Techniques (ed. Rietdorf, J. ) 201–243 (Springer, 2005).

J. G. McNally , 等 . Three-dimensional imaging by deconvolution microscopy . Methods , 1999 . 19 373 -385 . DOI:10.1006/meth.1999.0873http://doi.org/10.1006/meth.1999.0873 .

D. Sage , 等 . DeconvolutionLab2: an open-source software for deconvolution microscopy . Methods , 2017 . 115 28 -41 . DOI:10.1016/j.ymeth.2016.12.015http://doi.org/10.1016/j.ymeth.2016.12.015 .

G. Barbastathis , A. Ozcan , G. Situ . On the use of deep learning for computational imaging . Optica , 2019 . 6 921 -943 . DOI:10.1364/OPTICA.6.000921http://doi.org/10.1364/OPTICA.6.000921 .

T. Vettenburg , 等 . Light-sheet microscopy using an Airy beam . Nat. Methods , 2014 . 11 541 -544 . DOI:10.1038/nmeth.2922http://doi.org/10.1038/nmeth.2922 .

T. A. Planchon , 等 . Rapid three-dimensional isotropic imaging of living cells using Bessel beam plane illumination . Nat. Methods , 2011 . 8 417 -423 . DOI:10.1038/nmeth.1586http://doi.org/10.1038/nmeth.1586 .

J. Nylk , 等 . Enhancement of image quality and imaging depth with Airy light-sheet microscopy in cleared and non-cleared neural tissue . Biomed. Opt. Express , 2016 . 7 4021 -4033 . DOI:10.1364/BOE.7.004021http://doi.org/10.1364/BOE.7.004021 .

F. O. Fahrbach , P. Simon , A. Rohrbach . Microscopy with self-reconstructing beams . Nat. Photonics , 2010 . 4 780 -785 . DOI:10.1038/nphoton.2010.204http://doi.org/10.1038/nphoton.2010.204 .

B.-C. Chen , 等 . Lattice light-sheet microscopy: Imaging molecules to embryos at high spatiotemporal resolution . Science , 2014 . 346 1257998 DOI:10.1126/science.1257998http://doi.org/10.1126/science.1257998 .

Z. Yang , 等 . A compact Airy beam light sheet microscope with a tilted cylindrical lens . Biomed. Opt. Express , 2014 . 5 3434 -3442 . DOI:10.1364/BOE.5.003434http://doi.org/10.1364/BOE.5.003434 .

J. Arlt , K. Dholakia . Generation of high-order Bessel beams by use of an axicon . Optics Commun. , 2000 . 177 297 -301 . DOI:10.1016/S0030-4018(00)00572-1http://doi.org/10.1016/S0030-4018(00)00572-1 .

P. Wijesinghe , K. Dholakia . Emergent physics-informed design of deep learning for microscopy . J. Phys. Photonics , 2021 . 3 021003 DOI:10.1088/2515-7647/abf02chttp://doi.org/10.1088/2515-7647/abf02c .

M. Weigert , 等 . Content-aware image restoration: pushing the limits of fluorescence microscopy . Nat. Methods , 2018 . 15 1090 -1097 . DOI:10.1038/s41592-018-0216-7http://doi.org/10.1038/s41592-018-0216-7 .

H. Wang , 等 . Deep learning enables cross-modality super-resolution in fluorescence microscopy . Nat. Methods , 2019 . 16 103 -110 . DOI:10.1038/s41592-018-0239-0http://doi.org/10.1038/s41592-018-0239-0 .

J. Chen , 等 . Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes . Nat. Methods , 2021 . 18 678 -687 . DOI:10.1038/s41592-021-01155-xhttp://doi.org/10.1038/s41592-021-01155-x .

H. Zhang , 等 . Exceeding the limits of 3D fluorescence microscopy using a dual-stage-processing network . Optica , 2020 . 7 1627 -1640 . DOI:10.1364/OPTICA.402046http://doi.org/10.1364/OPTICA.402046 .

C. Belthangady , L. A. Royer . Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction . Nat. Methods , 2019 . 16 1215 -1225 . DOI:10.1038/s41592-019-0458-zhttp://doi.org/10.1038/s41592-019-0458-z .

M. Guo , 等 . Rapid image deconvolution and multiview fusion for optical microscopy . Nat. Biotechnol. , 2020 . 38 1337 -1346 . DOI:10.1038/s41587-020-0560-xhttp://doi.org/10.1038/s41587-020-0560-x .

Weigert, M. et al. Isotropic reconstruction of 3D fluorescence microscopy images using convolutional neural networks. In Medical Image Computing and Computer-Assisted Intervention—MICCAI 2017 (eds. Descoteaux, M. et al. ) 126–134 (Springer, 2017).

C. Zuo , 等 . Deep learning in optical metrology: a review . Light Sci. Appl. , 2022 . 11 39 DOI:10.1038/s41377-022-00714-xhttp://doi.org/10.1038/s41377-022-00714-x .

C. N. Christensen , 等 . ML-SIM: universal reconstruction of structured illumination microscopy images using transfer learning . Biomed. Opt. Express , 2021 . 12 2720 -2733 . DOI:10.1364/BOE.414680http://doi.org/10.1364/BOE.414680 .

M. Deng , 等 . On the interplay between physical and content priors in deep learning for computational imaging . Opt. Express , 2020 . 28 24152 -24170 . DOI:10.1364/OE.395204http://doi.org/10.1364/OE.395204 .

Goodfellow, I. et al. Generative adversarial nets. In Proceedings of the 27th International Conference on Neural Information Processing Systems. 2672–2680 (MIT Press, 2014).

He, K. et al. Deep residual learning for image recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. 770–778 (IEEE, 2016).

Isola, P. et al. Image-to-image translation with conditional adversarial networks. in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 5967–5976 (IEEE, 2017).

Ledig, C. et al. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. 105–114 (IEEE, 2017).

Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. Preprint at https://arxiv.org/abs/1409.1556https://arxiv.org/abs/1409.1556 (2015).

Johnson, J., Alahi, A. & Fei-Fei, L. Perceptual losses for real-time style transfer and super-resolution. In Proceedings of the 14th European Conference on Computer Vision. 694–711 (Springer, 2016).

S. Li , G. Barbastathis . Spectral pre-modulation of training examples enhances the spatial resolution of the phase extraction neural network (PhENN) . Opt. Express , 2018 . 26 29340 -29352 . DOI:10.1364/OE.26.029340http://doi.org/10.1364/OE.26.029340 .

W. H. Richardson . Bayesian-based iterative method of image restoration . J. Optical Soc. Am. , 1972 . 62 55 -59 . DOI:10.1364/JOSA.62.000055http://doi.org/10.1364/JOSA.62.000055 .

L. B. Lucy . An iterative technique for the rectification of observed distributions . Astronomical J. , 1974 . 79 745 -754 . DOI:10.1086/111605http://doi.org/10.1086/111605 .

Ulyanov, D., Vedaldi, A. & Lempitsky, V. Deep image prior. Int. J. Computer Vision 128, 1867–1888 (2020).

J. Huisken , D. Y. R. Stainier . Selective plane illumination microscopy techniques in developmental biology . Development , 2009 . 136 1963 -1975 . DOI:10.1242/dev.022426http://doi.org/10.1242/dev.022426 .

M. E. Gosnell , 等 . Quantitative non-invasive cell characterisation and discrimination based on multispectral autofluorescence features . Sci. Rep. , 2016 . 6 23453 DOI:10.1038/srep23453http://doi.org/10.1038/srep23453 .

Xenopoulos, P., Kang, M. & Hadjantonakis, A. -K. Cell lineage allocation within the inner cell mass of the mouse blastocyst. In Mouse Development From Oocyte to Stem Cells (ed. Kubiak, J. Z. ) 185–202 (Springer, 2012).

D. Richani , 等 . Metabolic co-dependence of the oocyte and cumulus cells: essential role in determining oocyte developmental competence . Hum. Reprod. Update , 2021 . 27 27 -47 . DOI:10.1093/humupd/dmaa043http://doi.org/10.1093/humupd/dmaa043 .

S. Corsetti , F. Gunn-Moore , K. Dholakia . Light sheet fluorescence microscopy for neuroscience . J. Neurosci. Methods , 2018 . 319 16 -27 . DOI:10.1016/j.jneumeth.2018.07.011http://doi.org/10.1016/j.jneumeth.2018.07.011 .

S. Corsetti , 等 . Widefield light sheet microscopy using an Airy beam combined with deep-learning super-resolution . OSA Contin. , 2020 . 3 1068 -1083 . DOI:10.1364/OSAC.391644http://doi.org/10.1364/OSAC.391644 .

T. F. Freund , G. Buzsáki . Interneurons of the hippocampus . Hippocampus , 1996 . 6 347 -470 . DOI:10.1002/(SICI)1098-1063(1996)6:4<347::AID-HIPO1>3.0.CO;2-Ihttp://doi.org/10.1002/(SICI)1098-1063(1996)6:4<347::AID-HIPO1>3.0.CO;2-I .

Y. Kawaguchi , Y. Kubota . GABAergic cell subtypes and their synaptic connections in rat frontal cortex . Cerebral Cortex , 1997 . 7 476 -486 . DOI:10.1093/cercor/7.6.476http://doi.org/10.1093/cercor/7.6.476 .

F. M. Gasparoli , 等 . Is laser repetition rate important for two-photon light sheet microscopy? . OSA Contin. , 2020 . 3 2935 -2942 . DOI:10.1364/OSAC.405369http://doi.org/10.1364/OSAC.405369 .

S. Takanezawa , T. Saitou , T. Imamura . Wide field light-sheet microscopy with lens-axicon controlled two-photon Bessel beam illumination . Nat. Commun. , 2021 . 12 2979 DOI:10.1038/s41467-021-23249-yhttp://doi.org/10.1038/s41467-021-23249-y .

L. Xiao , 等 . Deep learning-enabled efficient image restoration for 3D microscopy of turbid biological specimens . Opt. Express , 2020 . 28 30234 -30247 . DOI:10.1364/OE.399542http://doi.org/10.1364/OE.399542 .

K. Yanny , 等 . Deep learning for fast spatially varying deconvolution . Optica , 2022 . 9 96 -99 . DOI:10.1364/OPTICA.442438http://doi.org/10.1364/OPTICA.442438 .

Y. Zhang , 等 . Computational optical sectioning with an incoherent multiscale scattering model for light-field microscopy . Nat. Commun. , 2021 . 12 6391 DOI:10.1038/s41467-021-26730-whttp://doi.org/10.1038/s41467-021-26730-w .

E. Bostan , 等 . Deep phase decoder: self-calibrating phase microscopy with an untrained deep neural network . Optica , 2020 . 7 559 -562 . DOI:10.1364/OPTICA.389314http://doi.org/10.1364/OPTICA.389314 .

Mao, X. et al. Least squares generative adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision. 2813–2821 (IEEE, 2017).

Albuquerque, I. et al. Multi-objective training of generative adversarial networks with multiple discriminators. In Proceedings of the 36th International Conference on Machine Learning. 202–211 (ICML, 2019).

M. Deng , 等 . Probing shallower: perceptual loss trained phase extraction neural network (PLT-PhENN) for artifact-free reconstruction at low photon budget . Opt. Express , 2020 . 28 2511 -2535 . DOI:10.1364/OE.381301http://doi.org/10.1364/OE.381301 .

Lyngholm, D. & Sakata, S. Cre-dependent optogenetic transgenic mice without early age-related hearing loss. Front. Aging Neurosci. 29, 11 (2019).

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰