Unsupervised full-color cellular image reconstruction through disordered optical fiber

Xiaowen Hu; Jian Zhao; Jose Enrique Antonio-Lopez; Rodrigo Amezcua Correa; Axel Schülzgen

doi:10.1038/s41377-023-01183-6

Go to Nature Website

Your Location：

Home >

Browse articles >

Unsupervised full-color cellular image reconstruction through disordered optical fiber

Original Articles

- Unsupervised full-color cellular image reconstruction through disordered optical fiber
- Light: Science & Applications Vol. 12, Issue 6, Pages: 1091-1102(2023)
- Affiliations：
  
  1.CREOL, The College of Optics and Photonics, University of Central Florida, Orlando, FL 32816, USA
  2.The Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
- Author bio：
  
  Jian Zhao (jianzhao@knights.ucf.edu)
- Funds：
- DOI：10.1038/s41377-023-01183-6
  CLC：
- Published：30 June 2023，
  
  Published Online：23 May 2023，
  
  Received：21 December 2022，
  
  Revised：11 May 2023，
  
  Accepted：12 May 2023
Scan for full text
Hu, X. W. et al. Unsupervised full-color cellular image reconstruction through disordered optical fiber. Light: Science & Applications, 12, 1091-1102 (2023).
DOI：

Hu, X. W. et al. Unsupervised full-color cellular image reconstruction through disordered optical fiber. Light: Science & Applications, 12, 1091-1102 (2023). DOI： 10.1038/s41377-023-01183-6.

摘要

Abstract

Recent years have witnessed the tremendous development of fusing fiber-optic imaging with supervised deep learning to enable high-quality imaging of hard-to-reach areas. Nevertheless

the supervised deep learning method imposes strict constraints on fiber-optic imaging systems

where the input objects and the fiber outputs have to be collected in pairs. To unleash the full potential of fiber-optic imaging

unsupervised image reconstruction is in demand. Unfortunately

neither optical fiber bundles nor multimode fibers can achieve a point-to-point transmission of the object with a high sampling density

as is a prerequisite for unsupervised image reconstruction. The recently proposed disordered fibers offer a new solution based on the transverse Anderson localization. Here

we demonstrate unsupervised full-color imaging with a cellular resolution through a meter-long disordered fiber in both transmission and reflection modes. The unsupervised image reconstruction consists of two stages. In the first stage

we perform a pixel-wise standardization on the fiber outputs using the statistics of the objects. In the second stage

we recover the fine details of the reconstructions through a generative adversarial network. Unsupervised image reconstruction does not need paired images

enabling a much more flexible calibration under various conditions. Our new solution achieves full-color high-fidelity cell imaging within a working distance of at least 4 mm by only collecting the fiber outputs after an initial calibration. High imaging robustness is also demonstrated when the disordered fiber is bent with a central angle of 60°. Moreover

the cross-domain generality on unseen objects is shown to be enhanced with a diversified object set.

关键词

Keywords

references

Flusberg,B.A.etal.Fiber-opticfluorescenceimaging.Nat. Methods2,941–950(2005)..

Wallace,M.B.&Fockens,P.Probe-basedconfocallaserendomicroscopy.Gastroenterology136,1509–1513(2009)..

Wang,K.K.etal.Useofprobe‐basedconfocallaserendomicroscopy(pCLE)ingastrointestinalapplications.Aconsensusreportbasedonclinicalevidence.U. Eur. Gastroenterol. J.3,230–254(2015)..

Fugazza,A.etal.Confocallaserendomicroscopyingastrointestinalandpancreatobiliarydiseases:asystematicreviewandmeta-analysis.BioMed. Res. Int.2016,4638683(2016)..

McGoran,J.J.etal.Miniaturegastrointestinalendoscopy:nowandthefuture.World J. Gastroenterol.25,4051–4060(2019)..

Flusberg,B.A.etal.High-speed,miniaturizedfluorescencemicroscopyinfreelymovingmice.Nat. Methods5,935–938(2008)..

Szabo,V.etal.Spatiallyselectiveholographicphotoactivationandfunctionalfluorescenceimaginginfreelybehavingmicewithafiberscope.Neuron84,1157–1169(2014)..

Zong,W.J.etal.Fasthigh-resolutionminiaturetwo-photonmicroscopyforbrainimaginginfreelybehavingmice.Nat. Methods14,713–719(2017)..

Turtaev,S.etal.High-fidelitymultimodefibre-basedendoscopyfordeepbraininvivoimaging.Light Sci. Appl.7,92(2018)..

Vasquez-Lopez,S.A.etal.Subcellularspatialresolutionachievedfordeep-brainimaginginvivousingaminimallyinvasivemultimodefiber.Light Sci. Appl.7,110(2018)..

Ghaemi,H.F.etal.Fiberimageguidewithsubwavelengthresolution.Appl. Phys. Lett.72,1137–1139(1998)..

Muldoon,T.J.etal.Subcellular-resolutionmolecularimagingwithinlivingtissuebyfibermicroendoscopy.Opt. Express15,16413–16423(2007)..

Hughes,M.,Chang,T.P.&Yang,G.Z.Fiberbundleendocytoscopy.Biomed. Opt. Express4,2781–2794(2013)..

Krstajić,N.etal.Two-colorwidefieldfluorescencemicroendoscopyenablesmultiplexedmolecularimaginginthealveolarspaceofhumanlungtissue.J. Biomed. Opt.21,046009(2016)..

Perperidis,A.etal.Imagecomputingforfibre-bundleendomicroscopy:areview.Med. Image Anal.62,101620(2020)..

LeCun,Y.,Bengio,Y.&Hinton,G.Deeplearning.Nature521,436–444(2015)..

Barbastathis,G.,Ozcan,A.&Situ,G.H.Ontheuseofdeeplearningforcomputationalimaging.Optica6,921–943(2019)..

Shao,J.B.etal.Fiberbundleimagerestorationusingdeeplearning.Opt. Lett.44,1080–1083(2019)..

Shao,J.B.etal.Fiberbundleimagingresolutionenhancementusingdeeplearning.Opt. Express27,15880–15890(2019)..

Borhani,N.etal.Learningtoseethroughmultimodefibers.Optica5,960–966(2018)..

Rahmani,B.etal.Multimodeopticalfibertransmissionwithadeeplearningnetwork.Light Sci. Appl.7,69(2018)..

Zhu,J.Y.etal.Unpairedimage-to-imagetranslationusingcycle-consistentadversarialnetworks.InProc 2017 IEEE International Conference on Computer Vision (ICCV),Venice,Italy,pp.2242–2251(IEEE,2017).https://doi.org/10.1109/ICCV.2017.244https://doi.org/10.1109/ICCV.2017.244..

Hu,X.W.etal.Adaptiveinversemapping:amodel-freesemi-supervisedlearningapproachtowardsrobustimagingthroughdynamicscatteringmedia.Opt. Express31,14343–14357(2023)..

Weiss,U.&Katz,O.Two-photonlenslessmicro-endoscopywithin-situwavefrontcorrection.Opt. Express26,28808–28817(2018)..

Karbasi,S.etal.TransverseAndersonlocalizationinadisorderedglassopticalfiber.Opt Mater. Express2,1496–1503(2012)..

Chen,M.H.&Li,M.J.ObservingtransverseAndersonlocalizationinrandomairlinebasedfiber.In:Proc SPIE 8994, Photonic and Phononic Properties of Engineered Nanostructures IV,(eds.Adibi,A.,Lin,S.-Y.&Scherer,A.)89941S(SPIE,SanFrancisco,CA,USA,2014).

Zhao,J.etal.Imagetransportthroughmeter-longrandomlydisorderedsilica-airopticalfiber.Sci. Rep.8,3065(2018)..

Zhao,J.etal.Deeplearningimagingthroughfully-flexibleglass-airdisorderedfiber.ACS Photonics5,3930–3935(2018)..

Zhao,J.etal.Deep-learningcellimagingthroughAndersonlocalizingopticalfiber.Adv. Photonics1,066001(2019)..

Mafi,A.etal.Disorderedandersonlocalizationopticalfibersforimagetransport—areview.J. Lightwave Technol.37,5652–5659(2019)..

Hu,X.W.etal.Learning-supportedfull-colorcellimagingthroughdisorderedopticalfiber.InProc 2020 Conference on Lasers and Electro-Optics,WashingtonDC,USA,p.SM2L.5(OpticalSocietyofAmerica,2020).https://doi.org/10.1364/CLEO_SI.2020.SM2L.5https://doi.org/10.1364/CLEO_SI.2020.SM2L.5..

Hu,X.W.etal.Robustimaging-freeobjectrecognitionthroughandersonlocalizingopticalfiber.J. Lightwave Technol.39,920–926(2021)..

Zhao,J.etal.Learning-basedimagetransportthroughdisorderedopticalfiberswithtransverseandersonlocalization.Front. Phys.9,710351(2021)..

Abaie,B.etal.Disorder-inducedhigh-qualitywavefrontinanAndersonlocalizingopticalfiber.Optica5,984–987(2018)..

Abdullaev,S.S.&Abdullaev,F.K.Onpropagationoflightinfiberbundleswithrandomparameters.Radiofizika23,766–767(1980)..

DeRaedt,H.,Lagendijk,A.&deVries,P.Transverselocalizationoflight.Phys. Rev. Lett.62,47–50(1989)..

Karbasi,S.,Koch,K.W.&Mafi,A.Multiple-beampropagationinanAndersonlocalizedopticalfiber.Opt. Express21,305–313(2013)..

Schirmacher,W.etal.Whatistherighttheoryforandersonlocalizationoflight?Anexperimentaltest.Phys. Rev. Lett.120,067401(2018)..

Chen,X.P.,Reichenbach,K.L.&Xu,C.Experimentalandtheoreticalanalysisofcore-to-corecouplingonfiberbundleimaging.Opt. Express16,21598–21607(2008)..

Reichenbach,K.L.&Xu,C.Numericalanalysisoflightpropagationinimagefibersorcoherentfiberbundles.Opt. Express15,2151–2165(2007)..

Choi,Y.etal.Scanner-freeandwide-fieldendoscopicimagingbyusingasinglemultimodeopticalfiber.Phys. Rev. Lett.109,203901(2012)..

Ohayon,S.etal.Minimallyinvasivemultimodeopticalfibermicroendoscopefordeepbrainfluorescenceimaging.Biomed. Opt. Express9,1492–1509(2018)..

Caravaca-Aguirre,A.M.&Piestun,R.Singlemultimodefiberendoscope.Opt. Express25,1656–1665(2017)..

Shabairou,N.etal.Colorimageidentificationandreconstructionusingartificialneuralnetworksonmultimodefiberimages:towardsanall-opticaldesign.Opt. Lett.43,5603–5606(2018)..

Kürüm,U.etal.Deeplearningenabledrealtimespecklerecognitionandhyperspectralimagingusingamultimodefiberarray.Opt. Express27,20965–20979(2019)..

Ronneberger,O.,Fischer,P.&Brox,T.U-Net:convolutionalnetworksforbiomedicalimagesegmentation.In:Proceedings of the 18th International Conference on Medical Image Computing and Computer-assisted Intervention, 234–241(Springer,Munich,Germany,2015).

Isola,P.etal.Efros.Image-to-imagetranslationwithconditionaladversarialnetworks.InProc 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu,HI,USA,5967–5976.(IEEE,2017).

Karbasi,S.etal.DetailedinvestigationoftheimpactofthefiberdesignparametersonthetransverseAndersonlocalizationoflightindisorderedopticalfibers.Opt. Express20,18692–18706(2012)..

Karbasi,S.,Koch,K.W.&Mafi,A.Imagetransportqualitycanbeimprovedindisorderedwaveguides.Opt. Commun.311,72–76(2013)..

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰