Toward implantable devices for angle-sensitive, lens-less, multifluorescent, single-photon lifetime imaging in the brain using Fabry–Perot and absorptive color filters

Adriaan J. Taal; Changhyuk Lee; Jaebin Choi; Björn Hellenkamp; Kenneth L. Shepard

doi:10.1038/s41377-022-00708-9

Go to Nature Website

Your Location：

Home >

Browse articles >

Toward implantable devices for angle-sensitive, lens-less, multifluorescent, single-photon lifetime imaging in the brain using Fabry–Perot and absorptive color filters

Articles

- Toward implantable devices for angle-sensitive, lens-less, multifluorescent, single-photon lifetime imaging in the brain using Fabry–Perot and absorptive color filters
- Light: Science & Applications Vol. 11, Issue 2, Pages: 197-211(2022)
- Affiliations：
  
  1.Columbia University - Department of Electrical Engineering, 500W. 120th St., Mudd 1310, New York, 10027, NY, USA
  2.Korea Institute of Science and Technology - Brain Science Institute, 5 Hwarang-ro 14-gil, Seongbuk-gu, Seoul, 02792, Republic of Korea
- Author bio：
  
  Kenneth L. Shepard (shepard@ee.columbia.edu)
- Funds：
- DOI：10.1038/s41377-022-00708-9
  CLC：
- Published：28 February 2022，
  
  Published Online：24 January 2022，
  
  Received：16 June 2021，
  
  Revised：29 December 2021，
  
  Accepted：04 January 2022
- Accepted：
Scan QR Code
Taal, A. J. et al. Toward implantable devices for angle-sensitive, lens-less, multifluorescent, single-photon lifetime imaging in the brain using Fabry–Perot and absorptive color filters. Light: Science & Applications, 11, 197-211 (2022).
DOI：

Taal, A. J. et al. Toward implantable devices for angle-sensitive, lens-less, multifluorescent, single-photon lifetime imaging in the brain using Fabry–Perot and absorptive color filters. Light: Science & Applications, 11, 197-211 (2022). DOI： 10.1038/s41377-022-00708-9.

摘要

Abstract

Implantable image sensors have the potential to revolutionize neuroscience. Due to their small form factor requirements; however

conventional filters and optics cannot be implemented. These limitations obstruct high-resolution imaging of large neural densities. Recent advances in angle-sensitive image sensors and single-photon avalanche diodes have provided a path toward ultrathin lens-less fluorescence imaging

enabling plenoptic sensing by extending sensing capabilities to include photon arrival time and incident angle

thereby providing the opportunity for separability of fluorescence point sources within the context of light-field microscopy (LFM). However

the addition of spectral sensitivity to angle-sensitive LFM reduces imager resolution because each wavelength requires a separate pixel subset. Here

we present a 1024-pixel

50 µm thick implantable shank-based neural imager with color-filter-grating-based angle-sensitive pixels. This angular-spectral sensitive front end combines a metal–insulator–metal (MIM) Fabry–Perot color filter and diffractive optics to produce the measurement of orthogonal light-field information from two distinct colors within a single photodetector. The result is the ability to add independent color sensing to LFM while doubling the effective pixel density. The implantable imager combines angular-spectral and temporal information to demix and localize multispectral fluorescent targets. In this initial prototype

this is demonstrated with 45 μm diameter fluorescently labeled beads in scattering medium. Fluorescent lifetime imaging is exploited to further aid source separation

in addition to detecting pH through lifetime changes in fluorescent dyes. While these initial fluorescent targets are considerably brighter than fluorescently labeled neurons

further improvements will allow the application of these techniques to in-vivo multifluorescent structural and functional neural imaging.

关键词

Keywords

references

Zong, W. J. et al. Fast high-resolution miniature two-photon microscopy for brain imaging in freely behaving mice.Nat. Methods14, 713–719 (2017)..

Liberti, W. A. III et al. An open source, wireless capable miniature microscope system.J. Neural Eng.14, 045001 (2017)..

Corder, G. et al. An amygdalar neural ensemble that encodes the unpleasantness of pain.Science363, 276–281 (2019)..

de Groot, A. et al. NINscope: a versatile miniscope for multi-region circuit investigations.bioRxivin press.

Ghosh, K. K. et al. Miniaturized integration of a fluorescence microscope.Nat. Methods8, 871–878 (2011)..

Jacob, A. D. et al. A compact head-mounted endoscope for in vivo calcium imaging in freely behaving mice.Curr. Protoc. Neurosci.84, e51 (2018)..

Shuman, T. et al. Breakdown of spatial coding and interneuron synchronization in epileptic mice.Nat. Neurosci.23, 229–238 (2020)..

Skocek, O. et al. High-speed volumetric imaging of neuronal activity in freely moving rodents.Nat. Methods15, 429–432 (2018)..

Choi, J. et al. A 512-pixel, 51-kHz-frame-rate, dual-shank, lens-less, filter-less single-photon avalanche diode CMOS neural imaging probe.IEEE J. Solid-State Circuits54, 2957–2968 (2019)..

Moreaux, L. C. et al. Integrated neurophotonics: toward dense volumetric interrogation of brain circuit activity—at depth and in real time.Neuron108, 66–92 (2020)..

Wetzstein, G., Ihrke, I.&Heidrich, W. On plenoptic multiplexing and reconstruction.Int. J. Comput. Vis.101, 384–400 (2013)..

Antipa, N. et al. DiffuserCam: lensless single-exposure 3D imaging.Optica5, 1–9 (2018)..

Ozcan, A.&McLeod, E. Lensless imaging and sensing.Annu. Rev. Biomed. Eng.18, 77–102 (2016)..

Hirsch, M. et al. A switchable light field camera architecture with Angle Sensitive Pixels and dictionary-based sparse coding. InProc. 2014 IEEE International Conference on Computational Photography(IEEE, 2014).https://doi.org/10.1109/ICCPHOT.2014.6831813https://doi.org/10.1109/ICCPHOT.2014.6831813..

Tropp, J. A.&Gilbert, A. C. Signal recovery from random measurements via orthogonal matching pursuit.IEEE Trans. Inf. Theory53, 4655–4666 (2007)..

Amelunxen, D. et al. Living on the edge: phase transitions in convex programs with random data. Information and Inference.J. IMA3, 224–294 (2014)..

Adams, J. K. et al. Single-frame 3D fluorescence microscopy with ultraminiature lensless FlatScope.Sci. Adv.3, e1701548 (2017)..

Moazeni, S. et al. A mechanically flexible implantable neural interface for computational imaging and optogenetic stimulation over 5.4 × 5.4mm2FoV. InProc. 2021 IEEE International Solid- State Circuits Conference(IEEE, 2021).https://doi.org/10.1109/ISSCC42613.2021.9365796https://doi.org/10.1109/ISSCC42613.2021.9365796..

Sivaramakrishnan, S. et al. Design and characterization of enhanced angle sensitive pixels.IEEE Trans. Electron Devices63, 113–119 (2016)..

Jayasuriya, S. et al. Dual light field and polarization imaging using CMOS diffractive image sensors.Opt. Lett.40, 2433–2436 (2015)..

Wang, A., Gill, P.&Molnar, A. Light field image sensors based on the Talbot effect.Appl. Opt.48, 5897–5905 (2009)..

Sun, P. F., Ishihara, R.&Charbon, E. Flexible ultrathin-body single-photon avalanche diode sensors and CMOS integration.Opt. Express24, 3734–3748 (2016)..

Aull, B. Geiger-mode avalanche photodiode arrays integrated to all-digital CMOS circuits.Sensors16, 495 (2016)..

Lee, C. et al. 11.5 A 512-Pixel 3kHz-frame-rate dual-shank lensless filterless single-photon-avalanche-diode CMOS neural imaging probe. InProc. 2019 IEEE International Solid-State Circuits Conference(IEEE, 2019).https://doi.org/10.1109/ISSCC.2019.8662408https://doi.org/10.1109/ISSCC.2019.8662408..

Gill, P. R. et al. Robustness of planar Fourier capture arrays to colour changes and lost pixels.J. Instrum.7, C01061 (2012)..

Gill, P. R. Odd-symmetry phase gratings produce optical nulls uniquely insensitive to wavelength and depth.Opt. Lett.38, 2074–2076 (2013)..

Li, Z. Y., Butun, S.&Aydin, K. Large-area, Lithography-free super absorbers and color filters at visible frequencies using ultrathin metallic films.ACS Photonics2, 183–188 (2015)..

Aalizadeh, M. et al. Toward electrically tunable, lithography-free, ultra-thin color filters covering the whole visible spectrum.Sci. Rep.8, 11316 (2018)..

Ordinario, D. D. et al. Stretchable structural color filters based on a metal–insulator–metal structure.Adv. Optical Mater.6, 1800851 (2018)..

Pimenta, S. et al. High-selectivity neural probe based on a Fabry–Perot optical filter and a CMOS silicon photodiodes array at visible wavelengths.J. Biomed. Opt.23, 105004 (2018)..

Gu, Y. H. et al. Color generation via subwavelength plasmonic nanostructures.Nanoscale7, 6409–6419 (2015)..

Yu, Y. et al. Transmissive/reflective structural color filters: theory and applications.J. Nanomaterials2014, 212637 (2014)..

Hong, L. Y. et al. A fully integrated CMOS fluorescence biosensor with on-chip nanophotonic filter. InProc. 2015 Symposium on VLSI Circuits (VLSI Circuits)(IEEE, 2015).https://doi.org/10.1109/VLSIC.2015.7231260https://doi.org/10.1109/VLSIC.2015.7231260..

Vereecke, B. et al. Fabrication of a CMOS-based imaging chip with monolithically integrated RGB and NIR filters.Proceedings2, 751 (2018)..

Nesci, A. et al. Measuring amplitude and phase distribution of fields generated by gratings with sub-wavelength resolution.Opt. Commun.205, 229–238 (2002)..

Taal, A. J., Rabinowitz, J.&Shepard, K. L. mr-EBL: ultra-high sensitivity negative-tone electron beam resist for highly selective silicon etching and large-scale direct patterning of permanent structures.Nanotechnology32, 245302 (2021)..

Nemani, K. V. et al. In vitro and in vivo evaluation of SU-8 biocompatibility.Mater. Sci. Eng.: C.33, 4453–4459 (2013)..

Matarèse, B. F. E. et al. Use of SU8 as a stable and biocompatible adhesion layer for gold bioelectrodes.Sci. Rep.8, 5560 (2018)..

Gill, P. R. et al. A microscale camera using direct Fourier-domain scene capture.Opt. Lett.36, 2949–2951 (2011)..

Shi, L. X. et al. Light field reconstruction using sparsity in the continuous Fourier domain.ACM Trans. Graph.34, 12 (2014)..

Levene, M. J. et al. In vivo multiphoton microscopy of deep brain tissue.J. Neurophysiol.91, 1908–1912 (2004)..

Yanny, K. et al. Miniscope3D: optimized single-shot miniature 3D fluorescence microscopy.Light.: Sci. Appl.9, 171 (2020)..

Vasquez-Lopez, S. A. et al. Subcellular spatial resolution achieved for deep-brain imaging in vivo using a minimally invasive multimode fiber.Light.: Sci. Appl.7, 110 (2018)..

Schwartz, D. E., Charbon, E.&Shepard, K. L. A single-photon avalanche diode array for fluorescence lifetime imaging microscopy.IEEE J. Solid-State Circuits43, 2546–2557 (2008)..

Akerboom, J. et al. Crystal structures of the GCaMP calcium sensor reveal the mechanism of fluorescence signal change and aid rational design.J. Biol. Chem.284, 6455–6464 (2009)..

Choi, S. Algorithms for orthogonal nonnegative matrix factorization. InProc. 2008 IEEE International Joint Conference on Neural Networks(IEEE, 2008).https://doi.org/10.1109/IJCNN.2008.4634046https://doi.org/10.1109/IJCNN.2008.4634046..

Gill, P. R., Wang, A.&Molnar, A. The in-crowd algorithm for fast basis pursuit denoising.IEEE Trans. Signal Process.59, 4595–4605 (2011)..

Leino, A. A., Pulkkinen, A.&Tarvainen, T.ValoMC: a Monte Carlo software and MATLAB toolbox for simulating light transport in biological tissue2, (957–972. Continuum, OSA, 2019).

Chapman, G. H., Paulsen, S.&Zhang, Y. T. Characterizing long lived intralipid-infused tissue phantoms scattering using imaging sensors. InProc. SPIE 11238, Optical Interactions with Tissue and Cells XXXI. (SPIE, 2020).https://doi.org/10.1117/12.2548689https://doi.org/10.1117/12.2548689..

Akarçay, H. G. et al. Determining the optical properties of a gelatin-TiO2 phantom at 780 nm.Biomed. Opt. Express3, 418–434 (2012)..

Yona, G. et al. Realistic numerical and analytical modeling of light scattering in brain tissue for optogenetic applications.eNeuro3, e0059–15.2015 (2016)..

Jacques, S. L. Optical properties of biological tissues: a review.Phys. Med. Biol.58, R37–R61 (2013)..

Kolb, K. Signal-to-noise ratio of Geiger-mode avalanche photodiode single-photon counting detectors.Optical Eng.53, 081904 (2014)..

Rapp, J.&Goyal, V. K. A few photons among many: unmixing signal and noise for photon-efficient active imaging.IEEE Trans. Comput. Imaging3, 445–459 (2017)..

Lin, H. J., Herman, P.&Lakowicz, J. R. Fluorescence lifetime-resolved pH imaging of living cells.Cytometry A52A, 77–89 (2003)..

Ryder, A. G. et al. Time-domain measurement of fluorescence lifetime variation with pH. InProc. SPIE 4259, Biomarkers and Biological Spectral Imaging(SPIE, 2001).https://doi.org/10.1117/12.432487https://doi.org/10.1117/12.432487..

Martin, M. M.&Lindqvist, L. The pH dependence of fluorescein fluorescence.J. Lumin.10, 381–390 (1975)..

Chesler, M. Regulation and modulation of pH in the brain.Physiol. Rev.83, 1183–1221 (2003)..

Wemmie, J. A., Taugher, R. J.&Kreple, C. J. Acid-sensing ion channels in pain and disease.Nat. Rev. Neurosci.14, 461–471 (2013)..

Miesenböck, G., De Angelis, D. A.&Rothman, J. E. Visualizing secretion and synaptic transmission with pH-sensitive green fluorescent proteins.Nature394, 192–195 (1998)..

Cao, H. et al. Batteryless implantable dual-sensor capsule for esophageal reflux monitoring.Gastrointest. Endosc.77, 649–653 (2013)..

Arifuzzaman, M. et al. An implanted pH sensor read using radiography.Analyst144, 2984–2993 (2019)..

Huang, H. Y. et al. Implantable tin porphyrin-PEG hydrogels with pH-responsive fluorescence.Biomacromolecules18, 562–567 (2017)..

Jayant, K. et al. Targeted intracellular voltage recordings from dendritic spines using quantum-dot-coated nanopipettes.Nat. Nanotechnol.12, 335–342 (2017)..

Mohanty, A. et al. Reconfigurable nanophotonic silicon probes for sub-millisecond deep-brain optical stimulation.Nat. Biomed. Eng.4, 223–231 (2020)..

Pisano, F. et al. Depth-resolved fiber photometry with a single tapered optical fiber implant.Nat. Methods16, 1185–1192 (2019)..

Pisanello, F. et al. Multipoint-emitting optical fibers for spatially addressable in vivo optogenetics.Neuron82, 1245–1254 (2014)..

Papouin, T. et al. Obtaining acute brain slices.Bio-protocol8,https://doi.org/10.21769/BioProtoc.2699https://doi.org/10.21769/BioProtoc.2699(2018)..

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰