Deeply learned broadband encoding stochastic hyperspectral imaging

Wenyi Zhang; Hongya Song; Xin He; Longqian Huang; Xiyue Zhang; Junyan Zheng; Weidong Shen; Xiang Hao; Xu Liu

doi:10.1038/s41377-021-00545-2

Go to Nature Website

Your Location：

Home >

Browse articles >

Deeply learned broadband encoding stochastic hyperspectral imaging

Letters

- Deeply learned broadband encoding stochastic hyperspectral imaging
- Light: Science & Applications Vol. 10, Issue 6, Pages: 969-975(2021)
- Affiliations：
  
  1.State Key Laboratory of Modern Optical Instrumentation, College of Optical Science and Technology, Zhejiang University, Hangzhou 310027, China
  2.Intelligent Optics & Photonics Research Center, Jiaxing Institute of Zhejiang University, Jiaxing 314000 Zhejiang Province, China
- Author bio：
  
  Xiang Hao (haox@zju.edu.cn)
  Xu Liu (liuxu@zju.edu.cn)
- Funds：
- DOI：10.1038/s41377-021-00545-2
  CLC：
- Published：30 June 2021，
  
  Published Online：25 May 2021，
  
  Received：10 January 2021，
  
  Revised：22 April 2021，
  
  Accepted：23 April 2021
- Accepted：
Scan QR Code
Zhang, W. Y. et al. Deeply learned broadband encoding stochastic hyperspectral imaging. Light: Science & Applications, 10, 969-975 (2021).
DOI：

Zhang, W. Y. et al. Deeply learned broadband encoding stochastic hyperspectral imaging. Light: Science & Applications, 10, 969-975 (2021). DOI： 10.1038/s41377-021-00545-2.

摘要

Abstract

Many applications requiring both spectral and spatial information at high resolution benefit from spectral imaging. Although different technical methods have been developed and commercially available

computational spectral cameras represent a compact

lightweight

and inexpensive solution. However

the tradeoff between spatial and spectral resolutions

dominated by the limited data volume and environmental noise

limits the potential of these cameras. In this study

we developed a deeply learned broadband encoding stochastic hyperspectral camera. In particular

using advanced artificial intelligence in filter design and spectrum reconstruction

we achieved 7000–11

000 times faster signal processing and ~10 times improvement regarding noise tolerance. These improvements enabled us to precisely and dynamically reconstruct the spectra of the entire field of view

previously unreachable with compact computational spectral cameras.

关键词

Keywords

references

Gabrieli, F. et al. Near-UV to mid-IR reflectance imaging spectroscopy of paintings on the macroscale.Sci. Adv.5, eaaw7794 (2019)..

Bahauddin, S. M., Bradshaw, S. J.&Winebarger, A. R. The origin of reconnection-mediated transient brightenings in the solar transition region.Nat. Astron.5, 237–245 (2021)..

Williams, L. J. et al. Remote spectral detection of biodiversity effects on forest biomass.Nat. Ecol. Evol.5, 46–54 (2021)..

Zhang, D. L. et al. Depth-resolved mid-infrared photothermalimaging of living cells and organisms with submicrometer spatial resolution.Sci. Adv.2, e1600521 (2016)..

Greaves, J. S. et al. Phosphine gas in the cloud decks of Venus.Nature Astronomy,https://doi.org/10.1038/s41550-020-1174-4https://doi.org/10.1038/s41550-020-1174-4(2020)..

Pavia, D. L. et al.Introduction to Spectroscopy(Cengage Learning, 2014).

Hagen, N. A.&Kudenov, M. W. Review of snapshot spectral imaging technologies.Opt. Eng.52, 090901 (2013)..

Cao, X. et al. Computational snapshot multispectral cameras: toward dynamic capture of the spectral world.IEEE Signal Process. Mag.33, 95–108 (2016)..

Gehm, M. E. et al. Single-shot compressive spectral imaging with a dual-disperser architecture.Opt. Express15, 14013–14027 (2007)..

Cai, X. S. et al. One-shot ultraspectral imaging with reconfigurable metasurfaces. Preprint athttps://arxiv.org/abs/2005.02689https://arxiv.org/abs/2005.02689(2020).

Wang, Z. et al. Single-shot on-chip spectral sensors based on photonic crystal slabs.Nat. Commun.10, 1020 (2019)..

Oliver, J., Lee, W. B.&Lee, H. N. Filters with random transmittance for improving resolution in filter-array-based spectrometers.Opt. Express21, 3969–3989 (2013)..

Bao, J.&Bawendi, M. G. A colloidal quantum dot spectrometer.Nature523, 67–70 (2015)..

Yang, Z. Y. et al. Single-nanowire spectrometers.Science365, 1017–1020 (2019)..

Baraniuk, R. G. Compressive sensing.IEEE Signal Process. Mag.24, 118–121 (2007)..

Choi, I. et al. High-quality hyperspectral reconstruction using a spectral prior.ACM Trans. Graph.36, 218 (2017)..

Wang, L. et al. Hyperspectral image reconstruction using a deep spatial-spectral prior. InProc. IEEE/CVF Conference on Computer Vision and Pattern Recognition15–20 June 2019 (IEEE, Long Beach, CA, USA, 2019).

Gewali, U. B., Monteiro, S. T.&Saber, E. Spectral super-resolution with optimized bands.Remote Sens.11, 1648 (2019)..

Nie, S. J. et al. Deeply learned filter response functions for hyperspectral reconstruction. InProc. IEEE/CVF Conference on Computer Vision and Pattern Recognition18–23 June 2018 (IEEE, Salt Lake City, UT, USA, 2018).

So, S. et al. Deep learning enabled inverse design in nanophotonics.Nanophotonics9, 1041–1057 (2020)..

Voznyy, O. et al. Machine learning accelerates discovery of optimal colloidal quantum dot synthesis.ACS Nano13, 11122–11128 (2019)..

Gao, L. et al. A bidirectional deep neural network for accurate silicon color design.Adv. Mater.31, 1905467 (2019)..

Liu, D. J. et al. Training deep neural networks for the inverse design of nanophotonic structures.ACS Photonics5, 1365–1369 (2018)..

Song, H. Y. et al. Deep-learned broadband encoding stochastic filters for computational spectroscopic instruments.Adv. Theory Simul.4, 2000299 (2021)..

Goodfellow, I., Bengio, Y.&Courville, A.Deep Learning1 (MIT Press, 2016).

Figueiredo, M. A. T., Nowak, R. D.&Wright, S. J. Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems.IEEE J. Sel. Top. Signal Process.1, 586–597 (2007)..

Alford, T. L., Feldman, L. C.&Mayer, J. W.Fundamentals of Nanoscale Film. Analysis(Springer, 2007).

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰