All-optical image classification through unknown random diffusers using a single-pixel diffractive network

Bijie Bai; Yuhang Li; Yi Luo; Xurong Li; Ege Çetintaş; Mona Jarrahi; Aydogan Ozcan

doi:10.1038/s41377-023-01116-3

Go to Nature Website

Your Location：

Home >

Browse articles >

All-optical image classification through unknown random diffusers using a single-pixel diffractive network

Original Articles

- All-optical image classification through unknown random diffusers using a single-pixel diffractive network
- All-optical image classification through unknown random diffusers using a single-pixel diffractive network
- 光：科学与应用 2023年12卷第4期页码：570-584
- Affiliations：
  
  1.Electrical and Computer Engineering Department, University of California, Los Angeles, California 90095, USA
  2.Bioengineering Department, University of California, Los Angeles, California 90095, USA
  3.California Nano Systems Institute (CNSI), University of California, Los Angeles, California 90095, USA
- Author bio：
  
  Aydogan Ozcan (ozcan@ucla.edu)
- Funds：
- DOI：10.1038/s41377-023-01116-3
  中图分类号：
- 纸质出版日期：2023-4-30，
  
  网络出版日期：2023-3-9，
  
  收稿日期：2022-8-20，
  
  修回日期：2023-2-22，
  
  录用日期：2023-2-22
Scan for full text
All-optical image classification through unknown random diffusers using a single-pixel diffractive network[J]. 光：科学与应用, 2023,12(4):570-584.

Bai, B. J.et al. All-optical image classification through unknown random diffusers using a single-pixel diffractive network.Light: Science & Applications,12, 570-584 (2023).
All-optical image classification through unknown random diffusers using a single-pixel diffractive network[J]. 光：科学与应用, 2023,12(4):570-584. DOI： 10.1038/s41377-023-01116-3.

Bai, B. J.et al. All-optical image classification through unknown random diffusers using a single-pixel diffractive network.Light: Science & Applications,12, 570-584 (2023). DOI： 10.1038/s41377-023-01116-3.

摘要

Abstract

Classification of an object behind a random and unknown scattering medium sets a challenging task for computational imaging and machine vision fields. Recent deep learning-based approaches demonstrated the classification of objects using diffuser-distorted patterns collected by an image sensor. These methods demand relatively large-scale computing using deep neural networks running on digital computers. Here

we present an all-optical processor to directly classify unknown objects through unknown

random phase diffusers using broadband illumination detected with a single pixel. A set of transmissive diffractive layers

optimized using deep learning

forms a physical network that all-optically maps the spatial information of an input object behind a random diffuser into the power spectrum of the output light detected through a single pixel at the output plane of the diffractive network. We numerically demonstrated the accuracy of this framework using broadband radiation to classify unknown handwritten digits through random new diffusers

never used during the training phase

and achieved a blind testing accuracy of 87.74 ± 1.12%. We also experimentally validated our single-pixel broadband diffractive network by classifying handwritten digits "0" and "1" through a random diffuser using terahertz waves and a 3D-printed diffractive network. This single-pixel all-optical object classification system through random diffusers is based on passive diffractive layers that process broadband input light and can operate at any part of the electromagnetic spectrum by simply scaling the diffractive features proportional to the wavelength range of interest. These results have various potential applications in

e.g.

biomedical imaging

security

robotics

and autonomous driving.

关键词

Keywords

references

Ji, N., Milkie, D. E. & Betzig, E. Adaptive optics via pupil segmentation for high-resolution imaging in biological tissues.Nat. Methods7, 141–147 (2010).

Ntziachristos, V. Going deeper than microscopy: the optical imaging frontier in biology.Nat. Methods7, 603–614 (2010).

Jaffe, J. S., Moore, K. D., Mclean, J. & Strand, M. R. Underwater optical imaging: status and prospects.Oceanography14, 64–66 (2001).

Schettini, R. & Corchs, S. Underwater image processing: state of the art of restoration and image enhancement methods.EURASIP J. Adv. Signal Process.2010, 746052, https://doi.org/10.1155/2010/746052 (2010).

Jia, Z. et al. A two-step approach to see-through bad weather for surveillance video quality enhancement.Mach. Vis. Appl.23, 1059–1082 (2012).

Johnson-Roberson, M. et al. High-resolution underwater robotic vision-based mapping and three-dimensional reconstruction for archaeology.J. Field Robot.34, 625–643 (2017).

He, K., Sun, J. & Tang, X. Single image haze removal using dark channel prior.IEEE Trans. Pattern Anal. Mach. Intell.33, 2341–2353 (2011).

Hao, Z., You, S., Li, Y., Li, K. & Lu, F. Learning from synthetic photorealistic raindrop for single image raindrop removal. InProc. IEEE/CVF International Conference on Computer Vision Workshops(IEEE, 2019).

Roggemann, M. C., Welsh, B. M. & Hunt, B. R.Imaging Through Turbulence(CRC Press, 1996).

Popoff, S., Lerosey, G., Fink, M., Boccara, A. C. & Gigan, S. Image transmission through an opaque material.Nat. Commun.1, 81 (2010).

Wang, K. et al. Direct wavefront sensing for high-resolution in vivo imaging in scattering tissue.Nat. Commun.6, 7276, https://doi.org/10.1038/ncomms8276 (2015).

Horstmeyer, R., Ruan, H. & Yang, C. Guidestar-assisted wavefront-shaping methods for focusing light into biological tissue.Nat. Photonics9, 563–571 (2015).

Katz, O., Heidmann, P., Fink, M. & Gigan, S. Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations.Nat. Photonics8, 784–790 (2014).

Edrei, E. & Scarcelli, G. Memory-effect based deconvolution microscopy for super-resolution imaging through scattering media.Sci. Rep.6, 33558, https://doi.org/10.1038/srep33558 (2016).

Li, X., Greenberg, J. A. & Gehm, M. E. Single-shot multispectral imaging through a thin scatterer.Optica6, 864–871 (2019).

Velten, A. et al. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging.Nat. Commun.3, 745 (2012).

Li, Y., Xue, Y. & Tian, L. Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media.Optica5, 1181 (2018).

Li, S., Deng, M., Lee, J., Sinha, A. & Barbastathis, G. Imaging through glass diffusers using densely connected convolutional networks.Optica5, 803–813 (2018).

Tahir, W., Wang, H. & Tian, L. Adaptive 3D descattering with a dynamic synthesis network.Light Sci. Appl11, 42 (2022).

Shi, Y. et al. Prior-free imaging unknown target through unknown scattering medium.Opt. Express30, 17635–17651 (2022).

Fan, P., Zhao, T. & Su, L. Deep learning the high variability and randomness inside multimode fibers.Opt. Express27, 20241–20258 (2019).

Sun, Y., Shi, J., Sun, L., Fan, J. & Zeng, G. Image reconstruction through dynamic scattering media based on deep learning.Opt. Express27, 16032–16046 (2019).

Ando, T., Horisaki, R. & Tanida, J. Speckle-learning-based object recognition through scattering media.Opt. Express23, 33902 (2015).

Satat, G., Tancik, M., Gupta, O., Heshmat, B. & Raskar, R. Object classification through scattering media with deep learning on time resolved measurement.Opt. Express25, 17466 (2017).

Le, X. et al. Direct object recognition without line-of-sight using optical coherence. InProc. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)11729–11738 (IEEE, 2019).

Luo, Y. et al. Computational imaging without a computer: seeing through random diffusers at the speed of light.eLight2, 4 (2022).

Li, Y., Luo, Y., Bai, B. & Ozcan, A. Analysis of diffractive neural networks for seeing through random diffusers.IEEE J. Sel. Top. Quantum Electron.29, 1–17 (2022).

Lin, X. et al. All-optical machine learning using diffractive deep neural networks.Science361, 1004 (2018).

Li, J., Mengu, D., Luo, Y., Rivenson, Y. & Ozcan, A. Class-specific differential detection in diffractive optical neural networks improves inference accuracy.Adv. Photonics1, 046001 (2019).

Mengu, D., Luo, Y., Rivenson, Y. & Ozcan, A. Analysis of diffractive optical neural networks and their integration with electronic neural networks.IEEE J. Sel. Top. Quantum Electron.26, 1–14 (2019).

Rahman, M. S. S., Li, J., Mengu, D., Rivenson, Y. & Ozcan, A. Ensemble learning of diffractive optical networks.Light Sci. Appl.10, 14 (2021).

Rahman, M. S. S. & Ozcan, A. Computer-free, all-optical reconstruction of holograms using diffractive networks.ACS Photonics8, 3375–3384 (2021).

Mengu, D. & Ozcan, A. All-optical phase recovery: diffractive computing for quantitative phase imaging.Adv. Opt. Mater.10, 2200281 (2022).

Bai, B. et al. To image, or not to image: class-specific diffractive cameras with all-optical erasure of undesired objects.eLight2, 14, https://doi.org/10.1186/s43593-022-00021-3 (2022).

Qian, C. et al. Performing optical logic operations by a diffractive neural network.Light Sci. Appl.9, 59, https://doi.org/10.1038/s41377-020-0303-2 (2020).

Luo, Y., Mengu, D. & Ozcan, A. Cascadable all-optical NAND gates using diffractive networks.Sci. Rep.12, 7121 (2022).

Kulce, O., Mengu, D., Rivenson, Y. & Ozcan, A. All-optical synthesis of an arbitrary linear transformation using diffractive surfaces.Light Sci. Appl.10, 196 (2021).

Li, J., Hung, Y. -C., Kulce, O., Mengu, D. & Ozcan, A. Polarization multiplexed diffractive computing: all-optical implementation of a group of linear transformations through a polarization-encoded diffractive network.Light Sci. Appl.11, 153 (2022).

Yan, T. et al. Fourier-space diffractive deep neural network.Phys. Rev. Lett.123, 023901 (2019).

Chen, H. et al. Diffractive deep neural networks at visible wavelengths.Engineering7, 1483–1491 (2021).

Huang, Z. et al. All-optical signal processing of vortex beams with diffractive deep neural networks.Phys. Rev. Appl.15, 014037 (2021).

Li, Y., Chen, R., Sensale-Rodriguez, B., Gao, W. & Yu, C. Real-time multi-task diffractive deep neural networks via hardware-software co-design.Sci. Rep.11, 11013 (2021).

Xiao, Y. -L., Li, S., Situ, G. & You, Z. Unitary learning for diffractive deep neural network.Opt. Lasers Eng.139, 106499 (2021).

Zhou, T. et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit.Nat. Photonics15, 367–373 (2021).

Liu, C. et al. A programmable diffractive deep neural network based on a digital-coding metasurface array.Nat. Electron.5, 113–122 (2022).

Luo, X. et al. Metasurface-enabled on-chip multiplexed diffractive neural networks in the visible.Light Sci. Appl.11, 158 (2022).

Wang, P. et al. Diffractive deep neural network for optical orbital angular momentum multiplexing and demultiplexing.IEEE J. Sel. Top. Quantum Electron.28, 1–11 (2022).

Zhu, H. H. et al. Space-efficient optical computing with an integrated chip diffractive neural network.Nat. Commun.13, 1044 (2022).

Veli, M. et al. Terahertz pulse shaping using diffractive surfaces.Nat. Commun.12, 37 (2021).

Luo, Y. et al. Design of task-specific optical systems using broadband diffractive neural networks.Light Sci. Appl.8, 1–14 (2019).

Li, J. et al. Spectrally encoded single-pixel machine vision using diffractive networks.Sci. Adv.7, eabd7690 (2021).

LeCun, Y. et al. Handwritten digit recognition with a back-propagation network. InProc.Advances in Neural Information Processing Systems2. (ed. Touretzky, D. S.) 396–404 (Morgan-Kaufmann, 1990).

Mengu, D. et al. Misalignment resilient diffractive optical networks.Nanophotonics9, 4207–4219 (2020).

Mengu, D., Veli, M., Rivenson, Y. & Ozcan, A. Classification and reconstruction of spatially overlapping phase images using diffractive optical networks.Sci. Rep.12, 8446 (2022).

Li, J., Bai, B., Luo, Y. & Ozcan, A. Massively parallel universal linear transformations using a wavelength-multiplexed diffractive optical network.Adv. Photonics5, 016003 (2022).

Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. InProc. 3rd International Conference on Learning Representations(ICLR), (2015).

Turan, D., Corzo-Garcia, S. C., Yardimci, N. T., Castro-Camus, E. & Jarrahi, M. Impact of the metal adhesion layer on the radiation power of plasmonic photoconductive terahertz sources.J. Infrared Millim. Terahertz Waves38, 1448–1456 (2017).

Yardimci, N. T. & Jarrahi, M. High sensitivity terahertz detection through large-area plasmonic nano-antenna arrays.Sci. Rep.7, 42667 (2017).

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

Light | 光计算实现透过散射介质的物体分类

上海交通大学邹卫文杰青团队招聘博士后、专职科研、长聘岗位

暂无数据