Super-resolution diffractive neural network for all-optical direction of arrival estimation beyond diffraction limits

Sheng Gao; Hang Chen; Yichen Wang; Zhengyang Duan; Haiou Zhang; Zhi Sun; Yuan Shen; Xing Lin

doi:10.1038/s41377-024-01511-4

Your Location：

Home >

Browse articles >

Super-resolution diffractive neural network for all-optical direction of arrival estimation beyond diffraction limits

Original Articles

- Super-resolution diffractive neural network for all-optical direction of arrival estimation beyond diffraction limits
- Light: Science & Applications Vol. 13, Issue 8, Pages: 1602-1614(2024)
- Affiliations：
  
  1.Department of Electronic Engineering, Tsinghua University, Beijing 100084, China
  2.Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing 100084, China
- Author bio：
  
  Xing Lin (lin-x@tsinghua.edu.cn)
- Funds：
- DOI：10.1038/s41377-024-01511-4
  CLC：
- Received：09 January 2024，
  
  Revised：03 June 2024，
  
  Accepted：2024-06-25，
  
  Published Online：10 July 2024，
  
  Published：31 August 2024
- Accepted：
Scan QR Code
Gao, S. et al. Super-resolution diffractive neural network for all-optical direction of arrival estimation beyond diffraction limits.,Light: Science & Applications, 13, 1602-1614 (2024).
DOI：

Gao, S. et al. Super-resolution diffractive neural network for all-optical direction of arrival estimation beyond diffraction limits.,Light: Science & Applications, 13, 1602-1614 (2024). DOI： 10.1038/s41377-024-01511-4.

摘要

Abstract

Wireless sensing of the wave propagation direction from radio sources lays the foundation for communication

radar

navigation

etc. However

the existing signal processing paradigm for the direction of arrival estimation requires the radio frequency electronic circuit to demodulate and sample the multichannel baseband signals followed by a complicated computing process

which places the fundamental limit on its sensing speed and energy efficiency. Here

we propose the super-resolution diffractive neural networks (S-DNN) to process electromagnetic (EM) waves directly for the DOA estimation at the speed of light. The multilayer meta-structures of S-DNN generate super-oscillatory angular responses in local angular regions that can perform the all-optical DOA estimation with angular resolutions beyond the diffraction limit. The spatial-temporal multiplexing of passive and reconfigurable S-DNNs is utilized to achieve high-resolution DOA estimation over a wide field of view. The S-DNN is validated for the DOA estimation of multiple radio sources over 5 GHz frequency bandwidth with estimation latency over two to four orders of magnitude lower than the state-of-the-art commercial devices in principle. The results achieve the angular resolution over an order of magnitude

experimentally demonstrated with four times

higher than diffraction-limited resolution. We also apply S-DNN's edge computing capability

assisted by reconfigurable intelligent surfaces

for extremely low-latency integrated sensing and communication with low power consumption. Our work is a significant step towards utilizing photonic computing processors to facilitate various wireless sensing and communication tasks with advantages in both computing paradigms and performance over electronic computing.

关键词

Keywords

references

Schmidt, R. Multiple emitter location and signal parameter estimation. IEEE Trans. Antennas Propag. 34 , 276–280 (1986)..

Tan, Z., Eldar, Y. C. & Nehorai, A. Direction of arrival estimation using co-prime arrays: a super resol ution viewpoint. IEEE Trans. Signal Process. 62 , 5565–5576 (2014)..

Huang, H. J. et al. Deep learning for super-resolution channel estimation and DOA estimation based massive MIMO system. IEEE Trans. Vehicular Technol. 67 , 8549–8560 (2018)..

Lin, X. Artificial intelligence built on wireless signals. Nat. Electron. 5 , 69–70 (2022)..

Dang, S. P. et al. What should 6 G be? Nat. Electron. 3 , 20–29 (2020)..

Bogaerts, W. et al. Programmable photonic circuits. Nature 586 , 207–216 (2020)..

Shastri, B. J. et al. Photonics for artificial intelligence and neuromorphic computing. Nat. Photonics 15 , 102–114 (2021)..

Wetzstein, G. et al. Inference in artificial intelligence with deep optics and photonics. Nature 588 , 39–47 (2020)..

Miscuglio, M. et al. Massively parallel amplitude-only Fourier neural network. Optica 7 , 1812–1819 (2020)..

Zuo, Y. et al. All-optical neural network with nonlinear activation functions. Optica 6 , 1132–1137 (2019)..

Feldmann, J. et al. Parallel convolutional processing using an integrated photonic tensor core. Nature 589 , 52–58 (2021)..

Feldmann, J. et al. All-optical spiking neurosynaptic networks with self-learning capabilities. Nature 569 , 208–214 (2019)..

Shen, Y. C. et al. Deep learning with coherent nanophotonic circuits. Nat. Photonics 11 , 441–446 (2017)..

Xu, X. Y. et al. 11 TOPS photonic convolutional accelerator for optical neural networks. Nature 589 , 44–51 (2021)..

Antonik, P. et al. Human action recognition with a large-scale brain-inspired photonic computer. Nat. Mach. Intell. 1 , 530–537 (2019)..

Vandoorne, K. et al. Experimental demonstration of reservoir computing on a silicon photonics chip. Nat. Commun. 5 , 3541 (2014)..

Sludds, A. et al. Delocalized photonic deep learning on the internet's edge. Science 378 , 270–276 (2022)..

Pai, S. et al. Experimentally realized in situ backpropagation for deep learning in photonic neural networks. Science 380 , 398–404 (2023)..

Li, Y. H. et al. Quantitative phase imaging (QPI) through random diffusers using a diffractive optical network. Light 4 , 19 (2023)..

Luo, Y. et al. Computational imaging without a computer: seeing through random diffusers at the speed of light. eLight 2 , 4 (2022)..

Wu, Z. et al. Three-dimensional nanoscale reduced-angle ptycho-tomographic imaging with deep learning (RAPID). eLight 3 , 7 (2023)..

Zhou, Z. et al. Prospects and applications of on-chip lasers. eLight 3 , 1 (2023)..

Marpaung, D., Yao, J. P. & Capmany, J. Integrated microwave photonics. Nat. Photonics 13 , 80–90 (2019)..

Liu, W. L. et al. A fully reconfigurable photonic integrated signal processor. Nat. Photonics 10 , 190–195 (2016)..

Zhang, W. P. et al. Broadband physical layer cognitive radio with an integrated photonic processor for blind source separation. Nat. Commun. 14 , 1107 (2023)..

Gao, S., Wu, C. & Lin, X. Demixing microwave signals using system-on-chip photonic processor. Light 13 , 58 (2024)..

Lin, X. et al. All-optical machine learning using diffractive deep neural networks. Science 361 , 1004–1008 (2018)..

Liu, C. et al. A programmable diffractive deep neural network based on a digital-coding metasurface array. Nat. Electron. 5 , 113–122 (2022)..

Qian, C. et al. Dynamic recognition and mirage using neuro-metamaterials. Nat. Commun. 13 , 2694 (2022)..

Gao, X. X. et al. Programmable surface plasmonic neural networks for microwave detection and processing. Nat. Electron. 6 , 319–328 (2023)..

Chen, M. K. et al. A meta-device for intelligent depth perception. Adv. Mater. 35 , 2107465 (2023)..

Cui, T. J. et al. Coding metamaterials, digital metamaterials and programmable metamaterials. Light 3 , e218 (2014)..

Huang, C. X. et al. Polarization modulation for wireless communications based on metasurfaces. Adv. Funct. Mater. 31 , 2103379 (2021)..

Zhang, L. et al. A wireless communication scheme based on space- and frequency-division multiplexing using digital metasurfaces. Nat. Electron. 4 , 218–227 (2021)..

Wei, M. L. et al. Metasurface-enabled smart wireless attacks at the physical layer. Nat. Electron. 6 , 610–618 (2023)..

Zhang, X. G. et al. An optically driven digital metasurface for programming electromagnetic functions. Nat. Electron. 3 , 165–171 (2020)..

Wei, L. et al. Channel estimation for RIS-empowered multi-user MISO wireless co mmunications. IEEE Trans. Commun. 69 , 4144–4157 (2021)..

Zhu, J. A. et al. Sensing RISs: enabling dimension-independent CSI acquisition for beamforming. IEEE Trans. Inf. Theory 69 , 3795–3813 (2023)..

Zheng, Z. Y. et al. Dual adaptive training of photonic neural networks. Nat. Mach. Intell. 5 , 1119–1129, https://doi.org/10.1038/s42256-023-00723-4 (2023)..

Wood, R. W. Physical Optics . (New York: The MacMillan Company, 1919).

Zhou, T. K. et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat. Photonics 15 , 367–373 (2021)..

Xiong, H. et al. A metamaterial energy power detector based on electromagnetic energy harvesting technology. ACS Appl. Electron. Mater. 6 , 1204–1210 (2024)..

Li, W. H. et al. Intelligent metasurface system for automatic tracking of moving targets and wireless communications based on computer vision. Nat. Commun. 14 , 989 (2023)..

Huang, M. et al. Machine-learning-enabled metasurface for direction of arrival estimation. Nanophotonics 11 , 2001–2010 (2022)..

Li, L. L. et al. Intelligent metasurface imager and recognizer. Light 8 , 97 (2019)..

Li, L. L. et al. Electromagnetic reprogrammable coding-metasurface holograms. Nat. Commun. 8 , 197 (2017)..

Zhang, J. C. et al. A 6 G meta-device for 3D varifocal. Sci. Adv. 9 , eadf8478 (2023)..

Chen, Y. T. et al. Photonic unsupervised learning variational autoencoder for high-throughput and low-latency image transmission. Sci. Adv. 9 , eadf8437, https://doi.org/10.1126/sciadv.adf8437 (2023)..

Goi, E., Schoenhardt, S. & Gu, M. Direct retrieval of Zernike-based pupil functions using integrated diffractive deep neural networks. Nat. Commun. 13 , 7531 (2022)..

Zhang, Y. N. et al. Ultra-broadband directional scattering by colloidally lithographed high-index Mie resonant oligomers and their energy-harvesting applications. ACS Appl. Mater. Interfaces 10 , 16776–16782 (2018)..

Chen, H. et al. Diffractive deep neural networks at visible wavelengths. Engineering 7 , 1483–1491 (2021)..

Duan, Z. Y., Chen, H. & Lin, X. Optical multi-task learning using multi-wavelength diffractive deep neural networks. Nanophotonics 12 , 893–903 (2023)..

Zheng, H. Y. et al. Multichannel meta-imagers for accelerating machine vision. Nat. Nanotechnol. 19 , 471–478 (2024)..

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰