Analog spatiotemporal feature extraction for cognitive radio-frequency sensing with integrated photonics

Shaofu Xu; Binshuo Liu; Sicheng Yi; Jing Wang; Weiwen Zou

doi:10.1038/s41377-024-01390-9

Your Location：

Home >

Browse articles >

Analog spatiotemporal feature extraction for cognitive radio-frequency sensing with integrated photonics

Original Articles

- Analog spatiotemporal feature extraction for cognitive radio-frequency sensing with integrated photonics
- Light: Science & Applications Vol. 13, Issue 3, Pages: 478-487(2024)
- Affiliations：
  
  State Key Laboratory of Advanced Optical Communication Systems and Networks, Intelligent Microwave Lightwave Integration Innovation Center (imLic), Department of Electronic Engineering, Shanghai Jiao Tong University, Shanghai, China
- Author bio：
  
  Weiwen Zou (wzou@sjtu.edu.cn)
- Funds：
- DOI：10.1038/s41377-024-01390-9
  CLC：
- Received：14 July 2023，
  
  Revised：04 January 2024，
  
  Accepted：2024-01-18，
  
  Published Online：14 February 2024，
  
  Published：31 March 2024
- Accepted：
Scan QR Code
Xu, S. F. et al. Analog spatiotemporal feature extraction for cognitive radio-frequency sensing with integrated photonics.,Light: Science & Applications, 13, 478-487 (2024).
DOI：

Xu, S. F. et al. Analog spatiotemporal feature extraction for cognitive radio-frequency sensing with integrated photonics.,Light: Science & Applications, 13, 478-487 (2024). DOI： 10.1038/s41377-024-01390-9.

摘要

Abstract

Analog feature extraction (AFE) is an appealing strategy for low-latency and efficient cognitive sensing systems since key features are much sparser than the Nyquist-sampled data. However

applying AFE to broadband radio-frequency (RF) scenarios is challenging due to the bandwidth and programmability bottlenecks of analog electronic circuitry. Here

we introduce a photonics-based scheme that extracts spatiotemporal features from broadband RF signals in the analog domain. The feature extractor structure inspired by convolutional neural networks is implemented on integrated photonic circuits to process RF signals from multiple antennas

extracting valid features from both temporal and spatial dimensions. Because of the tunability of the photonic devices

the photonic spatiotemporal feature extractor is trainable

which enhances the validity of the extracted features. Moreover

a digital-analog-hybrid transfer learning method is proposed for the effective and low-cost training of the photonic feature extractor. To validate our scheme

we demonstrate a radar target recognition task with a 4-GHz instantaneous bandwidth. Experimental results indicate that the photonic analog feature extractor tackles broadband RF signals and reduces the sampling rate of analog-to-digital converters to 1/4 of the Nyquist sampling while maintaining a high target recognition accuracy of 97.5%. Our scheme offers a promising path for exploiting the AFE strategy in the realm of cognitive RF sensing

with the potential to contribute to the efficient signal processing involved in applications such as autonomous driving

robotics

and smart factories.

关键词

Keywords

references

Liu, L. K. et al. Computing systems for autonomous driving: state of the art and challenges. IEEE Internet Things J. 8 , 6469–6486 (2021)..

Chen, Q. et al. F-Cooper: feature-based cooperative perception for autonomous vehicle edge computing system using 3D point clouds. Proceedings of the 4th ACM/IEEE Symposium on Edge Computing. Arlington: ACM, 2019, 88–100.

Huang, P. et al. Edge Robotics: edge-computing-accelerated multirobot simultaneous localization and mapping. IEEE Internet Things J. 9 , 14087–14102 (2022)..

Yu, N. N. et al. Image features extraction and fusion based on joint sparse representation. IEEE J. Sel. Top. Signal Process. 5 , 1074–1082 (2011)..

Liu, Z. H. et al. Structured optimal graph based sparse feature extraction for semi-supervised learning. Signal Process. 170 , 107456 (2020)..

Verhelst, M. & Bahai, A. Where analog meets digital: analog-to-information conversi on and beyond. IEEE Solid-State Circuits Mag. 7 , 67–80 (2015)..

Murmann, B., Verhelst, M.,&Manoli, Y. Analog-to-information conversion. in Nano-Chips 2030 (eds Murmann, B.&Hoefflinger, B. ) (Cham: Springer, 2020), 275–292.

Yang, M. H. et al. A 1μW voice activity Detector using analog feature extraction and digital deep neural network. Proceedings of 2018 IEEE International Solid-State Circuits Conference. San Francisco: IEEE, 2018, 346–348.

Yang, M. H. et al. Nanowatt acoustic inference sensing exploiting nonlinear analog feature extraction. IEEE J. Solid-State Circuits 56 , 3123–3133 (2021)..

Sadasivuni, S. et al. In-sensor neural network for high energy efficiency analog-to-information conversion. Sci. Rep. 12 , 18253 (2022)..

Sharifshazileh, M. et al. An electronic neuromorphic system for real-time detection of high frequency oscillations (HFO) in intracranial EEG. Nat. Commun. 12 , 3095 (2021)..

Shrestha, A. et al. Continuous human activity classification from FMCW radar with Bi-LSTM networks. IEEE Sens. J. 20 , 13607–13619 (2020)..

Saponara, S., Greco, M. S. & Gini, F. Radar-on-chip/in-package in autonomous driving vehicles and intelligent transport systems: opportunities and challenges. IEEE Signal Process. Mag. 36 , 71–84 (2019)..

Kirolos, S. et al. Analog-to-information conversion via random demodulation. Proceedings in IEEE Dallas/CAS Workshop on Design, Applications, Integration and Software. Richardson: IEEE, 2006, 71–74.

Pelissier, M. & Studer, C. Non-uniform wavelet sampling for RF analog-to-information conversion. IEEE Trans. Circuits Syst. I Regul. Pap. 65 , 471–484 (2018)..

Marpaung, D., Yao, J. P. & Capmany, J. Integrated microwave photonics. Nat. Photonics 13 , 80–90 (2019)..

Marpaung, D. et al. Low-power, chip-based stimulated Brillouin scattering microwave photonic filter with ultrahigh selectivity. Optica 2 , 76–83 (2015)..

Chi, H., Wang, C. & Yao, J. P. Photonic generation of wideband chirped microwave waveforms. IEEE J. Microw. 1 , 787–803 (2021)..

Tan, M. X. et al. RF and microwave photonic temporal signal processing with Kerr micro-combs. Adv. Phys. X 6 , 1838946 (2021)..

Xu, S. F. et al. Deep-learning-powered photonic analog-to-digital conversion. Light Sci. Appl. 8 , 66 (2019)..

Bogaerts, W. et al. Programmable photonic circuits. Nature 586 , 207–216 (2020)..

Pérez-López, D. et al. Multipurpose self-configuration of programmable photonic circuits. Nat. Commun. 11 , 6359 (2020)..

Dong, M. et al. High-speed programmable photonic circuits in a cryogenically compatible, visible–near-infrared 200 mm CMOS architecture. Nat. Photonics 16 , 59–65 (2022)..

Shastri, B. J. et al. Photonics for artificial intelligence and neuromorphic computing. Nat. Photonics 15 , 102–114 (2021)..

Zhou, H. L. et al. Photonic matrix multiplication lights up photonic accelerator and beyond. Light Sci. Appl. 11 , 30 (2022)..

Huang, C. R. et al. Prospects and applications of photonic neural networks. Adv. Phys. : X 7 , 1981155 (2022)..

Xu, S. F. et al. Parallel optical coherent dot-product architecture for large-scale matrix multiplication with compatibility for diverse phase shifters. Opt. Express 30 , 42057–42068 (2022)..

Tait, A. N. Quantifying power in silicon photonic neural networks. Phys. Rev. Appl. 17 , 054029 (2022)..

Shen, Y. C. et al. Deep learning with coherent nanophotonic circuits. Nat. Photonics 11 , 441–446 (2017)..

Sludds, A. et al. Delocalized photonic deep learning on the internet's edge. Science 378 , 270–276 (2022)..

Zhang, H. et al. An optical neural chip for implementing complex-valued neural network. Nat. Commun. 12 , 457 (2021)..

Huang, C. R. et al. A silicon photonic–electronic neural network for fibre nonlinearity compensation. Nat. Electron. 4 , 837–844 (2021)..

Mourgias-Alexandris, G. et al. Noise-resilient and high-speed deep learning with coherent silicon photonics. Nat. Commun. 13 , 5572 (2022)..

Xu, S. F. et al. Optical coherent dot-product chip for sophisticated deep learning regression. Light Sci. Appl. 10 , 221 (2021)..

Xu, X.Y. et al. 11 TOPS photonic convolutional accelerator for optical neural networks. Nature 589 , 44–51 (2021)..

Feldmann, J. et al. Parallel convolutional processing using an integrated photonic tensor core. Nature 589 , 52–58 (2021)..

Meng, X. Y. et al. Compact optical convolution processing unit based on multimode interference. Nat. Commun. 14 , 3000 (2023)..

Zhou, T. K. et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat. Photonics 15 , 367–373 (2021)..

Ashtiani, F., Geers, A. J. & Aflatouni, F. An on-chip photonic deep neural network for image classification. Nature 606 , 501–506 (2022)..

Wang, T. Y. et al. Image sensing with multilayer nonlinear optical neural networks. Nat. Photonics 17 , 408–415 (2023)..

Wang, Z. et al. Integrated photonic metasystem for image classifications at telecommunication wavelength. Nat. Commun. 13 , 2131 (2022)..

Zhang, W. P. et al. Broadband physical layer cognitive radio with an integrated photonic processor for blind source separation. Nat. Commun. 14 , 1107 (2023)..

Xu, S. F. et al. High-order tensor flow processing using integrated photonic circuits. Nat. Commun. 13 , 7970 (2022)..

Williamson, I. A. D. et al. Reprogrammable electro-optic nonlinear activation functions for optical neural networks. IEEE J. Sel. Top. Quantum Electron. 26 , 7700412 (2020)..

Huang, Y. et al. Programmable low-threshold optical nonlinear activation functions for photonic neural networks. Opt. Lett. 47 , 1810–1813 (2022)..

Campo, J. R. R. & Pérez-López, D. Reconfigurable activation functions in integrated optical neural networks. IEEE J. Sel. Top. Quantum Electron. 28 , 8300513 (2022)..

Xu, S. F., Wang, J. & Zou, W. W. Optical convolutional neural network with WDM-based optical patching and microring weighting banks. IEEE Photonics Technol. Lett. 33 , 89–92 (2021)..

Horst, F. et al. Cascaded Mach-Zehnder wavelength filters in silicon photonics for low loss and flat pass-band WDM (de-) multiplexing. Opt. Express 21 , 11652–11658 (2013)..

Van Der Maaten, L. & Hinton, G. Visualizing data using t-SNE. J. Mach. Learn. Res. 9 , 2579–2605 (2008)..

Zhang, W. P. et al. Silicon microring synapses enable photonic deep learning beyond 9-bit precision. Optica 9 , 579–584 (2022)..

Knott, E. F. Radar cross section. in Radar Handbook 3rd edn (McGraw-Hill, 2008), 14.1–14.36.

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰