Integrated photonic 3D tensor processing engine

Yue Wu; Ziheng Ni; Xin Li; Yuanxun Wang; Liangjun Lu; Jianping Chen; Linjie Zhou

doi:10.1038/s41377-026-02183-y

Your Location：

Home >

Browse articles >

Integrated photonic 3D tensor processing engine

Original Articles

- Integrated photonic 3D tensor processing engine
- Integrated photonic 3D tensor processing engine
- Light: Science & Applications 2026年15卷第5期页码：1679-1689
- Affiliations：
  
  1.State Key Laboratory of Photonics and Communications, School of Integrated Circuits, Shanghai Jiao Tong University, Shanghai 200240, China
  2.SJTU-Pinghu Institute of Intelligent Optoelectronics, Pinghu 314200, China
- Author bio：
  
  Liangjun Lu (luliangjun@sjtu.edu.cn)
  Linjie Zhou (ljzhou@sjtu.edu.cn)
- Funds：
- DOI：10.1038/s41377-026-02183-y
  中图分类号：
- 收稿：2025-04-16，
  
  修回：2025-12-27，
  
  录用：2026-01-03，
  
  网络首发：2026-03-06，
  
  纸质出版：2026-05-31
- Accepted：
Scan QR Code
Yue Wu, Ziheng Ni, Xin Li, 等. Integrated photonic 3D tensor processing engine[J]. Light: Science & Applications, 2026,15(5):1679-1689.

Wu, Y. et al. Integrated photonic 3D tensor processing engine. Light: Science & Applications, 15, 1679-1689 (2026).
Yue Wu, Ziheng Ni, Xin Li, 等. Integrated photonic 3D tensor processing engine[J]. Light: Science & Applications, 2026,15(5):1679-1689. DOI： 10.1038/s41377-026-02183-y.

Wu, Y. et al. Integrated photonic 3D tensor processing engine. Light: Science & Applications, 15, 1679-1689 (2026). DOI： 10.1038/s41377-026-02183-y.

摘要

Abstract

Optical computing leverages high bandwidth

low latency

and power efficiency

which is considered as one of the most effective solutions for accelerating deep learning tasks. However

mainstream photonic hardware accelerators are primarily optimized for two-dimensional (2D) matrix-vector multiplications (MVMs). To implement three-dimensional (3D) convolutional neural networks (CNNs)

high-order tensors must be reshaped in the electrical domain according to the size of the accelerators before computation

leading to extra memory usage and time overheads. Additionally

synchronization across multiple channels depends on external electronic clocks

which increases the complexity of the system. In this work

we propose an integrated photonic 3D tensor processing engine (3D-TPE) based on the interleaving modulation of time

wavelength

and space. Data caching

channel synchronization and computation are realized entirely within the optical domain

reducing memory and time usage

and simplifying the system. Optical caching and synchronization are achieved with an optical tunable delay line (OTDL) chip supporting versatile clock frequencies up to 200 GHz

and optical computing is accomplished with a dual-coupled micro-ring resonators (MRRs) based crossbar chip with a 3-dB passband width of 50 GHz. We verify the processing capabilities of the 3D-TPE at clock frequencies ranging from 10 GHz to 30 GHz and perform a proof-of-concept experiment for a LiDAR 3D point cloud image recognition task operating at 20 GHz

achieving a recognition accuracy of 97.06%. The proposed 3D-TPE is anticipated to facilitate high-order tensor convolutions

playing an important role in autonomous driving

healthcare

video analytics

virtual reality

etc.

关键词

Keywords

references

LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521 , 436–444 (2015)..

Alzubaidi, L. et al. Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. Journal of Big Data 8 , 53 (2021)..

Ha, N. et al. Machine Learning-Enabled Smart Sensor Systems. Advanced Intelligent Systems 2 , 2000063 (2020)..

Ballard, Z. et al. Machine learning and computation-enabled intelligent sensor design. Nature Machine Intelligence 3 , 556–565 (2021)..

Sidiropoulos, N. D. et al. Tensor Decomposition for Signal Processing and Machine Learning. IEEE Transactions on Signal Processing 65 , 3551–3582 (2017)..

Li, Z. et al. A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects. IEEE Transactions on Neural Networks and Learning Systems 33 , 6999–7019 (2022)..

Kamnitsas, K. et al. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Medical Image Analysis 36 , 61–78 (2017)..

Niyas, S. et al. Medical image segmentation with 3D convolutional neural networks: A survey. Neurocomputing 493 , 397–413 (2022)..

Zhao, D. et al. An attentive and adaptive 3D CNN for automatic pulmonary nodule detection in CT image. Expert Systems with Applications 211 , 118672 (2023)..

Duan, H. et al. Revisiting Skeleton-based Action Recognition. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR ) (2022).

De Castro, G. Z., Guerra, R. R. & Guimarães, F. G. Automatic translation of sign language with multi-stream 3D CNN and generation of artificial depth maps. Expert Systems with Applications 215 , 119394 (2023)..

Meng, Z. et al. HYDRO-3D: Hybrid Object Detection and Tracking for Cooperative Perception Using 3D LiDAR. IEEE Transactions on Intelligent Vehicles 8 , 4069–4080 (2023)..

Maturana, D. & Scherer, S. VoxNet: A 3D Convolutional Neural Network for real-time object recognition. 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2015).

Hong, Y.-Y. & Pula, R. A. Detection and classification of faults in photovoltaic arrays using a 3D convolutional neural network. Energy 246 , 123391 (2022)..

Faraji, M. An integrated 3D CNN-GRU deep learning method for short-term prediction of PM2.5 concentration in urban environment. Sci. Total Environ. (2022).

Li, Y., Zhang, H. & Shen, Q. Spectral–Spatial Classification of Hyperspectral Imagery with 3D Convolutional Neural Network. Remote Sensing 9 , 67 (2017)..

OpenAI. AI and compute. (2018) . at https://openai.com/index/ai-and-compute/ https://openai.com/index/ai-and-compute/ .

OpenAI. AI and efficiency. at https://openai.com/index/ai-and-efficiency/ https://openai.com/index/ai-and-efficiency/ (2020).

Mittal, S. & Vaishay, S. A survey of techniques for optimizing deep learning on GPUs. Journal of Systems Architecture 99 , 101635 (2019)..

Jouppi, N. P. et al. In-Datacenter Performance Analysis of a Tensor Processing Unit. Proceedings of the 44th Annual International Symposium on Computer Architecture (2017).

Yao, P. et al. Fully hardware-implemented memristor convolutional neural network. Nature 577 , 641–646 (2020)..

Shawahna, A., Sait, S. M. & El-Maleh, A. FPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A Review. IEEE Access 7 , 7823–7859 (2019)..

Mittal, S. & Vibhu A survey of accelerator architectures for 3D convolution neural networks. Journal of Systems Architecture 115 , 102041 (2021)..

Miller, D. A. B. Attojoule Optoelectronics for Low-Energy Information Processing and Communications. Journal of Lightwave Technology 35 , 346–396 (2017)..

Boes, A. et al. Lithium niobate photonics: Unlocking the electromagnetic spectrum. Science 379 , eabj4396 (2023)..

Wan, Z. et al. Ultra-Degree-of-Freedom Structured Light for Ultracapacity Information Carriers. ACS Photonics 10 , 2149–2164 (2023)..

Shen, Y. et al. Deep learning with coherent nanophotonic circuits. Nature Photonics 11 , 441–446 (2017)..

Zhang, H. et al. An optical neural chip for implementing complex-valued neural network. Nature Communications 12 , 457 (2021)..

Pai, S. et al. Experimentally realized in situ backpropagation for deep learning in photonic neural networks. Science 380 , 398–404 (2023)..

Zhan, Y. et al. Physics-Aware Analytic-Gradient Training of Photonic Neural Networks. Laser & Photonics Reviews 18 , 2300445 (2024)..

Feldmann, J. et al. Parallel convolutional processing using an integrated photonic tensor core. Nature 589 , 52–58 (2021)..

Huang, C. et al. A silicon photonic–electronic neural network for fibre nonlinearity compensation. Nature Electronics 4 , 837–844 (2021)..

Sludds, A. et al. Delocalized photonic deep learning on the internet’s edge. (2022).

Zhang, W. et al. Broadband physical layer cognitive radio with an integrated photonic processor for blind source separation. Nature Communications 14 , 1107 (2023)..

Dong, B. et al. Higher-dimensional processing using a photonic tensor core with continuous-time data. Nature Photonics 17 , 1080–1088 (2023)..

Dong, B. et al. Partial coherence enhances parallelized photonic computing. Nature 632 , 55–62 (2024)..

Xu, X. et al. 11 TOPS photonic convolutional accelerator for optical neural networks. Nature 589 , 44–51 (2021)..

Xu, S. et al. High-order tensor flow processing using integrated photonic circuits. Nature Communications 13 , 7970 (2022)..

Bai, B. et al. Microcomb-based integrated photonic processing unit. Nature Communications 14 , 66 (2023)..

Zhang, J. et al. Compact, efficient, and scalable nanobeam core for photonic matrix vector multiplication. Optica , https://doi.org/10.1364/OPTICA.506603 (2024).

Li, X. et al. Ultra-low-loss multi-layer 8 × 8 microring optical switch. Photonics Research 11 , 712 (2023)..

Li, X. et al. Low-Loss and Power-Efficient Polarization-Diversity 4 × 4 Microring Switch on a Multi-Layer Si3N4-on-SOI Platform. J. Lightwave Technol . 1–10, (2024).

Zhang, W. et al. Silicon microring synapses enable photonic deep learning beyond 9-bit precision. Optica 9 , 579 (2022)..

Liu, X. et al. Single-Monitor Calibration for Multiple Microring Syna pses. ACS Photonics , acsphotonics.4c00157, https://doi.org/10.1021/acsphotonics.4c00157 https://doi.org/10.1021/acsphotonics.4c00157 (2024)..

Cheng, J. et al. Self-calibrating microring synapse with dual-wavelength synchronization. Photonics Research 11 , 347 (2023)..

Che, D. & Chen, X. Modulation format and digital signal processing for IM-DD optics at post-200G era. Journal of Lightwave Technology 42 , 588–605 (2024)..

Guo, Y. et al. Deep Learning for 3D Point Clouds: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 43 , 4338–4364 (2021)..

Deuge, M. D. et al. Unsupervised feature learning for classification of outdoor 3D scans. Australasian conference on robitics and automation. 2 , 1 (2013)..

De Koninck, Y. et al. GaAs nano-ridge laser diodes fully fabricated ina 300-mm CMOS pilot line. Nature 637 , 63–69 (2025)..

Shu, H. et al. Microcomb-driven silicon photonic systems. Nature 605 , 457–463 (2022)..

Sobhanan, A. et al. Semiconductor optical amplifiers: recent advances and applications. Advances in Optics and Photonics 14 , 571 (2022)..

Yang, X. et al. Non-Volatile Optical Switch Element Enabled by Low-Loss Phase Change Material. Adv. Funct. Materi. 2304601, (2023).

Hong, S. et al. Multimode-enabled silicon photonic delay lines: Break the delay-density limit. Light: Science & Applications 14 , 145 (2025)..

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

Phonon-polaritonic skyrmions: transition from bubble- to Néel-type

Robust chirality through merging BICs

Integrated optomechanical ultrasonic sensors with nano-Pascal-level sensitivity

High-dimensional multiplexing through vortex electromagnetic wave manipulation by space-time-coding metasurfaces

Minimalist optical system for achromatic imaging within extended field of view based on monolithic integrated meta-axicon cluster