Ultra-compact multi-task processor based on in-memory optical computing

Wencan Liu; Yuyao Huang; Run Sun; Tingzhao Fu; Sigang Yang; Hongwei Chen

doi:10.1038/s41377-025-01814-0

Your Location：

Home >

Browse articles >

Ultra-compact multi-task processor based on in-memory optical computing

Original Articles

- Ultra-compact multi-task processor based on in-memory optical computing
- Ultra-compact multi-task processor based on in-memory optical computing
- LSA 2025年14卷第5期页码：1364-1376
- Affiliations：
  
  1.Department of Electronic Engineering, Tsinghua University, Beijing, China
  2.Beijing National Research Center for Information Science and Technology (BNRist), Beijing, China
  3.Hunan Provincial Key Laboratory of Novel Nano Optoelectronic Information Materials and Devices, College of Advanced Interdisciplinary Studies, National University of Defense Technology, Changsha, China
- Author bio：
  
  Hongwei Chen (chenhw@tsinghua.edu.cn)
- Funds：
- DOI：10.1038/s41377-025-01814-0
  中图分类号：
- 收稿日期：2024-10-28，
  
  修回日期：2025-02-13，
  
  录用日期：2025-03-05，
  
  网络出版日期：2025-03-24，
  
  纸质出版日期：2025-05-31
- Accepted：
Scan QR Code
Ultra-compact multi-task processor based on in-memory optical computing[J]. LSA, 2025,14(5):1364-1376.

Liu, W. C. et al. Ultra-compact multi-task processor based on in-memory optical computing. Light: Science & Applications, 14, 1364-1376 (2025).
Ultra-compact multi-task processor based on in-memory optical computing[J]. LSA, 2025,14(5):1364-1376. DOI： 10.1038/s41377-025-01814-0.

Liu, W. C. et al. Ultra-compact multi-task processor based on in-memory optical computing. Light: Science & Applications, 14, 1364-1376 (2025). DOI： 10.1038/s41377-025-01814-0.

摘要

Abstract

To enhance the computational density and energy efficiency of on-chip neuromorphic hardware

this study introduces a novel network architecture for multi-task processing with in-memory optical computing. On-chip optical neural networks are celebrated for their capability to transduce a substantial volume of parameters into optical form while conducting passive

computing

yet they encounter challenges in scalability and multitasking. Leveraging the principles of transfer learning

this approach involves embedding the majority of parameters into fixed optical components and a minority into adjustable electrical components. Furthermore

with deep regression algorithm in modeling physical propagation process

a compact optical neural network achieve to handle diverse tasks. In this work

two ultra-compact in-memory diffraction-based chips with integration of more than 60

000 parameters/mm

were fabricated

employing deep neural network model and the hard parameter sharing algorithm

to perform multifaceted classification and regression tasks

respectively. The experimental results demonstrate that these chips achieve accuracies comparable to those of electrical networks while significantly reducing the power-intensive digital computation by 90%. Our work heralds strong potential for advancing in-memory optical computing frameworks and next generation of artificial intelligence platforms.

关键词

Keywords

references

Forsyth, D. A. & Ponce, J. Computer Vision: A Modern Approach (Prentice Hall Professional Technical Reference, 2002).

He, K. M. et al. Deep residual learning for image recognition. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition . Las Vegas, NV, USA: 770–778 (IEEE, 2016).

Brown, T. B. et al. Language models are few-shot learners. In: Proc. 34th International Conference on Neural Information Processing Systems. Vancouver, Canada, 159 (Curran Associates Inc., 2020).

Chowdhary, K. R. Natural language processing. in Fundamentals of Artificial Intelligence (ed Chowdhary, K. R.) 603–649 (Springer, 2020).

Yurtsever, E. et al. A survey of autonomous driving: common practices and emerging technologies. IEEE Access 8 , 58443–58469 (2020)..

LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521 , 436–444 (2015)..

Krizhevsky, A. et al. ImageNet classification with deep convolutional neural networks. Commun. ACM 60 , 84–90 (2017)..

LeCun, Y. et al. Gradient-based learning appliedto document recognition. Proc. IEEE 86 , 2278–2324 (1998)..

Hennessy, J. L. & Patterson, D. A. Computer Architecture: A Quantitative Approach . 5th edn. (Elsevier, 2011).

Zhang, C. et al. Optimizing FPGA-based accelerator design for deep convolutional neural networks. In: Proc. ACM/SIGDA International Symposium on Field-Programmable Gate Arrays . Monterey, CA, USA, 161–170 (ACM, 2015).

Kirk, D. NVIDIA cuda software and GPU parallel computing architecture. In: Proc. 6th International Symposium on Memory Management . Montreal, Canada. 103–104 (ACM, 2007).

Theis, T. N. & Wong, H. S. P. The end of Moore’s law: a new beginning for information technology. Comput. Sci. Eng. 19 , 41–50 (2017)..

LeCun, Y. 1.1 deep learning hardware: past, present, and future. In: Proc. IEEE International Solid-State Circuits Conference-(ISSCC) . San Francisco, CA, USA: 12–19 (IEEE, 2019).

Cheng, Q. X. et al. Recent advances in optical technologies for data centers: a review. Optica 5 , 1354–1370 (2018)..

Wetzstein, G. et al. Inference in artificial intelligence with deep optics and photonics. Nature 588 , 39–47 (2020)..

Yao, J. P. Microwave photonics. J. Lightwave Technol. 27 , 314–335 (2009)..

Fu, T. Z. et al. Optical neural networks: progress and challenges. Light Sci. Appl. 13 , 263 (2024)..

Shen, Y. C. et al. Deep learning with coherent nanophotonic circuits. Nat. Photonics 11 , 441–446 (2017)..

Bogaerts, W. et al. Programmable photonic circuits. Nature 586 , 207–216 (2020)..

Zhang, H. et al. An optical neural chip for implementing complex-valued neural network. Nat. Commun. 12 , 457 (2021)..

Xu, S. F. et al. Optical coherent dot-product chip for sophisticated deep learning regression. Light Sci. Appl. 10 , 221 (2021)..

Hughes, T. W. et al. Training of photonic neural networks through in situ backpropagation and gradient measurement. Optica 5 , 864–871 (2018)..

Tait, A. N. et al. Broadcast and weight:an integrated network for scalable photonic spike processing. J. Lightwave Technol. 32 , 3427–3439 (2014)..

Huang, C. R. et al. A silicon photonic-electronic neural network for fibre nonlinearity compensation. Nat. Electron. 4 , 837–844 (2021)..

Ashtiani, F., Geers, A. J. & Aflatouni, F. An on-chip photonic deep neural network for image classification. Nature 606 , 501–506 (2022)..

Xu, X. Y. et al. 11 tops photonic convolutional accelerator for optical neural networks. Nature 589 , 44–51 (2021)..

Bai, B. W. et al. Microcomb-based integrated photonic processing unit. Nat. Commun. 14 , 66 (2023)..

Feldmann, J. et al. Parallel convolutional processing using an integrated photonic tensor core. Nature 589 , 52–58 (2021)..

Wu, C. M. et al. Programmable phase-change metasurfaces on waveguides for multimode photonic convolutional neural network. Nat. Commun. 12 , 96 (2021)..

Feldmann, J. et al. All-optical spiking neurosynaptic networks with self-learning capabilities. Nature 569 , 208–214 (2019)..

Ding, X. M. et al. Metasurface-based optical logic operators driven by diffractive neural networks. Adv. Mater. 36 , 2308993 (2024)..

Wang, X. W. et al. Advances in information processing and biological imaging using flat optics. Nat. Rev. Electr. Eng. 1 , 391–411 (2024)..

Silva, A. et al. Performing mathematical operations with metamaterials. Science 343 , 160–163 (2014)..

Goi, E., Schoenhardt, S. & Gu, M. Direct retrieval of Zernike-based pupil functions using integrated diffractive deep neural networks. Nat. Commun. 13 , 7531 (2022)..

Fu, T. Z. et al. Photonic machine learning with on-chip diffractive optics. Nat. Commun. 14 , 70 (2023)..

Yan, T. e t al. All-optical graph representation learning using integrated diffractive photonic computing units. Sci. Adv. 8 , eabn7630 (2022)..

Wang, Z. et al. Integrated photonic metasystem for image classifications at telecommunication wavelength. Nat. Commun. 13 , 2131 (2022)..

Zhu, H. H. et al. Space-efficient optical computing with an integrated chip diffractive neural network. Nat. Commun. 13 , 1044 (2022)..

Huang, Y. Y. et al. Sophisticated deep learning with on-chip optical diffractive tensor processing. Photonics Res. 11 , 1125–1138 (2023)..

Liu, W. C. et al. C-DONN: compact diffractive optical neural network with deep learning regression. Opt. Express 31 , 22127-22143 (2023)..

Fu, T. Z. et al. On-chip photonic diffractive optical neural network based on a spatial domain electromagnetic propagation model. Opt. Express 29 , 31924–31940 (2021)..

Lin, X. et al. All-opt ical machine learning using diffractive deep neural networks. Science 361 , 1004–1008 (2018)..

Chen, Y. T. et al. All-analog photoelectronic chip for high-speed vision tasks. Nature 623 , 48–57 (2023)..

Xu, Z. H. et al. A multichannel optical computing architecture for advanced machine vision. Light Sci. Appl. 11 , 255 (2022)..

Zhou, T. K. et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat. Photonics 15 , 367–373 (2021)..

Mengu, D. et al. Analysis of diffractive optical neural networks and their integration with electronic neural networks. IEEE J. Sel. Top. Quantum Electron. 26 , 3700114 (2020)..

Li, Y. J. et al. Real-time multi-task diffractive deep neural networks via hardware-software co-design. Sci. Rep. 11 , 11013 (2021)..

Caruana, R. Multitask learning: a knowledge-based source of inductive bia s. In: Proc. Tenth International Conference on International Conference on Machine Learning. Amherst, MA, USA . 41–48 (Morgan Kaufmann Publishers Inc.,1993).

Ruder, S. An overview of multi-task learning in deep neural networks. Print at https://doi.org/10.48550/arXiv.1706.05098 https://doi.org/10.48550/arXiv.1706.05098 (2017).

Sun, T. X. et al. Learning sparse sharing architectures for multiple tasks. In: Proc. of the 34th AAAI Conference on Artificial Intelligence . New York, NY, USA 8936–8943 (AAAI, 2020).

Huang, Y. Y. et al. Diffraction-driven parallel convolution processing with integrated photonics. Laser Photonics Rev . https://doi.org/10.1002/lpor.202400972 https://doi.org/10.1002/lpor.202400972 (2024)..

Peurifoy, J. et al. Nanophotonic particle simulation and inverse design using artificial neural networks. Sci. Adv. 4 , eaar4206 (2018)..

Strogatz, S. H. Exploring complex networks. Nature 410 , 268–276 (2001)..

Trabelsi, C. et al. Deep complex networks. In: Proc. 6th International Conference on Learning Representations . Vancouver, BC, Canada (ICLR, 2018).

Xu, Z. H. et al. Large-scale photonic chiplet Taichi empowers 160-TOPS/W artificial general intelligence. Science 384 , 202–209 (2024)..

Sun, R. et al. Multimode diffractive optical neural network. Adv. Photonics Nexus 3 , 026007–026007 (2024)..

Matthès, M. W. et al. Learning and avoiding disorder in multimode fibers. Phys. Rev. X 11 , 021060 (2021)..

Choquette, J. et al. NVIDIA a100 tensor core GPU: performance and innovation. IEEE Microsc. 41 , 29–35 (2021)..

Choquette, J. NVIDIA hopper h100 GPU: scaling performance. IEEE Microsc. 43 , 9–17 (2023)..

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

暂无数据