High-clockrate free-space optical in-memory computing

Yuanhao Liang; James Wang; Kaiwen Xue; Xinyi Ren; Ran Yin; Shaoyuan Ou; Lian Zhou; Yuan Li; Tobias Heuser; Niels Heermeier; Ian Christen; James A. Lott; Stephan Reitzenstein; Mengjie Yu; Zaijun Chen

doi:10.1038/s41377-026-02206-8

Your Location：

Home >

Browse articles >

High-clockrate free-space optical in-memory computing

Original Articles

- High-clockrate free-space optical in-memory computing
- High-clockrate free-space optical in-memory computing
- Light: Science & Applications 2026年15卷第4期页码：1292-1304
- Affiliations：
  
  1.Department of Electrical Engineering and Computing Sciences, University of California, Berkeley, CA 94720, USA
  2.Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, CA 90089, USA
  3.Institut für Physik und Astronomie, Technische Universität Berlin, Berlin, Germany
- Author bio：
  
  Zaijun Chen (zaijun@berkeley.edu)
- Funds：
- DOI：10.1038/s41377-026-02206-8
  中图分类号：
- 收稿：2025-09-22，
  
  修回：2026-01-19，
  
  录用：2026-01-21，
  
  网络首发：2026-02-13，
  
  纸质出版：2026-04-30
- Accepted：
Scan QR Code
Yuanhao Liang, James Wang, Kaiwen Xue, 等. High-clockrate free-space optical in-memory computing[J]. Light: Science & Applications, 2026,15(4):1292-1304.

Liang, Y. H. et al. High-clockrate free-space optical in-memory computing. Light: Science & Applications, 15, 1292-1304 (2026).
Yuanhao Liang, James Wang, Kaiwen Xue, 等. High-clockrate free-space optical in-memory computing[J]. Light: Science & Applications, 2026,15(4):1292-1304. DOI： 10.1038/s41377-026-02206-8.

Liang, Y. H. et al. High-clockrate free-space optical in-memory computing. Light: Science & Applications, 15, 1292-1304 (2026). DOI： 10.1038/s41377-026-02206-8.

摘要

Abstract

The ability to process and act on data in real time is increasingly critical for applications ranging from autonomous vehicles

three-dimensional environmental sensing

and remote robotics. However

the deployment of deep neural networks (DNNs) in edge devices is hindered by the lack of energy-efficient scalable computing hardware. Here

we introduce a fanout spatial time-of-flight optical neural network (FAST-ONN) that calculates billions of convolutions per second with ultralow latency and power consumption. This is enabled by the combination of high-speed dense arrays of vertical-cavity surface-emitting lasers (VCSELs) for input modulation with spatial light modulators of high pixel counts for in-memory weighting. In a three-dimensional optical system

parallel differential readout allows signed weight values for accurate inference in a single shot. The performance is benchmarked with feature extraction in You-Only-Look-Once (YOLO) for convolution at 100 million frames per second (MFPS)

and in-system backward propagation training with photonic reprogrammability. The VCSEL transmitters are implementable in any free-space optical computing systems to improve the clockrate to over gigahertz

where the high scalability in device counts and channel parallelism enables a new avenue to scale up free space computing hardware.

关键词

Keywords

references

LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521 , 436–444 (2015)..

Liu, S. S. et al. Edge computing for autonomous driving: opportunities and challenges. Proc. IEEE 107 , 1697–1716 (2019)..

Cui, Y. D. et al. Deep learning for image and point cloud fusion in autonomous driving: a review. IEEE Trans. Intell. Transp. Syst. 23 , 722–739 (2022)..

Mao, B. M. et al. Security and privacy on 6G network edge: a survey. IEEE Commun. Surv. Tutor. 25 , 1095–112 7 (2023)..

Kufakunesu, R., Myburgh, H. & De Freitas, A. The internet of battle things: a survey on communication challenges and recent solutions. Discov. Internet Things 5 , 3 (2025)..

Tahir, N. & Parasuraman, R. Edge computing and its application in robotics: a survey. J. Sens. Actuator Netw. 14 , 65 (2025)..

Motiwala, Z. Y. et al. Telesurgery: current status and strategies for latency reduction. J. Robot. Surg. 19 , 153 (2025)..

Theis, T. N. & Wong, H. S. P. The end of Moore's law: a new beginning for information technology. Comput. Sci. Eng. 19 , 41–50 (2017)..

Xu, X. et al. Scaling for edge inference of deep neural networks. Nat. Electron. 1 , 216–222 (2018)..

Markidis, S. et al. NVIDIA tensor core programmability, performance & precision. In Proc. 2018 IEEE International Parallel and Distributed Processing Symposium Workshops (IEEE, Vancouver, BC, Canada, 2018).

Jouppi, N. et al. T PU v4: an optically reconfigurable supercomputer for machine learning with hardware support for embeddings. In Proc. 50th Annual International Symposium on Computer Architecture 82 (ACM, Orlando, 2023).

Qiu, J. T. et al. Going deeper with embedded FPGA platform for convolutional neural network. In Proc. 2016 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (ACM, Monterey, 2016).

Modha, D. S. et al. Neural inference at the frontier of energy, space, and time. Science 382 , 329–335 (2023)..

Sipola, T. et al. Artificial intelligence in the IoT era: a review of edge AI hardware and software. In Proc. 2022 31st Conference of Open Innovations Association 320–331 (IEEE, Helsinki, 2022).

Mirza, F. et al. Effect of through-silicon-via joule heating on device performance for low-powered mobile applications. J. Electron. Packag. 136 , 041008 (2014)..

Tu, K. N., Liu, Y. X. & Li, M. L. Effect of Joule heating and current crowding on electromigration in mobile technology. Appl. Phys. Rev. 4 , 011101 (2017)..

Im, M. S. et al. Optimization pro blems with low SWAP tactical computing. In Proc. SPIE 11013, Disruptive Technologies in Information Sciences II (SPIE, Baltimore, 2019).

Blasch, E. et al. Edge-based computing challenges and opportunities for sensor fusion: panel review. In Proc. SPIE 13479, Signal Processing, Sensor/Information Fusion, and Target Recognition XXXIV 16 (SPIE, Orlando, 2025).

Liu, H. B., Wu, C. & Wang, H. J. Real time object detection using LiDAR and camera fusion for autonomous driving. Sci. Rep. 13 , 8056 (2023)..

Benoit-Cattin, T., Velasco-Montero, D. & Fernández-Berni, J. Impact of thermal throttling on long-term visual inference in a CPU-based edge device. Electronics 9 , 2106 (2020)..

Shi, W. S. et al. Edge computing: vision and challenges. IEEE Internet Things J. 3 , 637–646 (2016)..

Shen, Y. C. et al. Deep learning with coherent nanophotonic circuits. Nat. Photonics 11 , 441–446 (2017)..

Pai, S. et al. Experimentally realized in situ backpropagation for deep learning in photonic neural networks. Science 380 , 398–404 (2023)..

Bandyopadhyay, S. et al. Single-chip photonic deep neural network with forward-only training. Nat. Photonics 18 , 1335–1343 (2024)..

Ohno, S. et al. Si microring resonator crossbar array for on-chip inference and training of the optical neural network. ACS Photonics 9 , 2614–2622 (2022)..

Ren, Y. et al. A multifunctional optoelectronic memristor for in-memory computing. Light Adv. Manuf. 6 , 413–415 (2025)..

Xu, X. Y. et al. 11 TOPS photonic convolutional accelerator for optical neural networks. Nature 589 , 44–51 (2021)..

Dong, B. W. et al. Higher-dimensional processing using a photonic tensor core with continuous-time data. Nat. Photonics 17 , 1080–1088 (2023)..

Feldmann, J. et al. Parallel convolutional processing using an integrated photonic tensor core. Nature 589 , 52–58 (2021)..

Ou, S. Y. et al. Hypermultiplexed integrated photonics-based optical tensor pr ocessor. Sci. Adv. 11 , eadu0228 (2025)..

Lin, Z. J. et al. 120 GOPS Photonic tensor core in thin-film lithium niobate for inference and in situ training. Nat. Commun. 15 , 9081 (2024)..

Hu, Y. W. et al. Integrated lithium niobate photonic computing circuit based on efficient and high-speed electro-optic conversion. Nat. Commun. 16 , 8178 (2025)..

Hua, S. Y. et al. An integrated large-scale photonic accelerator with ultralow latency. Nature 640 , 361–367 (2025)..

Ahmed, S. R. et al. Universal photonic artificial intelligence acceleration. Nature 640 , 368–374 (2025)..

Lin, X. et al. All-optical machine learning using diffractive deep neural networks. Science 361 , 1004–1008 (2018)..

Li, J. X. et al. Spectrally encoded single-pixel machine vision using diffractive networks. Sci. Adv. 7 , eabd7690 (2021)..

Wang, T. Y. et al. An optical neural netwo rk using less than 1 photon per multiplication. Nat. Commun. 13 , 123 (2022)..

Hamerly, R. et al. Large-scale optical neural networks based on photoelectric multiplication. Phys. Rev. X 9 , 021032 (2019)..

Luo, Y. et al. Design of task-specific optical systems using broadband diffractive neural networks. Light Sci. Appl. 8 , 112 (2019)..

Luo, X. H. et al. Metasurface-enabled on-chip multiplexed diffractive neural networks in the visible. Light Sci. Appl. 11 , 158 (2022)..

Miscuglio, M. et al. Massively parallel amplitude-only Fourier neural network. Optica 7 , 1812–1819 (2020)..

Liu, Y. et al. Optical Fourier convolutional neural network with high efficiency in image classification. Opt. Express 32 , 23575–23583 (2024)..

Zhou, T. K. et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat. Photonics 15 , 367–373 (2021)..

Bernstein, L. et al. Single-shot optical neural network. Sci. Adv. 9 , eadg7904(2023)..

Wang, T. Y. et al. Image sensing with multilayer nonlinear optical neural networks. Nat. Photonics 17 , 408–415 (2023)..

Kalinin, K. P. et al. Analog optical computer for AI inference and combinatorial optimization. Nature 645 , 354–361 (2025)..

Redmon, J. et al. You only look once: unified, real-time object detection. In Proc. 2016 IEEE Conference on Computer Vision and Pattern Recognition (IEEE, Las Vegas, NV, USA, 2016).

Chen, Z. J. et al. Deep learning with coherent VCSEL neural networks. Nat. Photonics 17 , 723–730 (2023)..

Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. In Proc. 3rd International Conference on Learning Representations (ICLR, San Diego, USA, 2015).

He, K. M. et al. Deep residual learning for image recognition. In Proc. 2016 IEEE Conference on Computer Vision and Pattern Recognition 770–778 (IEEE, Las Vegas, NV, USA, 2016).

Xu, W. Z. et al. Accelerating convolutional neural network by exploiting sparsity on GPUs. ACM Trans. Archit. Code Optim. 20 , 36 (2023)..

Ronneberger, O., Fischer, P. & Brox, T. U-Net: convolutional networks for biomedical image segmentation. In Proc. 18th International Conference on Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015 . 234–241 (Springer, Munich, Germany, 2015)..

Heidari, E. et al. Hexagonal transverse-coupled-cavity VCSEL redefining the high-speed lasers. Nanophotonics 9 , 4743–4748 (2020)..

Hubara, I. et al. Quantized neural networks: training neural networks with low precision weights and activations. J. Mach. Learn. Res. 18 , 1–30 (2018)..

Chen, L. et al. Deep neural network based vehicle and pedestrian detection for autonomous driving: a survey. IEEE Trans. Intell. Transp. Syst. 22 , 3234–3246 (2021)..

Jia, X. et al. Fast and accurate object detector for autonomous driving based on improved YOLOv5. Sci. Rep. 13 , 9711 (2023)..

Li, K. Q. et al. Real-time monocular joint perception network for autonomous driving. IEEE Trans. Intell. Transp. Syst. 23 , 15864–15877 (2022)..

Lin, T. Y. et al. Microsoft COCO: common objects in context. In Proc. 13th European Conference on Computer Vision (ECCV) 2014. 740–755 (Springer, Zurich, Switzerland, 2014).

Wright, L. G. et al. Deep physical neural networks trained with backpropagation. Nature 601 , 549–555 (2022)..

NVIDIA. NVIDIA H100 GPU. 2026. at https://www.nvidia.com/en-us/data-center/h100/URL https://www.nvidia.com/en-us/data-center/h100/URL .

Shekhar, S. et al. Roadmapping the next generation of silicon photonics. Nat. Commun. 15 , 751 (2024)..

Zhou, Z. P., Yin, B. & Michel, J. On-chip light sources for silicon photonics. Light Sci. Appl. 4 , e358 (2015)..

Leuthold, J., Koos, C. & Freude, W. Nonlinear silicon photonics. Nat. Photonics 4 , 535–544 (2010)..

Davenport, M. L., Liu, S. T. & Bowers, J. E. Integrated heterogeneous silicon/III–V mode-locked lasers. Photonics Res. 6 , 468–478 (2018)..

Tait, A. N. et al. Neuromorphic photonic networks using silicon photonic weight banks. Sci. Rep. 7 , 7430 (2017)..

Ruan, Z. S. et al. Flexible orbital angular momentum mode switching in multimode fibre using an optical neural network chip. Light Adv. Manuf. 5 , 296–307 (2024)..

Tyszka, K. et al. Leaky integrate-and-fire mechanism in exciton–polariton condensates for photonic spiking neurons. Laser Photonics Rev. 17 , 2100660 (2023)..

Tyszka, K., Opala, A. & Piętka, B. Advancing optical spiking neural networks with exciton-polaritons. J. Phys. Photonics 7 , 041002 (2025)..

Matuszewski, M. et al. Energy-efficient neural network inference with microcavity exciton polaritons. Phys. Rev. Appl. 16 , 024045 (2021)..

Opala, A. et al. Perovskite microwires for room temperature exciton-polariton neural network. Adv. Mater. 37 , e07612 (2025)..

Chen, S. Q. et al. Optical generative models. Nature 644 , 903–911 (2025)..

Heuser, T. et al. Developing a photonic hardware platform for brain-inspired computing based on 5 × 5 VCSEL arrays. J. Phys. Photonics 2 , 044002 (2020)..

Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. In Proc. 3rd International Conference on Learning Representations (ICLR, San Diego, USA, 2015).

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

暂无数据