Deep learning with photonic neural cellular automata

Gordon H. Y. Li; Christian R. Leefmans; James Williams; Robert M. Gray; Midya Parto; Alireza Marandi

doi:10.1038/s41377-024-01651-7

Go to Nature Website

Your Location：

Home >

Browse articles >

Deep learning with photonic neural cellular automata

Original Articles

- Deep learning with photonic neural cellular automata
- Light: Science & Applications Vol. 13, Issue 12, Pages: 3030-3040(2024)
- Affiliations：
  
  1.Department of Applied Physics, California Institute of Technology, Pasadena, CA, USA
  2.Department of Electrical Engineering, California Institute of Technology, Pasadena, CA, USA
  3.Physics and Informatics Laboratories, NTT Research Inc., Sunnyvale, CA, USA
- Author bio：
  
  Alireza Marandi (marandi@caltech.edu)
- Funds：
- DOI：10.1038/s41377-024-01651-7
  CLC：
- Published：31 December 2024，
  
  Published Online：08 October 2024，
  
  Received：13 January 2024，
  
  Revised：17 September 2024，
  
  Accepted：22 September 2024
- Accepted：
Scan QR Code
Li, G. H. Y. et al. Deep learning with photonic neural cellular automata. Light: Science & Applications, 13, 3030-3040 (2024).
DOI：

Li, G. H. Y. et al. Deep learning with photonic neural cellular automata. Light: Science & Applications, 13, 3030-3040 (2024). DOI： 10.1038/s41377-024-01651-7.

摘要

Abstract

Rapid advancements in deep learning over the past decade have fueled an insatiable demand for efficient and scalable hardware. Photonics offers a promising solution by leveraging the unique properties of light. However

conventional neural network architectures

which typically require dense programmable connections

pose several practical challenges for photonic realizations. To overcome these limitations

we propose and experimentally demonstrate Photonic Neural Cellular Automata (PNCA) for photonic deep learning with sparse connectivity. PNCA harnesses the speed and interconnectivity of photonics

as well as the self-organizing nature of cellular automata through local interactions to achieve robust

reliable

and efficient processing. We utilize linear light interference and parametric nonlinear optics for all-optical computations in a time-multiplexed photonic network to experimentally perform self-organized image classification. We demonstrate binary (two-class) classification of images using as few as 3 programmable photonic parameters

achieving high experimental accuracy with the ability to also recognize out-of-distribution data. The proposed PNCA approach can be adapted to a wide range of existing photonic hardware and provides a compelling alternative to conventional photonic neural networks by maximizing the advantages of light-based computing whilst mitigating their practical challenges. Our results showcase the potential of PNCA in advancing photonic deep learning and highlights a path for next-generation photonic computers.

关键词

Keywords

references

LeCun, Y., Bengio, Y.&Hinton, G. Deep learning.Nature521, 436–444 (2015)..

Vaswani, A. et al. Attention is all you need. InProc. 31st International Conference on Neural Information Processing Systems. 6000–6010 (Curran Associates Inc., Long Beach, CA, USA, 2017).

Jumper, J. et al. Highly accurate protein structure prediction with AlphaFold.Nature596, 583–589 (2021)..

Goodfellow, I. J. et al. Generative adversarial nets. InProc. 27th International Conference on Neural Information Processing Systems. 2672–2680 (MIT Press, Montreal, Canada, 2014).

Sevilla, J. et al. Compute trends across three eras of machine learning. InProc. 2022 International Joint Conference on Neural Networks (IJCNN). 1–8 (IEEE, Padua, Italy, 2022).

Jouppi, N. P. et al. In-datacenter performance analysis of a tensor processing unit. InProc. 44th Annual International Symposium on Computer Architecture. 1–12 (ACM, Toronto, ON, Canada, 2017).

Shastri, B. J. et al. Photonics for artificial intelligence and neuromorphic computing.Nat. Photonics15, 102–114 (2021)..

Wetzstein, G. et al. Inference in artificial intelligence with deep optics and photonics.Nature588, 39–47 (2020)..

Shen, Y. C. et al. Deep learning with coherent nanophotonic circuits.Nat. Photonics11, 441–446 (2017)..

Feldmann, J. et al. Parallel convolutional processing using an integrated photonic tensor core.Nature589, 52–58 (2021)..

Xu, X. Y. et al. 11 TOPS photonic convolutional accelerator for optical neural networks.Nature589, 44–51 (2021)..

Feldmann, J. et al. All-optical spiking neurosynaptic networks with self-learning capabilities.Nature569, 208–214 (2019)..

Li, G. H. Y. et al. All-optical ultrafast ReLU function for energy-efficient nanophotonic deep learning.Nanophotonics12, 847–855 (2023)..

Ashtiani, F., Geers, A. J.&Aflatouni, F. An on-chip photonic deep neural network for image classification.Nature606, 501–506 (2022)..

Miller, D. A. B. Are optical transistors the logical next step?Nat. Photonics4, 3–5 (2010)..

Sze, V. et al. Efficient processing of deep neural networks: a tutorial and survey.Proc. IEEE105, 2295–2329 (2017)..

Farhat, N. H. et al. Optical implementation of the Hopfield model.Appl. Opt.24, 1469–1475 (1985)..

Zhou, T. K. et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit.Nat. Photonics15, 367–373 (2021)..

Lin, X. et al. All-optical machine learning using diffractive deep neural networks.Science361, 1004–1008 (2018)..

Mordvintsev, A. et al. Growing neural cellular automata.Distill5, e23 (2020)..

Wolfram, S. Statistical mechanics of cellular automata.Rev. Mod. Phys.55, 601–644 (1983)..

Li, G. H. Y. et al. Photonic elementary cellular automata for simulation of complex phenomena.Light Sci. Appl.12, 132 (2023)..

Gardner, M. The fantastic combinations of John Conway's new solitaire game "life".Sci. Am.223, 120–123 (1970)..

Randazzo, E. et al. Self-classifying MNIST digits.Distill5, e00027–002 (2020)..

Niklasson, E. et al. Self-organising textures.Distill6, e00027–003 (2021)..

Tait, A. N. et al. Neuromorphic photonic networks using silicon photonic weight banks.Sci. Rep.7, 7430 (2017)..

Williamson, I. A. D. et al. Reprogrammable electro-optic nonlinear activation functions for optical neural networks.IEEE J. Sel. Top. Quantum Electron.26, 7700412 (2020)..

Werbos, P. J. Backpropagation through time: what it does and how to do it.Proc. IEEE78, 1550–1560 (1990)..

Wright, L. G. et al. Deep physical neural networks trained with backpropagation.Nature601, 549–555 (2022)..

Pai, S. et al. Experimentally realized in situ backpropagation for deep learning in photonic neural networks.Science380, 398–404 (2023)..

Leefmans, C. et al. Topological dissipation in a time-multiplexed photonic resonator network.Nat. Phys.18, 442–449 (2022)..

Langrock, C.&Fejer, M.M. Fiber-feedback continuous-wave and synchronously-pumped singly-resonant ring optical parametric oscillators using reverse-proton-exchanged periodically-poled lithium niobate waveguides.Opt. Lett.32, 2263–2265 (2007)..

Xiao, H., Rasul, K.&Vollgraf, R. Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. Preprint athttps://doi.org/10.48550/arXiv.1708.07747https://doi.org/10.48550/arXiv.1708.07747(2017).

Guo, C. et al. On calibration of modern neural networks. InProc. 34th International Conference on Machine Learning. 1321–1330 (PMLR, Sydney, Australia, 2017).

Cohen, G. et al. EMNIST: extending MNIST to handwritten letters. InProc. 2017 International Joint Conference on Neural Networks (IJCNN). 2921–2926 (IEEE, Anchorage, AK, USA, 2017).

Deng, L. The MNIST database of handwritten digit images for machine learning research [best of the web].IEEE Signal Process. Mag.29, 141–142 (2012)..

Krizhevsky, A.Learning Multiple Layers of Features from Tiny Images. MSc thesis, University of Toronto (2009).

Cook, M. Universality in elementary cellular automata.Complex Syst.15, 1–40 (2004)..

Porte, X. et al. A complete, parallel and autonomous photonic neural network in a semiconductor multimode laser.J. Phys. Photonics3, 024017 (2021)..

Zhou, T. K. et al. Ultrafast dynamic machine vision with spatiotemporal photonic computing.Sci. Adv.9, eadg4391 (2023)..

Yildirim, M. et al. Nonlinear optical feature generator for machine learning.APL Photonics8, 106104 (2023)..

Yuan, L. Q. et al. Synthetic dimension in photonics.Optica5, 1396–1405 (2018)..

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰