Reconfigurable nonlinear photonic activation function for photonic neural network based on non-volatile opto-resistive RAM switch

Zefeng Xu; Baoshan Tang; Xiangyu Zhang; Jin Feng Leong; Jieming Pan; Sonu Hooda; Evgeny Zamburg; Aaron Voon-Yew Thean

doi:10.1038/s41377-022-00976-5

Go to Nature Website

Your Location：

Home >

Browse articles >

Reconfigurable nonlinear photonic activation function for photonic neural network based on non-volatile opto-resistive RAM switch

Original Articles

- Reconfigurable nonlinear photonic activation function for photonic neural network based on non-volatile opto-resistive RAM switch
- Reconfigurable nonlinear photonic activation function for photonic neural network based on non-volatile opto-resistive RAM switch
- LSA 2022年11卷第10期页码：2512-2522
- Affiliations：
  
  1.Integrative Sciences and Engineering Programme, NUS Graduate School, National University of Singapore, Singapore, Singapore
  2.Department of Electrical and Computer Engineering, National University of Singapore, 4 Engineering Drive 3, Singapore 117583, Singapore
- Author bio：
  
  Zefeng Xu (xuzefeng@u.nus.edu)
  Aaron VoonYew Thean (Aaron.Thean@nus.edu.sg)
- Funds：
- DOI：10.1038/s41377-022-00976-5
  中图分类号：
- 纸质出版日期：2022-10-31，
  
  网络出版日期：2022-10-06，
  
  收稿日期：2022-02-23，
  
  修回日期：2022-08-15，
  
  录用日期：2022-08-30
- Accepted：
Scan QR Code
Reconfigurable nonlinear photonic activation function for photonic neural network based on non-volatile opto-resistive RAM switch[J]. LSA, 2022,11(10):2512-2522.

Xu, Z. F. et al. Reconfigurable nonlinear photonic activation function for photonic neural network based on non-volatile opto-resistive RAM switch. Light: Science & Applications, 11, 2512-2522 (2022).
Reconfigurable nonlinear photonic activation function for photonic neural network based on non-volatile opto-resistive RAM switch[J]. LSA, 2022,11(10):2512-2522. DOI： 10.1038/s41377-022-00976-5.

Xu, Z. F. et al. Reconfigurable nonlinear photonic activation function for photonic neural network based on non-volatile opto-resistive RAM switch. Light: Science & Applications, 11, 2512-2522 (2022). DOI： 10.1038/s41377-022-00976-5.

摘要

Abstract

Photonic neural network has been sought as an alternative solution to surpass the efficiency and speed bottlenecks of electronic neural network. Despite that the integrated Mach–Zehnder Interferometer (MZI) mesh can perform vector-matrix multiplication in photonic neural network

a programmable in-situ nonlinear activation function has not been proposed to date

suppressing further advancement of photonic neural network. Here

we demonstrate an efficient in-situ nonlinear accelerator comprising a unique solution-processed two-dimensional (2D) MoS

Opto-Resistive RAM Switch (ORS)

which exhibits tunable nonlinear resistance switching that allow us to introduce nonlinearity to the photonic neuron which overcomes the linear voltage-power relationship of typical photonic components. Our reconfigurable scheme enables implementation of a wide variety of nonlinear responses. Furthermore

we confirm its feasibility and capability for MNIST handwritten digit recognition

achieving a high accuracy of 91.6%. Our accelerator constitutes a major step towards the realization of in-situ photonic neural network and pave the way for the integration of photonic integrated circuits (PIC).

关键词

Keywords

references

O. I. Abiodun , 等 . State-of-the-art in artificial neural network applications: a survey . Heliyon , 2018 . 4 e00938 DOI:10.1016/j.heliyon.2018.e00938http://doi.org/10.1016/j.heliyon.2018.e00938 .

O. I. Abiodun , 等 . Comprehensive review of artificial neural network applications to pattern recognition . IEEE Access , 2019 . 7 158820 -158846 . DOI:10.1109/ACCESS.2019.2945545http://doi.org/10.1109/ACCESS.2019.2945545 .

Y. H. Wang , J. M. L. Ribeiro , P. Tiwary . Past–future information bottleneck for sampling molecular reaction coordinate simultaneously with thermodynamics and kinetics . Nat. Commun. , 2019 . 10 3573 DOI:10.1038/s41467-019-11405-4http://doi.org/10.1038/s41467-019-11405-4 .

J. S. Zhang , 等 . Penetrating the influence of regularizations on neural network based on information bottleneck theory . Neurocomputing , 2020 . 393 76 -82 . DOI:10.1016/j.neucom.2020.02.009http://doi.org/10.1016/j.neucom.2020.02.009 .

S. Ambrogio , 等 . Equivalent-accuracy accelerated neural-network training using analogue memory . Nature , 2018 . 558 60 -67 . DOI:10.1038/s41586-018-0180-5http://doi.org/10.1038/s41586-018-0180-5 .

P. A. Merolla , 等 . A million spiking-neuron integrated circuit with a scalable communication network and interface . Science , 2014 . 345 668 -673 . DOI:10.1126/science.1254642http://doi.org/10.1126/science.1254642 .

L. Benini , G. De Micheli . System-level power optimization: techniques and tools . ACM Trans. Des. Autom. Electron. Syst. , 2000 . 5 115 -192 . DOI:10.1145/335043.335044http://doi.org/10.1145/335043.335044 .

K. Y. Guo , 等 . [DL] A survey of FPGA-based neural network inference accelerators . ACM Trans. Reconfigurable Technol. Syst. , 2019 . 12 2 DOI:10.1145/3289185http://doi.org/10.1145/3289185 .

R. R. Schaller . Moore's law: past, present and future . IEEE Spectr. , 1997 . 34 52 -59 . DOI:10.1109/6.591665http://doi.org/10.1109/6.591665 .

R. Won . Integrating silicon photonics . Nat. Photonics , 2010 . 4 498 -499 . DOI:10.1038/nphoton.2010.189http://doi.org/10.1038/nphoton.2010.189 .

D. R. Solli , B. Jalali . Analog optical computing . Nat. Photonics , 2015 . 9 704 -706 . DOI:10.1038/nphoton.2015.208http://doi.org/10.1038/nphoton.2015.208 .

Z. F. Ying , 等 . Electronic-photonic arithmetic logic unit for high-speed computing . Nat. Commun. , 2020 . 11 2154 DOI:10.1038/s41467-020-16057-3http://doi.org/10.1038/s41467-020-16057-3 .

Y. S. Abu-Mostafa , D. Psaltis . Optical neural computers . Sci. Am. , 1987 . 256 88 -95 . DOI:10.1038/scientificamerican0387-88http://doi.org/10.1038/scientificamerican0387-88 .

S. F. Xu , J. Wang , W. W. Zou . Optical patching scheme for optical convolutional neural networks based on wavelength-division multiplexing and optical delay lines . Opt. Lett. , 2020 . 45 3689 -3692 . DOI:10.1364/OL.397344http://doi.org/10.1364/OL.397344 .

A. Mehrabian , 等 . A winograd-based integrated photonics accelerator for convolutional neural networks . IEEE J. Sel. Top. Quantum Electron. , 2020 . 26 6100312 DOI:10.1109/JSTQE.2019.2957443http://doi.org/10.1109/JSTQE.2019.2957443 .

Y. C. Shen , 等 . Deep learning with coherent nanophotonic circuits . Nat. Photonics , 2017 . 11 441 -446 . DOI:10.1038/nphoton.2017.93http://doi.org/10.1038/nphoton.2017.93 .

W. Ma , 等 . Deep learning for the design of photonic structures . Nat. Photonics , 2021 . 15 77 -90 . DOI:10.1038/s41566-020-0685-yhttp://doi.org/10.1038/s41566-020-0685-y .

X. B. Sui , 等 . A review of optical neural networks . IEEE Access , 2020 . 8 70773 -70783 . DOI:10.1109/ACCESS.2020.2987333http://doi.org/10.1109/ACCESS.2020.2987333 .

I. A. D. Williamson , 等 . Reprogrammable electro-optic nonlinear activation functions for optical neural networks . IEEE J. Sel. Top. Quantum Electron. , 2020 . 26 7700412 DOI:10.1109/JSTQE.2019.2930455http://doi.org/10.1109/JSTQE.2019.2930455 .

H. T. Peng , 等 . Neuromorphic photonic integrated circuits . IEEE J. Sel. Top. Quantum Electron. , 2018 . 24 6101715 DOI:10.1109/JSTQE.2018.2840448http://doi.org/10.1109/JSTQE.2018.2840448 .

M. A. Nahmias , 等 . A leaky integrate-and-fire laser neuron for ultrafast cognitive computing . IEEE J. Sel. Top. Quantum Electron. , 2013 . 19 1800212 DOI:10.1109/JSTQE.2013.2257700http://doi.org/10.1109/JSTQE.2013.2257700 .

W. S. Cai , A. P. Vasudev , M. L. Brongersma . Electrically controlled nonlinear generation of light with plasmonics . Science , 2011 . 333 1720 -1723 . DOI:10.1126/science.1207858http://doi.org/10.1126/science.1207858 .

B. Yuen , 等 . Universal activation function for machine learning . Sci. Rep. , 2021 . 11 18757 DOI:10.1038/s41598-021-96723-8http://doi.org/10.1038/s41598-021-96723-8 .

S. Y. Xiang , 等 . A review: photonics devices, architectures, and algorithms for optical neural computing . J. Semiconductors , 2021 . 42 023105 DOI:10.1088/1674-4926/42/2/023105http://doi.org/10.1088/1674-4926/42/2/023105 .

K. Vandoorne , 等 . Experimental demonstration of reservoir computing on a silicon photonics chip . Nat. Commun. , 2014 . 5 3541 DOI:10.1038/ncomms4541http://doi.org/10.1038/ncomms4541 .

B. J. Shastri , 等 . Photonics for artificial intelligence and neuromorphic computing . Nat. Photonics , 2021 . 15 102 -114 . DOI:10.1038/s41566-020-00754-yhttp://doi.org/10.1038/s41566-020-00754-y .

S. Zarei , M. R. Marzban , A. Khavasi . Integrated photonic neural network based on silicon metalines . Opt. Express , 2020 . 28 36668 -36684 . DOI:10.1364/OE.404386http://doi.org/10.1364/OE.404386 .

R. Amin , 等 . ITO-based electro-absorption modulator for photonic neural activation function . APL Mater. , 2019 . 7 081112 DOI:10.1063/1.5109039http://doi.org/10.1063/1.5109039 .

F. P. Sunny , 等 . A survey on silicon photonics for deep learning . ACM J. Emerg. Technol. Comput. Syst. , 2021 . 17 61 DOI:10.1145/3459009http://doi.org/10.1145/3459009 .

M. A. Talib , 等 . A systematic literature review on hardware implementation of artificial intelligence algorithms . J. Supercomputing , 2021 . 77 1897 -1938 . DOI:10.1007/s11227-020-03325-8http://doi.org/10.1007/s11227-020-03325-8 .

F. Shokraneh , M. S. Nezami , O. Liboiron-Ladouceur . Theoretical and experimental analysis of a 4 × 4 reconfigurable MZI-based linear optical processor . J. Lightwave Technol. , 2020 . 38 1258 -1267 . DOI:10.1109/JLT.2020.2966949http://doi.org/10.1109/JLT.2020.2966949 .

M. Reck , 等 . Experimental realization of any discrete unitary operator . Phys. Rev. Lett. , 1994 . 73 58 -61 . DOI:10.1103/PhysRevLett.73.58http://doi.org/10.1103/PhysRevLett.73.58 .

D. A. B. Miller . Meshing optics with applications . Nat. Photonics , 2017 . 11 403 -404 . DOI:10.1038/nphoton.2017.104http://doi.org/10.1038/nphoton.2017.104 .

C. Guo , 等 . Simultaneous determination of optical constants, thickness, and surface roughness of thin film from spectrophotometric measurements . Opt. Lett. , 2013 . 38 40 -42 . DOI:10.1364/OL.38.000040http://doi.org/10.1364/OL.38.000040 .

R. C. Luo , 等 . Van der Waals interfacial reconstruction in monolayer transition-metal dichalcogenides and gold heterojunctions . Nat. Commun. , 2020 . 11 1011 DOI:10.1038/s41467-020-14753-8http://doi.org/10.1038/s41467-020-14753-8 .

L. B. Liang , V. Meunier . First-principles Raman spectra of MoS2, WS2 and their heterostructures . Nanoscale , 2014 . 6 5394 -5401 . DOI:10.1039/c3nr06906khttp://doi.org/10.1039/c3nr06906k .

M. A. Zidan , J. P. Strachan , W. D. Lu . The future of electronics based on memristive systems . Nat. Electron. , 2018 . 1 22 -29 . DOI:10.1038/s41928-017-0006-8http://doi.org/10.1038/s41928-017-0006-8 .

C. Y. Lin , 等 . Adaptive synaptic memory via lithium ion modulation in RRAM devices . Small , 2020 . 16 2003964 DOI:10.1002/smll.202003964http://doi.org/10.1002/smll.202003964 .

N. Youngblood , 等 . Waveguide-integrated black phosphorus photodetector with high responsivity and low dark current . Nat. Photonics , 2015 . 9 247 -252 . DOI:10.1038/nphoton.2015.23http://doi.org/10.1038/nphoton.2015.23 .

F. Bonaccorso , 等 . Graphene photonics and optoelectronics . Nat. Photonics , 2010 . 4 611 -622 . DOI:10.1038/nphoton.2010.186http://doi.org/10.1038/nphoton.2010.186 .

B. S. Tang , 等 . Wafer-scale solution-processed 2D material analog resistive memory array for memory-based computing . Nat. Commun. , 2022 . 13 3037 DOI:10.1038/s41467-022-30519-whttp://doi.org/10.1038/s41467-022-30519-w .

S. S. Baik , S. Im , H. J. Choi . Work function tuning in two-dimensional MoS2 field-effect-transistors with graphene and titanium source-drain contacts . Sci. Rep. , 2017 . 7 45546 DOI:10.1038/srep45546http://doi.org/10.1038/srep45546 .

P. R. Huang , 等 . The origin of the high work function of chlorinated indium tin oxide . NPG Asia Mater. , 2013 . 5 e57 DOI:10.1038/am.2013.33http://doi.org/10.1038/am.2013.33 .

C. Y. Liu , 等 . Silicon/2D-material photodetectors: from near-infrared to mid-infrared . Light Sci. Appl. , 2021 . 10 123 DOI:10.1038/s41377-021-00551-4http://doi.org/10.1038/s41377-021-00551-4 .

Y. Zuo , 等 . All-optical neural network with nonlinear activation functions . Optica , 2019 . 6 1132 -1137 . DOI:10.1364/OPTICA.6.001132http://doi.org/10.1364/OPTICA.6.001132 .

P. Sheridan . A method to perform a fast fourier transform with primitive image transformations . IEEE Trans. Image Process. , 2007 . 16 1355 -1369 . DOI:10.1109/TIP.2007.891790http://doi.org/10.1109/TIP.2007.891790 .

H. A. Ghani , 等 . A review on sparse Fast Fourier Transform applications in image processing . Int. J. Electr. Computer Eng. , 2020 . 10 1346 -1351. .

J. X. Gu , 等 . Recent advances in convolutional neural networks . Pattern Recognit. , 2018 . 77 354 -377 . DOI:10.1016/j.patcog.2017.10.013http://doi.org/10.1016/j.patcog.2017.10.013 .

M. Schuster , K. K. Paliwal . Bidirectional recurrent neural networks . IEEE Trans. Signal Process. , 1997 . 45 2673 -2681 . DOI:10.1109/78.650093http://doi.org/10.1109/78.650093 .

P. Rodriguez , 等 . Deep pain: exploiting long short-term memory networks for facial expression classification . IEEE Trans. Cybern. , 2022 . 52 3314 -3324 . DOI:10.1109/TCYB.2017.2662199http://doi.org/10.1109/TCYB.2017.2662199 .

Z. Y. Lin , 等 . Solution-processable 2D semiconductors for high-performance large-area electronics . Nature , 2018 . 562 254 -258 . DOI:10.1038/s41586-018-0574-4http://doi.org/10.1038/s41586-018-0574-4 .

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

Light | 基于光阻RAM开关的非线性激活加速器

暂无数据