Polarization multiplexed diffractive computing: all-optical implementation of a group of linear transformations through a polarization-encoded diffractive network

Jingxi Li; Yi-Chun Hung; Onur Kulce; Deniz Mengu; Aydogan Ozcan

doi:10.1038/s41377-022-00849-x

Go to Nature Website

Your Location：

Home >

Browse articles >

Polarization multiplexed diffractive computing: all-optical implementation of a group of linear transformations through a polarization-encoded diffractive network

Articles

- Polarization multiplexed diffractive computing: all-optical implementation of a group of linear transformations through a polarization-encoded diffractive network
- Polarization multiplexed diffractive computing: all-optical implementation of a group of linear transformations through a polarization-encoded diffractive network
- Light: Science & Applications 2022年11卷第7期页码：1423-1442
- Affiliations：
  
  1.Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095, USA
  2.Bioengineering Department, University of California, Los Angeles, CA 90095, USA
  3.California NanoSystems Institute (CNSI), University of California, Los Angeles, CA 90095, USA
- Author bio：
  
  Aydogan Ozcan (ozcan@ucla.edu)
- Funds：
- DOI：10.1038/s41377-022-00849-x
  中图分类号：
Scan for full text
Jingxi Li, Yi-Chun Hung, Onur Kulce, 等. Polarization multiplexed diffractive computing: all-optical implementation of a group of linear transformations through a polarization-encoded diffractive network[J]. Light: Science & Applications, 2022,11(7):1423-1442.

Li, J. X. et al. Polarization multiplexed diffractive computing: all-optical implementation of a group of linear transformations through a polarization-encoded diffractive network. Light: Science & Applications, 11, 1423-1442 (2022).
Jingxi Li, Yi-Chun Hung, Onur Kulce, 等. Polarization multiplexed diffractive computing: all-optical implementation of a group of linear transformations through a polarization-encoded diffractive network[J]. Light: Science & Applications, 2022,11(7):1423-1442. DOI： 10.1038/s41377-022-00849-x.

Li, J. X. et al. Polarization multiplexed diffractive computing: all-optical implementation of a group of linear transformations through a polarization-encoded diffractive network. Light: Science & Applications, 11, 1423-1442 (2022). DOI： 10.1038/s41377-022-00849-x.

摘要

Abstract

Research on optical computing has recently attracted significant attention due to the transformative advances in machine learning. Among different approaches, diffractive optical networks composed of spatially-engineered transmissive surfaces have been demonstrated for all-optical statistical inference and performing arbitrary linear transformations using passive, free-space optical layers. Here, we introduce a polarization-multiplexed diffractive processor to all-optically perform multiple, arbitrarily-selected linear transformations through a single diffractive network trained using deep learning. In this framework, an array of pre-selected linear polarizers is positioned between trainable transmissive diffractive materials that are isotropic, and different target linear transformations (complex-valued) are uniquely assigned to different combinations of input/output polarization states. The transmission layers of this polarization-multiplexed diffractive network are trained and optimized via deep learning and error-backpropagation by using thousands of examples of the input/output fields corresponding to each one of the complex-valued linear transformations assigned to different input/output polarization combinations. Our results and analysis reveal that a single diffractive network can successfully approximate and all-optically implement a group of arbitrarily-selected target transformations with a negligible error when the number of trainable diffractive features/neurons (,N,) approaches ,$$N_pN_iN_o$$, where ,N,i, and ,N,o, represent the number of pixels at the input and output fields-of-view, respectively, and ,N,p, refers to the number of unique linear transformations assigned to different input/output polarization combinations. This polarization-multiplexed all-optical diffractive processor can find various applications in optical computing and polarization-based machine vision tasks.

关键词

Keywords

references

D. R. Solli , B. Jalali . Analog optical computing . Nat. Photonics , 2015 . 9 704 -706 . DOI:10.1038/nphoton.2015.208http://doi.org/10.1038/nphoton.2015.208 .

R. Athale , D. Psaltis . Optical computing: past and future . Opt. Photonics News , 2016 . 27 32 -39 . DOI:10.1364/OPN.27.6.000032http://doi.org/10.1364/OPN.27.6.000032 .

G. Wetzstein , 等 . Inference in artificial intelligence with deep optics and photonics . Nature , 2020 . 588 39 -47 . DOI:10.1038/s41586-020-2973-6http://doi.org/10.1038/s41586-020-2973-6 .

B. J. Shastri , 等 . Photonics for artificial intelligence and neuromorphic computing . Nat. Photonics , 2021 . 15 102 -114 . DOI:10.1038/s41566-020-00754-yhttp://doi.org/10.1038/s41566-020-00754-y .

H. Zhou , 等 . Photonic matrix multiplication lights up photonic accelerator and beyond . Light Sci. Appl. , 2022 . 11 30 DOI:10.1038/s41377-022-00717-8http://doi.org/10.1038/s41377-022-00717-8 .

L. Cutrona , E. Leith , C. Palermo , L. Porcello . Optical data processing and filtering systems . IRE Trans. Inf. Theory , 1960 . 6 386 -400 . DOI:10.1109/TIT.1960.1057566http://doi.org/10.1109/TIT.1960.1057566 .

J. J. Hopfield . Neural networks and physical systems with emergent collective computational abilities . Proc. Natl Acad. Sci. , 1982 . 79 2554 -2558 . DOI:10.1073/pnas.79.8.2554http://doi.org/10.1073/pnas.79.8.2554 .

D. Psaltis , N. Farhat . Optical information processing based on an associative-memory model of neural nets with thresholding and feedback . Opt. Lett. , 1985 . 10 98 -100 . DOI:10.1364/OL.10.000098http://doi.org/10.1364/OL.10.000098 .

N. H. Farhat , D. Psaltis , A. Prata , E. Paek . Optical implementation of the Hopfield model . Appl. Opt. , 1985 . 24 1469 -1475 . DOI:10.1364/AO.24.001469http://doi.org/10.1364/AO.24.001469 .

K. Wagner , D. Psaltis . Multilayer optical learning networks . Appl. Opt. , 1987 . 26 5061 -5076 . DOI:10.1364/AO.26.005061http://doi.org/10.1364/AO.26.005061 .

D. Psaltis , D. Brady , X.-G. Gu , S. Lin . Holography in artificial neural networks . Nature , 1990 . 343 325 DOI:10.1038/343325a0http://doi.org/10.1038/343325a0 .

K. Vandoorne , J. Dambre , D. Verstraeten , B. Schrauwen , P. Bienstman . Parallel reservoir computing using optical amplifiers . IEEE Trans. Neural Netw. , 2011 . 22 1469 -1481 . DOI:10.1109/TNN.2011.2161771http://doi.org/10.1109/TNN.2011.2161771 .

A. Silva , 等 . Performing mathematical operations with metamaterials . Science , 2014 . 343 160 -163 . DOI:10.1126/science.1242818http://doi.org/10.1126/science.1242818 .

K. Vandoorne , 等 . Experimental demonstration of reservoir computing on a silicon photonics chip . Nat. Commun. , 2014 . 5 3541 DOI:10.1038/ncomms4541http://doi.org/10.1038/ncomms4541 .

J. Carolan , 等 . Universal linear optics . Science , 2015 . 349 711 -716 . DOI:10.1126/science.aab3642http://doi.org/10.1126/science.aab3642 .

Y. Shen , 等 . Deep learning with coherent nanophotonic circuits . Nat. Photonics , 2017 . 11 441 -446 . DOI:10.1038/nphoton.2017.93http://doi.org/10.1038/nphoton.2017.93 .

A. N. Tait , 等 . Neuromorphic photonic networks using silicon photonic weight banks . Sci. Rep. , 2017 . 7 7430 DOI:10.1038/s41598-017-07754-zhttp://doi.org/10.1038/s41598-017-07754-z .

J. Feldmann , N. Youngblood , C. D. Wright , H. Bhaskaran , W. H. P. Pernice . All-optical spiking neurosynaptic networks with self-learning capabilities . Nature , 2019 . 569 208 DOI:10.1038/s41586-019-1157-8http://doi.org/10.1038/s41586-019-1157-8 .

M. Miscuglio , V. J. Sorger . Photonic tensor cores for machine learning . Appl Phys. Rev. , 2020 . 7 031404 DOI:10.1063/5.0001942http://doi.org/10.1063/5.0001942 .

H. Zhang , 等 . An optical neural chip for implementing complex-valued neural network . Nat. Commun. , 2021 . 12 457 DOI:10.1038/s41467-020-20719-7http://doi.org/10.1038/s41467-020-20719-7 .

J. Feldmann , 等 . Parallel convolutional processing using an integrated photonic tensor core . Nature , 2021 . 589 52 -58 . DOI:10.1038/s41586-020-03070-1http://doi.org/10.1038/s41586-020-03070-1 .

X. Xu , 等 . 11 TOPS photonic convolutional accelerator for optical neural networks . Nature , 2021 . 589 44 -51 . DOI:10.1038/s41586-020-03063-0http://doi.org/10.1038/s41586-020-03063-0 .

X. Lin , 等 . All-optical machine learning using diffractive deep neural networks . Science , 2018 . 361 1004 -1008 . DOI:10.1126/science.aat8084http://doi.org/10.1126/science.aat8084 .

J. Bueno , 等 . Reinforcement learning in a large-scale photonic recurrent neural network . Optica , 2018 . 5 756 -760 . DOI:10.1364/OPTICA.5.000756http://doi.org/10.1364/OPTICA.5.000756 .

J. Chang , V. Sitzmann , X. Dun , W. Heidrich , G. Wetzstein . Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification . Sci Rep. , 2018 . 8 DOI:10.1038/s41598-018-30619-yhttp://doi.org/10.1038/s41598-018-30619-y .

Y. Zuo , 等 . All-optical neural network with nonlinear activation functions . Optica , 2019 . 6 1132 -1137 . DOI:10.1364/OPTICA.6.001132http://doi.org/10.1364/OPTICA.6.001132 .

N. M. Estakhri , B. Edwards , N. Engheta . Inverse-designed metastructures that solve equations . Science , 2019 . 363 1333 -1338 . DOI:10.1126/science.aaw2498http://doi.org/10.1126/science.aaw2498 .

L. G. Wright , 等 . Deep physical neural networks trained with backpropagation . Nature , 2022 . 601 549 -555 . DOI:10.1038/s41586-021-04223-6http://doi.org/10.1038/s41586-021-04223-6 .

T. W. Hughes , I. A. D. Williamson , M. Minkov , S. Fan . Wave physics as an analog recurrent neural network . Sci. Adv. , 2019 . 5 eaay6946 DOI:10.1126/sciadv.aay6946http://doi.org/10.1126/sciadv.aay6946 .

J. Dong , M. Rafayelyan , F. Krzakala , S. Gigan . Optical reservoir computing using multiple light scattering for chaotic systems prediction . IEEE J. Sel. Top. Quantum Electron , 2020 . 26 1 -12 . DOI:10.1109/JSTQE.2019.2936281http://doi.org/10.1109/JSTQE.2019.2936281 .

U. Teğin , M. Yıldırım , İ. Oğuz , C. Moser , D. Psaltis . Scalable optical learning operator . Nat. Comput Sci. , 2021 . 1 542 -549 . DOI:10.1038/s43588-021-00112-0http://doi.org/10.1038/s43588-021-00112-0 .

D. Mengu , Y. Luo , Y. Rivenson , A. Ozcan . Analysis of diffractive optical neural networks and their integration with electronic neural networks . IEEE J. Sel. Top. Quantum Electron , 2020 . 26 1 -14 . DOI:10.1109/JSTQE.2019.2921376http://doi.org/10.1109/JSTQE.2019.2921376 .

J. Li , D. Mengu , Y. Luo , Y. Rivenson , A. Ozcan . Class-specific differential detection in diffractive optical neural networks improves inference accuracy . Adv. Photonics , 2019 . 1 046001 .

T. Yan , 等 . Fourier-space diffractive deep neural network . Phys. Rev. Lett. , 2019 . 123 023901 DOI:10.1103/PhysRevLett.123.023901http://doi.org/10.1103/PhysRevLett.123.023901 .

Mengu D., Rivenson Y., Ozcan A. Scale-, shift-, and rotation-invariant diffractive optical networks. ACS Photonics. https://doi.org/10.1021/acsphotonics.0c01583https://doi.org/10.1021/acsphotonics.0c01583 (2020).

D. Mengu , 等 . Misalignment resilient diffractive optical networks . Nanophotonics , 2020 . 9 4207 -4219 . DOI:10.1515/nanoph-2020-0291http://doi.org/10.1515/nanoph-2020-0291 .

M. S. S. Rahman , J. Li , D. Mengu , Y. Rivenson , A. Ozcan . Ensemble learning of diffractive optical networks . Light Sci. Appl , 2021 . 10 14 DOI:10.1038/s41377-020-00446-whttp://doi.org/10.1038/s41377-020-00446-w .

J. Li , 等 . Spectrally encoded single-pixel machine vision using diffractive networks . Sci. Adv. , 2021 . 7 eabd7690 DOI:10.1126/sciadv.abd7690http://doi.org/10.1126/sciadv.abd7690 .

D. Mengu, M. Veli , Y. Rivenson , A. Ozcan . Classification and reconstruction of spatially overlapping phase images using diffractive optical networks . Sci. Rep. , 2022 . 12 8446 DOI:10.1038/s41598-022-12020-yhttp://doi.org/10.1038/s41598-022-12020-y.

O. Kulce , D. Mengu , Y. Rivenson , A. Ozcan . All-optical information-processing capacity of diffractive surfaces . Light Sci. Appl , 2021 . 10 25 DOI:10.1038/s41377-020-00439-9http://doi.org/10.1038/s41377-020-00439-9 .

T. Zhou , 等 . Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit . Nat. Photonics , 2021 . 15 367 -373 . DOI:10.1038/s41566-021-00796-whttp://doi.org/10.1038/s41566-021-00796-w .

H. Chen , 等 . Diffractive deep neural networks at visible wavelengths . Engineering , 2021 . 7 1483 -1491 . DOI:10.1016/j.eng.2020.07.032http://doi.org/10.1016/j.eng.2020.07.032 .

C. Liu , 等 . A programmable diffractive deep neural network based on a digital-coding metasurface array . Nat Electron , 2022 . 1 -10. .

Y. Luo , 等 . Computational imaging without a computer: seeing through random diffusers at the speed of light . eLight , 2022 . 2 4 DOI:10.1186/s43593-022-00012-4http://doi.org/10.1186/s43593-022-00012-4 .

D. Mengu , A. Ozcan . All-optical phase recovery: diffractive computing for quantitative phase imaging . Adv. Opt. Mat. , 2022 . 2200281 .

C. Qian , 等 . Performing optical logic operations by a diffractive neural network . Light Sci. Appl. , 2020 . 9 1 -7 . DOI:10.1038/s41377-020-0303-2http://doi.org/10.1038/s41377-020-0303-2 .

P. Wang , 等 . Orbital angular momentum mode logical operation using optical diffractive neural network . Photonics Res. , 2021 . 9 2116 -2124 . DOI:10.1364/PRJ.432919http://doi.org/10.1364/PRJ.432919 .

Y. Luo, D. Mengu , A. Ozcan . Cascadable all-optical NAND gates using diffractive networks . Sci. Rep. , 2022 . 12 7121 DOI:10.1038/s41598-022-11331-4http://doi.org/10.1038/s41598-022-11331-4.

Y. Luo , 等 . Design of task-specific optical systems using broadband diffractive neural networks . Light Sci. Appl. , 2019 . 8 1 -14 . DOI:10.1038/s41377-019-0223-1http://doi.org/10.1038/s41377-019-0223-1 .

M. Veli , 等 . Terahertz pulse shaping using diffractive surfaces . Nat. Commun. , 2021 . 12 37 DOI:10.1038/s41467-020-20268-zhttp://doi.org/10.1038/s41467-020-20268-z .

Z. Huang , 等 . All-optical signal processing of vortex beams with diffractive deep neural networks . Phys. Rev. Appl , 2021 . 15 014037 DOI:10.1103/PhysRevApplied.15.014037http://doi.org/10.1103/PhysRevApplied.15.014037 .

P. Wang , 等 . Diffractive deep neural network for optical orbital angular momentum multiplexing and demultiplexing . IEEE J. Sel. Top. Quantum Electron , 2022 . 28 1 -11. .

Y. Han , G. Li . Coherent optical communication using polarization multiple-input-multiple-output . Opt. Express , 2005 . 13 7527 -7534 . DOI:10.1364/OPEX.13.007527http://doi.org/10.1364/OPEX.13.007527 .

Z.-Y. Chen , 等 . Use of polarization freedom beyond polarization-division multiplexing to support high-speed and spectral-efficient data transmission . Light Sci. Appl. , 2017 . 6 e16207 -e16207 . DOI:10.1038/lsa.2016.207http://doi.org/10.1038/lsa.2016.207 .

N. Oshima , K. Hashimoto , S. Suzuki , M. Asada . Terahertz wireless data transmission with frequency and polarization division multiplexing using resonant-tunneling-diode oscillators . IEEE Trans. Terahertz Sci. Technol. , 2017 . 7 593 -598 . DOI:10.1109/TTHZ.2017.2720470http://doi.org/10.1109/TTHZ.2017.2720470 .

Kadambi, A., Taamazyan, V., Shi, B., Raskar, R. Polarized 3D: High-quality depth sensing with polarization cues. In 2015 IEEE International Conference on Computer Vision (ICCV) 3370–3378 (IEEE: Santiago, Chile, 2015).

S. Dongfeng , 等 . Polarization-multiplexing ghost imaging . Opt. Lasers Eng. , 2018 . 102 100 -105 . DOI:10.1016/j.optlaseng.2017.10.022http://doi.org/10.1016/j.optlaseng.2017.10.022 .

F. Liu , 等 . Deeply seeing through highly turbid water by active polarization imaging . Opt. Lett. , 2018 . 43 4903 -4906 . DOI:10.1364/OL.43.004903http://doi.org/10.1364/OL.43.004903 .

T. Liu , 等 . Deep learning-based holographic polarization microscopy . ACS Photonics , 2020 . 7 3023 -3034 . DOI:10.1021/acsphotonics.0c01051http://doi.org/10.1021/acsphotonics.0c01051 .

B. Bai , 等 . Pathological crystal imaging with single-shot computational polarized light microscopy . J. Biophotonics , 2020 . 13 e201960036 DOI:10.1002/jbio.201960036http://doi.org/10.1002/jbio.201960036 .

Deschaintre, V., Lin, Y., Ghosh, A. Deep polarization imaging for 3D shape and SVBRDF acquisition. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 15562–15571 (IEEE, Nashville, TN, USA, 2021).

L. B. Wolff . Polarization-based material classification from specular reflection . IEEE Trans. Pattern Anal. Mach. Intell. , 1990 . 12 1059 -1071 . DOI:10.1109/34.61705http://doi.org/10.1109/34.61705 .

Z. Zhan , 等 . Optical polarization–based seismic and water wave sensing on transoceanic cables . Science , 2021 . 371 931 -936 . DOI:10.1126/science.abe6648http://doi.org/10.1126/science.abe6648 .

A. Mecozzi , 等 . Polarization sensing using submarine optical cables . Optica , 2021 . 8 788 -795 . DOI:10.1364/OPTICA.424307http://doi.org/10.1364/OPTICA.424307 .

S. Zhou , S. Campbell , P. Yeh , H.-K. Liu . Two-stage modified signed-digit optical computing by spatial data encoding and polarization multiplexing . Appl Opt. , 1995 . 34 793 -802 . DOI:10.1364/AO.34.000793http://doi.org/10.1364/AO.34.000793 .

G. Tan , T. Zhan , Y.-H. Lee , J. Xiong , S.-T. Wu . Polarization-multiplexed multiplane display . Opt. Lett. , 2018 . 43 5651 -5654 . DOI:10.1364/OL.43.005651http://doi.org/10.1364/OL.43.005651 .

T. Zhan , 等 . Improving near-eye display resolution by polarization multiplexing . Opt. Express , 2019 . 27 15327 -15334 . DOI:10.1364/OE.27.015327http://doi.org/10.1364/OE.27.015327 .

S. G. Evangelides , L. F. Mollenauer , J. P. Gordon , N. S. Bergano . Polarization multiplexing with solitons . J. Light Technol. , 1992 . 10 28 -35 . DOI:10.1109/50.108732http://doi.org/10.1109/50.108732 .

O. Kulce , D. Mengu , Y. Rivenson , A. Ozcan . All-optical synthesis of an arbitrary linear transformation using diffractive surfaces . Light Sci. Appl , 2021 . 10 196 DOI:10.1038/s41377-021-00623-5http://doi.org/10.1038/s41377-021-00623-5 .

N. N. Kohn , R. E. Hughes , D. J. McCarty , J. S. Faires . The significance of calcium phosphate crystals in the synovial fluid of arthritic patients: the 'pseudogout syndrome'. Ⅱ. Identification of crystals . Ann. Intern Med. , 1962 . 56 738 -745 . DOI:10.7326/0003-4819-56-5-738http://doi.org/10.7326/0003-4819-56-5-738 .

C. Lawrence , J. A. Olson . Birefringent hemozoin identifies malaria . Am. J. Clin. Pathol. , 1986 . 86 360 -363 . DOI:10.1093/ajcp/86.3.360http://doi.org/10.1093/ajcp/86.3.360 .

P. Arun Gopinathan , 等 . Study of collagen birefringence in different grades of oral squamous cell carcinoma using picrosirius red and polarized light microscopy . Scientifica , 2015 . 2015 e802980 DOI:10.1155/2015/802980http://doi.org/10.1155/2015/802980 .

L.-W. Jin , 等 . Imaging linear birefringence and dichroism in cerebral amyloid pathologies . Proc. Natl Acad. Sci. USA , 2003 . 100 15294 -15298 . DOI:10.1073/pnas.2534647100http://doi.org/10.1073/pnas.2534647100 .

Y. Zhang , 等 . Wide-field imaging of birefringent synovial fluid crystals using lens-free polarized microscopy for gout diagnosis . Sci. Rep. , 2016 . 6 28793 DOI:10.1038/srep28793http://doi.org/10.1038/srep28793 .

D. R. Solli , C. F. McCormick , R. Y. Chiao , J. M. Hickmann . Photonic crystal polarizers and polarizing beam splitters . J. Appl Phys. , 2003 . 93 9429 -9431 . DOI:10.1063/1.1574174http://doi.org/10.1063/1.1574174 .

T. Liu , A. R. Zakharian , M. Fallahi , J. V. Moloney , M. Mansuripur . Design of a compact photonic-crystal-based polarizing beam splitter . IEEE Photonics Technol. Lett. , 2005 . 17 1435 -1437 . DOI:10.1109/LPT.2005.848278http://doi.org/10.1109/LPT.2005.848278 .

V. Zabelin , 等 . Self-collimating photonic crystal polarization beam splitter . Opt. Lett. , 2007 . 32 530 -532 . DOI:10.1364/OL.32.000530http://doi.org/10.1364/OL.32.000530 .

J. Hao , 等 . Optical metamaterial for polarization control . Phys. Rev. A , 2009 . 80 023807 DOI:10.1103/PhysRevA.80.023807http://doi.org/10.1103/PhysRevA.80.023807 .

L. Cong , 等 . A perfect metamaterial polarization rotator . Appl Phys. Lett. , 2013 . 103 171107 DOI:10.1063/1.4826536http://doi.org/10.1063/1.4826536 .

R. C. Jones . A new calculus for the treatment of optical systemsI. Description and discussion of the calculus . JOSA , 1941 . 31 488 -493 . DOI:10.1364/JOSA.31.000488http://doi.org/10.1364/JOSA.31.000488 .

Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. In Proc. 3rd International Conference on Learning Representations (ICLR, 2014).

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

暂无数据