Nonlinear encoding in diffractive information processing using linear optical materials

Yuhang Li; Jingxi Li; Aydogan Ozcan

doi:10.1038/s41377-024-01529-8

Go to Nature Website

Your Location：

Home >

Browse articles >

Nonlinear encoding in diffractive information processing using linear optical materials

Original Articles

- Nonlinear encoding in diffractive information processing using linear optical materials
- Light: Science & Applications Vol. 13, Issue 8, Pages: 1675-1688(2024)
- Affiliations：
  
  1.Electrical & Computer Engineering Department, University of California, Los Angeles, CA 90095, USA
  2.Bioengineering Department, University of California, Los Angeles, CA 90095, USA
  3.California NanoSystems Institute (CNSI), University of California, Los Angeles, CA 90095, USA
- Author bio：
  
  Aydogan Ozcan (ozcan@ucla.edu)
- Funds：
- DOI：10.1038/s41377-024-01529-8
  CLC：
- Published：31 August 2024，
  
  Published Online：23 July 2024，
  
  Received：02 March 2024，
  
  Revised：11 July 2024，
  
  Accepted：12 July 2024
- Accepted：
Scan QR Code
Li, Y. H., Li, J. X. & Ozcan, A. Nonlinear encoding in diffractive information processing using linear optical materials. Light: Science & Applications, 13, 1675-1688 (2024).
DOI：

Li, Y. H., Li, J. X. & Ozcan, A. Nonlinear encoding in diffractive information processing using linear optical materials. Light: Science & Applications, 13, 1675-1688 (2024). DOI： 10.1038/s41377-024-01529-8.

摘要

Abstract

Nonlinear encoding of optical information can be achieved using various forms of data representation. Here

we analyze the performances of different nonlinear information encoding strategies that can be employed in diffractive optical processors based on linear materials and shed light on their utility and performance gaps compared to the state-of-the-art digital deep neural networks. For a comprehensive evaluation

we used different datasets to compare the statistical inference performance of simpler-to-implement nonlinear encoding strategies that involve

e.g.

phase encoding

against data repetition-based nonlinear encoding strategies. We show that data repetition within a diffractive volume (e.g.

through an optical cavity or cascaded introduction of the input data) causes the loss of the universal linear transformation capability of a diffractive optical processor. Therefore

data repetition-based diffractive blocks cannot provide optical analogs to fully connected or convolutional layers commonly employed in digital neural networks. However

they can still be effectively trained for specific inference tasks and achieve enhanced accuracy

benefiting from the nonlinear encoding of the input information. Our results also reveal that phase encoding of input information without data repetition provides a simpler nonlinear encoding strategy with comparable statistical inference accuracy to data repetition-based diffractive processors. Our analyses and conclusions would be of broad interest to explore the push-pull relationship between linear material-based diffractive optical systems and nonlinear encoding strategies in visual information processors.

关键词

Keywords

references

Khoo, I. C., Wood, M., Shih, M. Y.&Chen, P. Extremely nonlinear photosensitive liquid crystals for image sensing and sensor protection.Opt. Express4, 432–442 (1999)..

Dongol, A., Thompson, J., Schmitzer, H., Tierney, D.&Wagner, H. P. Real-time contrast-enhanced holographic imaging using phase coherent photorefractive quantum wells.Opt. Express23, 12795–12807 (2015)..

Long, H. et al. Tuning nonlinear optical absorption properties of WS2 nanosheets.Nanoscale7, 17771–17777 (2015)..

Yi, F. et al. Optomechanical enhancement of doubly resonant 2D optical nonlinearity.Nano Lett.16, 1631–1636 (2016)..

Zhao, G. et al. Saturated absorption competition microscopy.Optica4, 633 (2017)..

Zuo, Y. et al. All-optical neural network with nonlinear activation functions.Optica6, 1132 (2019)..

Chen, X. et al. Optical nonlinearity and non-reciprocal transmission of graphene integrated metasurface.Carbon173, 126–134 (2021)..

Wang, Y. et al. Direct electrical modulation of second-order optical susceptibility via phase transitions.Nat. Electron4, 725–730 (2021)..

Zhang, D. et al. Broadband nonlinear modulation of incoherent light using a transparent optoelectronic neuron array.Nat. Commun.15.1, 2433 (2024)..

Lin, X. et al. All-optical machine learning using diffractive deep neural networks.Science361, 1004–1008 (2018)..

Li, J., Mengu, D., Luo, Y., Rivenson, Y.&Ozcan, A. Class-specific differential detection in diffractive optical neural networks improves inference accuracy.Adv. Photon.1, 046001 (2019)..

Mengu, D., Luo, Y., Rivenson, Y.&Ozcan, A. Analysis of diffractive optical neural networks and their integration with electronic neural networks.IEEE J. Sel. Top. Quantum Electron.26, 1–14 (2020)..

Qian, C. et al. Performing optical logic operations by a diffractive neural network.Light Sci. Appl.9, 59 (2020)..

Mengu, D.&Ozcan, A. All-optical phase recovery: diffractive computing for quantitative phase imaging.Adv. Opt. Mater.10, 2200281 (2022)..

Mengu, D., Veli, M., Rivenson, Y.&Ozcan, A. Classification and reconstruction of spatially overlapping phase images using diffractive optical networks.Sci. Rep.12, 8446 (2022)..

Bai, B. et al. Data-class-specific all-optical transformations and encryption.Adv. Mater.35, 2212091 (2023)..

Li, Y., Luo, Y., Mengu, D., Bai, B.&Ozcan, A. Quantitative phase imaging (QPI) through random diffusers using a diffractive optical network.Light Adv. Manuf.4, 17 (2023)..

Li, J. et al. Unidirectional imaging using deep learning–designed materials.Sci. Adv.9, eadg1505 (2023)..

Shen, C. -Y., Li, J., Mengu, D.&Ozcan, A. Multispectral quantitative phase imaging using a diffractive optical network.Adv. Intell. Syst.https://doi.org/10.1002/aisy.202300300https://doi.org/10.1002/aisy.202300300(2023)..

Yildirim, M., Dinc, N. U., Oguz, I., Psaltis, D.&Moser, C. Nonlinear processing with linear optics. Preprint athttps://doi.org/10.48550/arXiv.2307.08533https://doi.org/10.48550/arXiv.2307.08533(2023).

Xia, F. et al. Deep learning with passive optical nonlinear mapping. Preprint athttp://arxiv.org/abs/2307.08558http://arxiv.org/abs/2307.08558(2023).

Deng, L. The MNIST database of handwritten digit images for machine learning research [best of the web].IEEE Signal Process. Mag.29, 141–142 (2012)..

Xiao, H., Rasul, K.&Vollgraf, R. Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. Preprint athttps://doi.org/10.48550/arXiv.1708.07747https://doi.org/10.48550/arXiv.1708.07747(2017).

Krizhevsky, A.&Hinton, G. Learning multiple layers of features from tiny images.https://www.cs.utoronto.ca/~kriz/learning-features-2009-TR.pdfhttps://www.cs.utoronto.ca/~kriz/learning-features-2009-TR.pdf(2009).

Wanjura, C. C.&Marquardt, F. Fully nonlinear neuromorphic computing with linear wave scattering.Nat. Phys.https://doi.org/10.1038/s41567-024-02534-9https://doi.org/10.1038/s41567-024-02534-9(2024)..

Kulce, O., Mengu, D., Rivenson, Y.&Ozcan, A. All-optical synthesis of an arbitrary linear transformation using diffractive surfaces.Light Sci. Appl.10, 196 (2021)..

Kulce, O., Mengu, D., Rivenson, Y.&Ozcan, A. All-optical information-processing capacity of diffractive surfaces.Light Sci. Appl.10, 25 (2021)..

Li, J., Hung, Y. -C., Kulce, O., Mengu, D.&Ozcan, A. Polarization multiplexed diffractive computing: all-optical implementation of a group of linear transformations through a polarization-encoded diffractive network.Light Sci. Appl.11, 153 (2022)..

Li, J., Bai, B., Luo, Y.&Ozcan, A. Massively parallel universal linear transformations using a wavelength-multiplexed diffractive optical network.Adv. Photon.5, 016003 (2023).

Rahman, M. S. S., Yang, X., Li, J., Bai, B.&Ozcan, A. Universal linear intensity transformations using spatially incoherent diffractive processors.Light Sci. Appl.12, 195 (2023)..

Liu, W., Sun, Y.&Ji, Q. MDAN-UNet: multi-scale and dual attention enhanced nested U-Net architecture for segmentation of optical coherence tomography images.Algorithms13, 60 (2020)..

Ge, R. et al. MD-UNET: multi-input dilated U-shape neural network for segmentation of bladder cancer.Comput. Biol. Chem.93, 107510 (2021)..

Jia, X., De Brabandere, B., Tuytelaars, T.&V. Gool, L. Dynamic filter networks.Advances in neural information processing systems29(2016)..

Zamora Esquivel, J., Cruz Vargas, A., Lopez Meyer, P.&Tickoo, O. Adaptive convolutional kernels. In 2019IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)1998–2005 (IEEE, Seoul, Korea (South), 2019).

Su, H. et al. Pixel-adaptive convolutional neural networks. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition11158–11167 (IEEE, Long Beach, CA, USA, 2019).

Zhou, J., Jampani, V., Pi, Z., Liu, Q.&Yang, M. H. Decoupled dynamic filter networks. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6647–6656 (2021)

Tian, Z., Shen, C.&Chen, H. Conditional convolutions for instance segmentation.In Computer Vision–ECCV 2020: 16th European Conference, UK, August 23–28, Proceedings, Part Ⅰ16, 282–298 (Springer International Publishing, Glasgow, UK, 2020).

Wang, J. et al. CARAFE: content-aware reassembly of features. InProceedings of the IEEE/CVF international conference on computer vision, 3007–3016 (2019).

Mengu, D. et al. Misalignment resilient diffractive optical networks.Nanophotonics9, 4207–4219 (2020)..

Momeni, A., Rahmani, B., Malléjac, M., del Hougne, P.&Fleury, R. Backpropagation-free training of deep physical neural networks.Science382, 1297–1303 (2023)..

Yan, T. et al. Fourier-space diffractive deep neural network.Phys. Rev. Lett.123, 023901 (2019)..

Fang, X. et al. Orbital angular momentum-mediated machine learning for high-accuracy mode-feature encoding.Light Sci. Appl.13, 49 (2024)..

Bai, B. et al. Information hiding cameras: optical concealment of object information into ordinary images.Sci. Adv.10, eadn9420 (2024)..

Luo, Y. et al. Design of task-specific optical systems using broadband diffractive neural networks.Light Sci. Appl.8, 112 (2019)..

Li, J. et al. Spectrally encoded single-pixel machine vision using diffractive networks.Sci. Adv.7, eabd7690 (2021)..

Bai, B. et al. All-optical image classification through unknown random diffusers using a single-pixel diffractive network.Light Sci. Appl.12, 69 (2023)..

Mengu, D., Tabassum, A., Jarrahi, M.&Ozcan, A. Snapshot multispectral imaging using a diffractive optical network.Light Sci. Appl.12, 86 (2023)..

Yang, X., Rahman, M. S. S., Bai, B., Li, J.&Ozcan, A. Complex-valued universal linear transformations and image encryption using spatially incoherent diffractive networks.Adv. Photon. Nexus3, 016010 (2024)..

Agarwala, A., Pennington, J., Dauphin, Y.&Schoenholz, S. Temperature check: theory and practice for training models with softmax-cross-entropy losses. Preprint athttps://doi.org/10.48550/arXiv.2010.07344https://doi.org/10.48550/arXiv.2010.07344(2020).

Kingma, D. P.&Ba, J. L. Adam: a method for stochastic optimization. InProc. 3rd International Conference on Learning Representations (ICLR, 2014)(ICLR, 2014).

Loshchilov, I.&Hutter, F. Decoupled weight decay regularization. InInternational Conference on Learning Representations2019 Vol. 18 (ICLR, 2019).

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰