Image processing with Optical matrix vector multipliers implemented for encoding and decoding tasks

Minjoo Kim; Yelim Kim; Won Il Park

doi:10.1038/s41377-025-01904-z

Your Location：

Home >

Browse articles >

Image processing with Optical matrix vector multipliers implemented for encoding and decoding tasks

Original Articles

- Image processing with Optical matrix vector multipliers implemented for encoding and decoding tasks
- Light: Science & Applications Vol. 14, Issue 9, Pages: 2628-2641(2025)
- Affiliations：
  
  Division of Materials Science and Engineering, Hanyang University, Seoul, Republic of Korea
- Author bio：
  
  Won Il Park (wipark@hanyang.ac.kr)
- Funds：
- DOI：10.1038/s41377-025-01904-z
  CLC：
- Received：31 December 2024，
  
  Revised：2025-05-16，
  
  Accepted：21 May 2025，
  
  Published Online：22 July 2025，
  
  Published：30 September 2025
- Accepted：
Scan QR Code
Kim, M., Kim, Y. & Park, W. I. Image processing with Optical matrix vector multipliers implemented for encoding and decoding tasks. Light: Science & Applications, 14, 2628-2641 (2025).
DOI：

Kim, M., Kim, Y. & Park, W. I. Image processing with Optical matrix vector multipliers implemented for encoding and decoding tasks. Light: Science & Applications, 14, 2628-2641 (2025). DOI： 10.1038/s41377-025-01904-z.

摘要

Abstract

This study introduces an optical neural network (ONN)-based autoencoder for efficient image processing

utilizing specialized optical matrix-vector multipliers for both encoding and decoding tasks. To address the challenges in efficient decoding

we propose a method that optimizes output processing through scalar multiplications

enhancing performance in generating higher-dimensional outputs. By employing on-system iterative tuning

we mitigate hardware imperfections and noise

progressively improving image reconstruction accuracy to near-digital quality. Furthermore

our approach supports noise reduction and optical image generation

enabling models such as denoising autoencoders

variational autoencoders

and generative adversarial networks. Our results demonstrate that ONN-based systems have the potential to surpass the energy efficiency of traditional electronic systems

enabling real-time

low-power image processing in applications such as medical imaging

autonomous vehicles

and edge computing.

关键词

Keywords

references

Nguyen, A. et al. Autonomous navigation in com plex environments with deep multimodal fusion network. In Proc. 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) IEEE, 5824–5830. (IEEE, 2020).

Capper, D. et al. DNA methylation-based classification of central nervous system tumours. Nature 555 , 469–474 (2018)..

Wu, J. M. et al. An integrated imaging sensor for aberration-corrected 3D photography. Nature 612 , 62–71 (2022)..

Obermeyer, Z. & Emanuel, E. J. Predicting the future—big data, machine learning, and clinical medicine. N. Engl. J. Med. 375 , 1216–1219 (2016)..

Wang, T. Y. et al. Image sensing with multilayer nonlinear optical neural networks. Nat. Photonics 17 , 408–415 (2023)..

Chen, Y. T. et al. All-analog photoelectronic chip for high-speed vision tasks. Nature 623 , 48–57 (2023)..

Martel, J. N. P. et al. Neural sensors: learning pixel exposures for HDR imaging and video compressive sensing with programmable sensors. IEEE Trans. pattern Anal. Mach. Intell. 42 , 1642–1653 (2020)..

Wetzstein, G. et al. Inference in artificial intelligence with deep optics and photonics. Nature 588 , 39–47 (2020)..

Gibson, G. M., Johnson, S. D. & Padgett, M. J. Single-pixel imaging 12 years on: a review. Opt. express 28 , 28190–28208 (2020)..

Goodfellow, I. et al. Deep Learning (Cambridge: MIT Press, 2016).

Hinton, G. E., Krizhevsky, A. & Wang, S. D. Transforming auto-encoders. In Proc. 21st International Conference on Artificial Neural Networks 44–51 (Springer, 2011).

Rahman, M. S. S. et al. Learning diffractive optical communication around arbitrary opaque occlusions. Nat. Commun. 14 , 6830 (2023)..

Kingma, D. P. & Welling, M. Auto-encoding variational bayes. In Proc. 2nd ed International Conference on Learning Representations (ICLR, 2014).

Pinheiro Cinelli, L. et al. Variational autoencoder. In Proc. Variational Methods for Machine Learning with Applications to Deep Networks (eds Cinelli, L. P. et al.) 1 11–149. (Cham: Springer, 2021).

Goodfellow, I. et al. Generative adversarial nets. In Proc. 28th International Conference on Neural Information Processing Systems 2672–2680 (MIT Press, 2014).

LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521 , 436–444 (2015)..

Yao, P. et al. Fully hardware-implemented memristor convolutional neural network. Nature 577 , 641–646 (2020)..

Shastri, B. J. et al. Photonics for artificial intelligence and neuromorphic computing. Nat. Photonics 15 , 102–114 (2021)..

Xu, X. Y. et al. 11 TOPS photonic convolutional accelerator for optical neural networks. Nature 589 , 44–51 (2021)..

Feldmann, J. et al. Parallel convolutional processing using an integrated photonic tensor core. Nature 589 , 52–58 (2021)..

Zhou, H. L. et al. Photonic matrix multiplication lights up photonic accelerator and beyond. Light Sci. Appl. 11 , 30 (2022)..

Zhong, C. Y. et al. Graphene/silicon heterojunction for reconfigurable phase-relevant activation function in coherent optical neural networks. Nat. Commun. 14 , 6939 (2023)..

Hazan, A. et al. MXene‐nanoflakes‐enabled all‐optical nonlinear activation function for on‐chip photonic deep neural networks. Adv. Mater. 35 , 2210216 (2023)..

Zuo, Y. et al. All-optical neural network with nonlinear activation functions. Optica 6 , 1132–1137 (2019)..

Feldmann, J. et al. All-optical spiking neurosynaptic networks with self-learning capabilities. Nature 569 , 208–214 (2019)..

Pour Fard, M. M. et al. Experimental realization of arbitrary activation functions for optical neural networks. Opt. Express 28 , 12138–12148 (2020)..

Ryou, A. et al. Free-space optical neural network based on thermal atomic nonlinearity. Photonics Res. 9 , B128–B134 (2021)..

Mennel, L. et al. Ultrafast machine vision with 2D material neural network image sensors. Nature 579 , 62–66 (2020)..

Subbulakshmi Radhakrishnan, S. et al. A biomimetic neural encoder for spiking neural network. Nat. Commun. 12 , 2143 (2021)..

Lee, D. et al. In-sensor image memorization and encoding via optical neurons for bio-stimulus domain reduction toward visual cognitive processing. Nat. Commun. 13 , 5223 (2022)..

Chen, Y. T. et al. Photonic unsupervised learning variational autoencoder for high-throughput and low-latency image transmission. Sci. Adv. 9 , eadf8437 (2023)..

Feng, P. J. et al. All-optical autoencoder machine learning framework using linear diffractive processors. Laser Photonics Rev. https://doi.org/10.1002/lpor.202401945 (2025).

Kim, M. et al. Overcoming hardware imperfections in optical neural networks through a machine learning-driven self-correction mechanism. IEEE Photonics J. 16 , 8800108 (2024)..

Bernstein, L. et al. Single-shot optical neural network. Sci. Adv. 9 , eadg7904 (2023)..

Kim, B. et al. Optical convolution operations with optical neural networks for incoherent color image recognition. Opt. Lasers Eng. 185 , 108740 (2025)..

Wang, T. Y. et al. An optical neural network using less than 1 photon per multiplication. Nat. Commun. 13 , 123 (2022)..

Caulfield, H. J. & Dolev, S. Why future supercomputing requires optics. Nat. Photonics 4 , 261–263 (2010)..

Shen, Y. C. et al. Deep learning with coherent nanophotonic circuits. Nat. photonics 11 , 441–446 (2017)..

Zhang, H. et al. An optical neural chip for implementing complex-valued neural network. Nat. Commun. 12 , 457 (2021)..

Zhu, H. H. et al. Space-efficient optical computing with an integrated chip diffractive neural network. Nat. Commun. 13 , 1044 (2022)..

Wright, L. G. et al. Deep physical neural networks trained with backpropagation. Nature 601 , 549–555 (2022)..

Rafayelyan, M. et al. Large-scale optical reservoir computing for spatiotemporal chaotic systems prediction. Phys. Rev. X 10 , 041037 (2020)..

Banko, L. et al. A. Deep learning for visualization and novelty detection in large X-ray diffraction datasets. Npj Comput. Mater. 7 , 104 (2021)..

Kumar, A., Sattigeri, P. & Fletcher, P. T. Semi-supervised learning with GANs: manifold invariance with improved inference. In Proc. 31st International Conference on Neural Information Processing Systems 5540–5550 (Curran Associates Inc., 2017).

Hamerly, R. et al. Large-scale optical neural networks based on photoelectric multiplication. Phys. Rev. X 9 , 021032 (2019)..

Lin, X. et al. All-optical machine learning using diffractive deep neural networks. Science 361 , 1004–1008 (2018)..

Song, A. et al. Low-power scalable multilayer optoelectronic neural networks enabled with incoherent light. Nat. Commun. 15 , 10692 (2024)..

De Marinis, L. et al. Photonic neural networks: a survey. IEEE Access 7 , 175827–175841 (2019)..

Horowitz, M. 1.1 Computing's energy problem (and what we can do about it). In Proc. 2014 IEEE International Solid-state Circuits Conference Digest of Technical Papers (ISSCC) 10–14 (IEEE, 2014).

Ebersole, J. F. & Wyant, J. C. Real-time optical subtraction of photographic imagery for difference detection. Appl. Optics 15 , 871–876 (1976)..

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰