Ultracompact meta-imagers for arbitrary all-optical convolution

Weiwei Fu; Dong Zhao; Ziqin Li; Songde Liu; Chao Tian; Kun Huang

doi:10.1038/s41377-022-00752-5

Go to Nature Website

Your Location：

Home >

Browse articles >

Ultracompact meta-imagers for arbitrary all-optical convolution

Articles

- Ultracompact meta-imagers for arbitrary all-optical convolution
- Light: Science & Applications Vol. 11, Issue 4, Pages: 595-607(2022)
- Affiliations：
  
  1.Department of Optics and Optical Engineering, University of Science and Technology of China, Hefei, Anhui, 230026, China
  2.Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, Anhui, 230088, China
  3.Department of Precision Machinery and Precision Instrumentation, University of Science and Technology of China, Hefei, Anhui, 230026, China
- Author bio：
  
  Chao Tian (ctian@ustc.edu.cn)
  Kun Huang (huangk17@ustc.edu.cn)
- Funds：
- DOI：10.1038/s41377-022-00752-5
  CLC：
- Published：30 April 2022，
  
  Published Online：18 March 2022，
  
  Received：09 September 2021，
  
  Revised：17 February 2022，
  
  Accepted：23 February 2022
- Accepted：
Scan QR Code
Fu, W. W. et al. Ultracompact meta-imagers for arbitrary all-optical convolution. Light: Science & Applications, 11, 595-607 (2022).
DOI：

Fu, W. W. et al. Ultracompact meta-imagers for arbitrary all-optical convolution. Light: Science & Applications, 11, 595-607 (2022). DOI： 10.1038/s41377-022-00752-5.

摘要

Abstract

Electronic digital convolutions could extract key features of objects for data processing and information identification in artificial intelligence

but they are time-cost and energy consumption due to the low response of electrons. Although massless photons enable high-speed and low-loss analog convolutions

two existing all-optical approaches including Fourier filtering and Green's function have either limited functionality or bulky volume

thus restricting their applications in smart systems. Here

we report all-optical convolutional computing with a metasurface-singlet or -doublet imager

considered as the third approach

where its point spread function is modified arbitrarily via a complex-amplitude meta-modulator that enables functionality-unlimited kernels. Beyond one- and two-dimensional spatial differentiation

we demonstrate real-time

parallel

and analog convolutional processing of optical and biological specimens with challenging pepper-salt denoising and edge enhancement

which significantly enrich the toolkit of all-optical computing. Such meta-imager approach bridges multi-functionality and high-integration in all-optical convolutions

meanwhile possessing good architecture compatibility with digital convolutional neural networks.

关键词

Keywords

references

Krizhevsky, A. Sutskever, I.,&Hinton, G. E. Imagenet classification with deep convolutional neural networks. inProc. 25th International Conference on Neural Information Processing Systems. (NIPS, Lake Tahoe, 2012) 1097–1105.

LeCun, Y., Bengio, Y.&Hinton, G. Deep learning.Nature521, 436–444 (2015)..

Wang, P. S. et al. O-Cnn: Octree-based convolutional neural networks for 3d shape analysis.ACM Trans. Graph.36, 72 (2017)..

Zhang, C. et al. Optimizing FPGA-based accelerator design for deep convolutional neural networks. inProc. 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. (ACMs, Monterey, 2015) 161–170.

Jouppi, N. P. et al. In-datacenter performance analysis of a tensor processing unit. inProc. 44th Annual International Symposium on Computer Architecture. (ACMs, Toronto, 2017) 1–12.

Miller, D. A. B. Attojoule optoelectronics for low-energy information processing and communications.J. Lightwave Technol.35, 346–396 (2017)..

Zangeneh-Nejad, F. et al. Analogue computing with metamaterials.Nat. Rev. Mater.6, 207–225 (2020)..

Solli, D. R.&Jalali, B. Analog optical computing.Nat. Photonics9, 704–706 (2015)..

Abdollahramezani, S., Hemmatyar, O.&Adibi, A. Meta-optics for spatial optical analog computing.Nanophotonics9, 4075–4095 (2020)..

Silva, A. et al. Performing mathematical operations with metamaterials.Science343, 160–163 (2014)..

Wang, Z. et al. On-chip wavefront shaping with dielectric metasurface.Nat. Commun.10, 3547 (2019)..

Zhou, J. et al. Optical edge detection based on high-efficiency dielectric metasurface.Proc. Natl Acad. Sci. USA116, 11137–11140 (2019)..

Zhou, J. et al. Two-dimensional optical spatial differentiation and high-contrast imaging.Natl Sci. Rev.8, nwaa176 (2021)..

Zhou, J. et al. Metasurface enabled quantum edge detection.Sci. Adv.6, eabc4385 (2020)..

Pors, A., Nielsen, M. G.&Bozhevolnyi, S. I. Analog computing using reflective plasmonic metasurfaces.Nano Lett.15, 791–797 (2015)..

Qiu, X. et al. Spiral phase contrast imaging in nonlinear optics: seeing phase objects using invisible illumination.Optica5, 208–212 (2018)..

Huo, P. et al. Photonic spin-multiplexing metasurface for switchable spiral phase contrast imaging.Nano Lett.20, 2791–2798 (2020)..

Zhu, T. et al. Plasmonic computing of spatial differentiation.Nat. Commun.8, 15391 (2017)..

Cordaro, A. et al. High-index dielectric metasurfaces performing mathematical operations.Nano Lett.19, 8418–8423 (2019)..

Guo, C. et al. Photonic crystal slab Laplace operator for image differentiation.Optica5, 251–256 (2018)..

Kwon, H. et al. Nonlocal metasurfaces for optical signal processing.Phys. Rev. Lett.121, 173004 (2018)..

Zhou, Y. et al. Flat optics for image differentiation.Nat. Photonics14, 316–323 (2020)..

Davis, T. J. et al. Metasurfaces with asymmetric optical transfer functions for optical signal processing.Phys. Rev. Lett.123, 013901 (2019)..

Kwon, H. et al. Single-shot quantitative phase gradient microscopy using a system of multifunctional metasurfaces.Nat. Photonics14, 109–114 (2020)..

Komar, A. et al. Edge detection with MIE-resonant dielectric metasurfaces.ACS Photonics8, 864–871 (2021)..

Zhu, T. et al. Topological optical differentiator.Nat. Commun.12, 680 (2021)..

Zhu, T. et al. Generalized spatial differentiation from the spin hall effect of light and its application in image processing of edge detection.Phys. Rev. Appl.11, 034043 (2019)..

Feldmann, J. et al. Parallel convolutional processing using an integrated photonic tensor core.Nature589, 52–58 (2021)..

Xu, X. et al. 11 Tops photonic convolutional accelerator for optical neural networks.Nature589, 44–51 (2021)..

Born, M.&Wolf, E.Principles of Optics: Electromagnetic Theory of Propagation, Interference and Diffraction of Light, (Cambridge University Press, 1999).

Huang, K. et al. Planar diffractive lenses: fundamentals, functionalities, and applications.Adv. Mater.30, 1704556(2018)..

Wilson, T.&Sheppard, C.Theory and Practice of Scanning Optical Microscopy, (Academic Press London, 1984).

Russ, J. C.The Image Processing Handbook, (CRC Press, 2016).

Luo, X. Subwavelength optical engineering with metasurface waves.Adv. Optical Mater.6, 1701201 (2018)..

Astilean, S. et al. High-efficiency subwavelength diffractive element patterned in a high-refractive-index material for 633nm.Opt. Lett.23, 552–554 (1998)..

Yu, N. et al. Light propagation with phase discontinuities: generalized laws of reflection and refraction.Science334, 333–337 (2011)..

Bomzon, Z., Kleiner, V.&Hasman, F. Computer-generated space-variant polarization elements with subwavelength metal stripes.Opt. Lett.26, 33–35 (2001)..

Zhang, L. et al. Advances in full control of electromagnetic waves with metasurfaces.Adv. Optical Mater.4, 818–833 (2016)..

Kuznetsov, A. I. et al. Optically resonant dielectric nanostructures.Science354, aag2472 (2016)..

Luo, X. Principles of electromagnetic waves in metasurfaces.Sci. China Phys. Mech. Astron.58, 594201 (2015)..

Huang, K. et al. Ultraviolet metasurfaces of ≈80% efficiency with antiferromagnetic resonances for optical vectorial anti-counterfeiting.Laser Photonics Rev.13, 1800289 (2019)..

Huang, K. et al. Silicon multi-meta-holograms for the broadband visible light.Laser Photonics Rev.10, 500–509 (2016)..

Brown, B. R.&Lohmann, A. W. Complex spatial filtering with binary masks.Appl. Opt.5, 967–969 (1966)..

Huang, K. et al. Design of Dpe for modulating the electric field at the out-of-focus plane in a lens system.Appl. Opt.51, 5149–5153 (2012)..

Ren, Y. X. et al. Experimental generation of Laguerre-Gaussian beam using digital micromirror device.Appl. Opt.49, 1838–1844 (2010)..

Ritsch-Marte, M. Orbital angular momentum light in microscopy.Philos. Trans. R. Soc. A Math. Phys. Eng. Sci.375, 20150437 (2017)..

Davis, J. A. et al. Image processing with the Radial Hilbert transform: theory and experiments.Opt. Lett.25, 99–101 (2000)..

Fürhapter, S. et al. Spiral phase contrast imaging in microscopy.Opt. Express13, 689–694 (2005)..

Arbabi, A. et al. Miniature optical planar camera based on a wide-angle metasurface doublet corrected for monochromatic aberrations.Nat. Commun.7, 13682 (2016)..

Chen, Y. et al. Generalized perfect optical vortices along arbitrary trajectories.J. Phys. D Appl. Phys.54, 214001 (2021)..

Davis, J. A. et al. Encoding amplitude information onto phase-only filters.Appl. Opt.38, 5004–5013 (1999)..

Goodman, J. W.Introduction to Fourier Optics(Roberts and Company Publishers, 2005).

Huang, K. et al. Spiniform phase-encoded metagratings entangling arbitrary rational-order orbital angular momentum.Light7, 17156 (2018)..

Huang, K. et al. Bio-inspired photonic masquerade with perturbative metasurfaces.ACS Nano14, 7529–7537 (2020)..

Zhang, J. et al. Polarization-enabled tunable focusing by visible-light metalenses with geometric and propagation phase.J. Opt.21, 115102 (2019)..

Balthasar Mueller, J. P. et al. Metasurface polarization optics: independent phase control of arbitrary orthogonal states of polarization.Phys. Rev. Lett.118, 113901 (2017)..

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰