Pyramid diffractive optical networks for unidirectional image magnification and demagnification

Bijie Bai; Xilin Yang; Tianyi Gan; Jingxi Li; Deniz Mengu; Mona Jarrahi; Aydogan Ozcan

doi:10.1038/s41377-024-01543-w

Your Location：

Home >

Browse articles >

Pyramid diffractive optical networks for unidirectional image magnification and demagnification

Original Articles

- Pyramid diffractive optical networks for unidirectional image magnification and demagnification
- Light: Science & Applications Vol. 13, Issue 9, Pages: 1841-1864(2024)
- Affiliations：
  
  1.Electrical and Computer Engineering Department, University of California, Los Angeles, CA, USA
  2.Bioengineering Department, University of California, Los Angeles, CA, USA
  3.California NanoSystems Institute (CNSI), University of California, Los Angeles, CA, USA
- Author bio：
  
  Aydogan Ozcan (ozcan@ucla.edu)
- Funds：
- DOI：10.1038/s41377-024-01543-w
  CLC：
- Received：23 January 2024，
  
  Revised：18 July 2024，
  
  Accepted：2024-07-19，
  
  Published Online：31 July 2024，
  
  Published：30 September 2024
- Accepted：
Scan QR Code
Bai, B. J. et al. Pyramid diffractive optical networks for unidirectional image magnification and demagnification. Light: Science & Applications, 13, 1841-1864 (2024).
DOI：

Bai, B. J. et al. Pyramid diffractive optical networks for unidirectional image magnification and demagnification. Light: Science & Applications, 13, 1841-1864 (2024). DOI： 10.1038/s41377-024-01543-w.

摘要

Abstract

Diffractive deep neural networks (D

NNs) are composed of successive transmissive layers optimized using supervised deep learning to all-optically implement various computational tasks between an input and output field-of-view. Here

we present a pyramid-structured diffractive optical network design (which we term P-D

NN)

optimized specifically for unidirectional image magnification a

nd demagnification. In this design

the diffractive layers are pyramidally scaled in alignment with the direction of the image magnification or demagnification. This P-D

NN design creates high-fidelity magnified or demagnified images in only one direction

while inhibiting the image formation in the opposite direction—achieving the desired unidirectional imaging operation using a much smaller number of diffractive degrees of freedom within the optical processor volume. Furthermore

the P-D

NN design maintains its unidirectional image magnification/demagnification functionality across a large band of illumination wavelengths despite being trained with a single wavelength. We also designed a wavelength-multiplexed P-D

where a unidirectional magnifier and a unidirectional demagnifier operate simultaneously in opposite directions

at two distinct illumination wavelengths. Furthermore

we demonstrate that by cascading multiple unidirectional P-D

NN modules

we can achieve higher magnification factors. The efficacy of the P-D

NN architecture was also validated experimentally using terahertz illumination

successfully matching our numerical simulations. P-D

NN offers a physics-inspired strategy for designing task-specific visual processors.

关键词

Keywords

references

Zhu, S. Q. et al. Intelligent computing: the latest advances, challenges, and future. Intell. Comput. 2 , 0006. https://doi.org/10.34133/icomputing.0006 (2023)..

Mengu, D. et al. At the intersection of optics and deep learning: statistical inference, computing, and inverse design. Adv. Opt. Photonics 14 , 209–290. https://doi.org/10.1364/AOP.450345 (2022)..

Wetzstein, G. et al. Inference in artificial intelligence with deep optics and photonics. Nature 588 , 39–47. https://doi.org/10.1038/s41586-020-2973-6 (2020)..

Sitzmann, V. et al. End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging. ACM Trans. Graph. 37 , 114. https://doi.org/10.1145/3197517.3201333 (2018)..

Côté, G., Lalonde, J. F. & Thibault, S. Deep learning-enabled framework for automatic lens design starting point generation. Opt. Express 29 , 3841–3854. https://doi.org/10.1364/OE.401590 (2021)..

Wang, C. L., Chen, N. & Heidrich, W. dO: a differentiable engine for deep lens design of computational imaging systems. IEEE Trans. Comput. Imaging 8 , 905–916. https://doi. org/10.1109/TCI.2022.3212837 (2022)..

Li, Y. X. et al. Deep-learning-enabled dual-frequency composite fringe projection profilometry for single-shot absolute 3D shape measurement. Opto-Electron. Adv. 5 , 210021. https://doi.org/10.29026/oea.2022.210021 (2022)..

Carolan, J. et al. Universal linear optics. Science 349 , 711–716. https://doi.org/10.1126/science.aab3642 (2015)..

Feldmann, J. et al. Parallel convolutional processing using an integrated photonic tensor core. Nature 589 , 52–58. https://doi.org/10.1038/s41586-020-03070-1 (2021)..

Lin, X. et al. All-optical machine learning using diffractive deep neural networks. Science 361 , 1004–1008. https://doi.org/10.1126/science.aat8084 (2018)..

Mengu, D. et al. Analysis of diffractive optical neural networks and their integration with electronic neural networks. IEEE J. Sel. Top. Quantum Electron. 26 , 3700114. https://doi.org/10.1109/JSTQE.2019.2921376 (2020)..

Li, J. X. et al. Class-specific differential detection in diffractive optical neural networks improves inference accuracy. Adv. Photonics 1 , 046001. https://doi.org/10.1117/1.AP.1.4.046001 (2019)..

Rahman, M. S. S. et al. Ensemble learning of diffractive optical networks. Light Sci. Appl. 10 , 14. https://doi.org/10.1038/s41377-020-00446-w (2021)..

Li, J. X. et al. Spectrally encoded single-pixel machine vision using diffractive networks. Sci. Adv. 7 , eabd7690. https://doi.org/10.1126/sciadv.abd7690 (2021)..

Bai, B. J. et al. All-optical image classification through unknown random diffusers using a single-pixel diffractive network. Light Sci. Appl. 12 , 69. https://doi.org/10.1038/s41377-023-01116-3 (2023)..

Mengu, D. & Ozcan, A. All-optical phase recovery: diffractive computing for quantitative phase imaging. Adv. Optical Mater. 10 , 2200281. https://doi.org/10.1002/adom.202200281 (2022)..

Shen, C. Y. et al. Multispectral quantitative phase imaging using a diffractive optical network. Adv. Intell. Syst. 5 , 2300300. https://doi.org/10.1002/aisy.202300300 (2023)..

Rahman, M. S. S. et al. Universal linear intensity transformations using spati ally incoherent diffractive processors. Light Sci. Appl. 12 , 195. https://doi.org/10.1038/s41377-023-01234-y (2023)..

Li, J. X. et al. Massively parallel universal linear transformations using a wavelength-multiplexed diffractive optical network. Adv. Photonics 5 , 016003. https://doi.org/10.1117/1.AP.5.1.016003 (2023)..

Kulce, O. et al. All-optical synthesis of an arbitrary linear transformation using diffractive surfaces. Light Sci. Appl. 10 , 196. https://doi.org/10.1038/s41377-021-00623-5 (2021)..

Li, Y. et al. Universal polarization transformations: spatial programming of polarization scattering matrices using a deep learning-designed diffractive polarization transformer. Adv. Mater. 35 , 2303395. https://doi.org/10.1002/adma.202303395 (2023)..

Bai, B. J. et al. Data-class-specific all-optical transformations and encryption. Adv. Mater. 35 , 2212091. https://doi.org/10.1002/adma.202212091 (2023)..

Bai, B. J. et al. To image, or not to image: class-specific diffractive cameras with all-optical erasure of undesired objects. eLight 2 , 14. https://doi.org/10.1186/s43593-022-00021-3 (2022)..

Mengu, D. et al. Diffractive interconnects: all-optical permutation operation using diffractive networks. Nanophotonics 12 , 905–923. https://doi.org/10.1515/nanoph-2022-0358 (2023)..

Luo, Y. et al. Computational imaging without a computer: seeing through random diffusers at the speed of light. eLight 2 , 4. https://doi.org/10.1186/s43593-022-00012-4 (2022)..

Li, Y. H. et al. Quantitative phase imaging (QPI) through random diffusers using a diffractive optical network. Light Adv. Manuf. 4 , 17. https://doi.org/10.37188/lam.2023.017 (2023)..

Li, J. X. et al. Unidirectional imaging using deep learning–designed materials. Sci. Adv. 9 , eadg1505. https://doi.org/10.1126/sciadv.adg1505 (2023)..

Mengu, D. et al. Snapshot multispectral imaging using a diffractive optical network. Light Sci. Appl. 12 , 86. https://doi.org/10.1038/s41377-023-01135-0 (2023)..

Rahman, M. S. S. & Ozcan, A. Computer-free, all-optical reconstruction of holograms using diffractive networks. ACS Photonics 8 , 3375–3384. https://doi.org/10.1021/acsphotonics.1c01365 (2021)..

Huang, Z. B. et al. All-optical signal processing of vortex beams with diffractive deep neural networks. Phys. Rev. Appl. 15 , 014037. https://doi.org/10.1103/PhysRevApplied.15.014037 (2021)..

Zhu, H. H. et al. Space-efficient optical computing with an integrated chip diffractive neural network. Nat. Commun. 13 , 1044. https://doi.org/10.1038/s41467-022-28702-0 (2022)..

Goi, E., Schoenhardt, S. & Gu, M. Direct retrieval of Zernike-based pupil functions using integrated diffractive deep neural networks. Nat. Commun. 13 , 7531. https://doi.org/10.1038/s41467-022-35349-4 (2022)..

Liu, C. et al. A programmable diffractive deep neural network based on a digital-coding metasurface array. Nat. Electron. 5 , 113–122. https://doi.org/10.1038/s41928-022-00719-9 (2022)..

Luo, X. H. et al. Metasurface-enabled on-chip multiplexed diffractive neural networks in the visible. Light Sci. Appl. 11 , 158. https://doi.org/10.1038/s41377-022-00844-2 (2022)..

Benesty, J. et al. Pearson correlation coefficient. In Noise Reduction in Speech Processing (eds Cohen, I. et al.) 1-4 (Springer, 2009). https://doi.org/10.1007/978-3-642-00296-0_5 https://doi.org/10.1007/978-3-642-00296-0_5 .

Jongejan, J. et al. The Quick, Draw !— AI experiment . https://quickdraw.withgoogle.com/data https://quickdraw.withgoogle.com/data (2016)..

Zhang, S. Design and fabrication of 3D‐printed planar Fresnel zone plate lens. Electron. Lett. 52 , 833–835. https://doi.org/10.1049/el.2016.0736 (2016)..

Kuschmierz, R. et al. Ultra-thin 3D lensless fiber endoscopy using diffractive optical elements and deep neural networks. Light Adv. Manuf. 2 , 30. https://doi.org/10.37188/lam.2021.030 (2021)..

Gopakumar, M. et al. Full-colour 3D holographic augmented-reality displays with metasurface waveguides. Nature 629 , 791–797 (2024)..

Haider, T. A review of magneto-optic effects and its application. Int. J. Electromagn. Appl. 7 , 17–24 (2017)..

Bi, L. et al. On-chip optical isolation in monolithically integrated non-reciprocal optical resonators. Nat. Photonics 5 , 758–762 (2011)..

Yu, Z. F. & Fan, S. H. Complete optical isolation created by indirect interband photonic transitions. Nat. Photonics 3 , 91–94 (2009)..

Sounas, D. L. & Alù, A. Non-reciprocal photonics based on time modulation. Nat. Photonics 11 , 774–783 (2017)..

Xu, Y. & Miroshnichenko, A. E. Reconfigurable nonreciprocity with a nonlinear Fano diode. Phys. Rev. B 89 , 134306. https://doi.org/10.1103/PhysRevB.89.134306 (2014)..

Poulton, C. G. et al. Design for broadband on-chip isolator using stimulated Brillouin scattering in dispersion-engineered chalcogenide waveguides. Opt. Express 20 , 21235–21246 (2012)..

Liu, Z. et al. Rethinking the value of network pruning. In Proc of the 7th International Conference on Learning Representations (ICLR, New Orleans, 2019).

Safavian, S. R. & Landgrebe, D. A survey of decision tree classifier methodology. IEEE Trans. Syst. Man Cybern. 21 , 660–674. https://doi.org/10.1109/21.97458 (1991)..

Blumer, A. et al. Occam's razor. Inf. Process. Lett. 24 , 377–380 (1987)..

Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. Proceedings of the 3rd International Conference on Learning Representations . (ICLR, San Diego, 2015).

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰