Physical twinning for joint encoding-decoding optimization in computational optics: a review

Liheng Bian; Xinrui Zhan; Rong Yan; Xuyang Chang; Hua Huang; Jun Zhang

doi:10.1038/s41377-025-01810-4

Your Location：

Home >

Browse articles >

Physical twinning for joint encoding-decoding optimization in computational optics: a review

Reviews

- Physical twinning for joint encoding-decoding optimization in computational optics: a review
- Light: Science & Applications Vol. 14, Issue 6, Pages: 1484-1503(2025)
- Affiliations：
  
  1.State Key Laboratory of CNS/ATM & MIIT Key Laboratory of Complex-field Intelligent Sensing, Beijing Institute of Technology, Beijing & Zhuhai, China
  2.Yangtze Delta Region Academy of Beijing Institute of Technology (Jiaxing), Jiaxing, China
  3.School of Artificial Intelligence, Beijing Normal University, Beijing, China
- Author bio：
  
  Liheng Bian (bian@bit.edu.cn)
  Hua Huang (huahuang@bnu.edu.cn)
  Jun Zhang (zhjun@bit.edu.cn)
- Funds：
- DOI：10.1038/s41377-025-01810-4
  CLC：
- Received：03 April 2024，
  
  Revised：13 January 2025，
  
  Accepted：04 March 2025，
  
  Published Online：15 April 2025，
  
  Published：30 June 2025
- Accepted：
Scan QR Code
Bian, L. H. et al. Physical twinning for joint encoding-decoding optimization in computational optics: a review. Light: Science & Applications, 14, 1484-1503 (2025).
DOI：

Bian, L. H. et al. Physical twinning for joint encoding-decoding optimization in computational optics: a review. Light: Science & Applications, 14, 1484-1503 (2025). DOI： 10.1038/s41377-025-01810-4.

摘要

Abstract

Computational optics introduces computation into optics and consequently helps overcome traditional optical limitations such as low sensing dimension

low light throughput

low resolution

and so on. The combination of optical encoding and computational decoding offers enhanced imaging and sensing capabilities with diverse applications in biomedicine

astronomy

agriculture

etc. With the great advance of artificial intelligence in the last decade

deep learning has further boosted computational optics with higher precision and efficiency. Recently

there developed an end-to-end joint optimization technique that digitally twins optical encoding to neural network layers

and then facilitates simultaneous optimization with the decoding process. This framework offers effective performance enhancement over conventional techniques. However

the reverse physical twinning from optimized encoding parameters to practical modulation elements faces a serious challenge

due to the discrepant gap in such as bit depth

numerical range

and stability. In this regard

this review explores various optical modulation elements across spatial

phase

and spectral dimensions in the digital twin model for joint encoding-decoding optimization. Our analysis offers constructive guidance for finding the most appropriate modulation element in diverse imaging and sensing tasks concerning various requirements of precision

speed

and robustness. The review may help tackle the above twinning challenge and pave the way for next-generation computational optics.

关键词

Keywords

references

Barbastathis, G., Ozcan, A. & Situ, G. H. On the use of deep learning for computational imaging. Optica 6 , 921–943 (2019)..

Eils, R. & Athale, C. Computational imaging in cell biology. J. Cell Biol. 161 , 477–481 (2003)..

Ntziachristos, V. Going deeper than microscopy: the optical imaging frontier in biology. Nat. Methods 7 , 603–614 (2010)..

Hebden, J. C., Arridge, S. R. & Delpy, D. T. Optical imaging in medicine: Ⅰ. Experimental techniques. Phys. Med. Biol. 42 , 825–840 (1997)..

Stender, A. S. et al. Single cell optical imaging and spectroscopy. Chem. Rev. 113 , 2469–2527 (2013)..

Waibel, A. et al. Phoneme recognition using time-delay neural networks. IEEE Trans. Acoust. Speech, Signal Process. 37 , 328–339 (1989)..

Elman, J. L. Finding structure in time. Cogn. Sci. 14 , 179–211 (1990)..

Minsky, M. L. Steps towards artificial intelligence. Proc. Inst. Radio Eng. 49 , 8–30 (1961)..

McCulloch, W. S. & Pitts, W. A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5 , 115–133 (1943)..

Rosenblatt, F. The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65 , 386–408 (1958)..

Rumelhart, D. E., Hinton, G. E. & Williams, R. J. Learning representations by back-propagating errors. Nature 323 , 533–536 (1986)..

Hunt, E. B. Artificial intelligence. (Amsterdam: Academic Press, 2014).

Adler, A., Elad, M. & Zibulevsky, M. Compressed learning: A deep neural network approach. Print at https://arxiv.org/abs/1610.09615 https://arxiv.org/abs/1610.09615 (2016).

Sitzmann, V. et al. End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging. ACM Trans. Graph. 37 , 114 (2018)..

Zhou, H. Q. et al. Single-shot phase retrieval based on anisotropicmetasurface. Appl. Phys. Lett. 120 , 161702 (2022)..

Shi, L., Li, B. C. & Matusik, W. End-to-end learning of 3D phase-only holograms for holographic display. Light Sci. Appl. 11 , 247 (2022)..

Xu, Y. B. et al. A compressive hyperspectral video imaging sys tem using a single-pixel detector. Nat. Commun. 15 , 1456 (2024)..

Wetzstein, G. et al. Inference in artificial intelligence with deep optics and photonics. Nature 588 , 39–47 (2020)..

Bian, L. H. et al. Image-free multi-character recognition. Opt. Lett. 47 , 1343–1346 (2022)..

Zhang, Z. B. et al. Fast Fourier single-pixel imaging via binary illumination. Sci. Rep. 7 , 12029 (2017)..

Zhang, Z. B. et al. Hadamard single-pixel imaging versus Fourier single-pixel imaging. Opt. Express 25 , 19619–19639 (2017)..

Fu, H., Bian, L. H. & Zhang, J. Single-pixel sensing with optimal binarized modulation. Opt. Lett. 45 , 3111–3114 (2020)..

Thorlabs. SLM at https://www.thorlabs.com/newgrouppage9.cfm?objectgroup_id=10378 https://www.thorlabs.com/newgrouppage9.cfm?objectgroup_id=10378 (2019).

Turtaev, S. et al. C omparison of nematic liquid-crystal and DMD based spatial light modulation in complex photonics. Opt. Express 25 , 29874–29884 (2017)..

Edgar, M. P., Gibson, G. M. & Padgett, M. J. Principles and prospects for single-pixel imaging. Nat. Photonics 13 , 13–20 (2019)..

Huang, H. et al. 25,000 fps computational ghost imaging with ultrafast structured illumination. Electron. Mater. 3 , 93–100 (2022)..

Shen, Y. C. et al. Deep learning with coherent nanophotonic circuits. Nat. Photonics 11 , 441–446 (2017)..

Fukui, T., Nakano, Y.&Tanemura, T. On ghost imaging using multimode fiber and integrated optical phased array. Proceedings of the 24th OptoElectronics and Communications Conference (OECC) and 2019 International Conference on Photonics in Switching and Computing (PSC) 1–3 (IEEE, 2019).

Bogaerts, W. et al. Programmable photonic circuits. Nature 586 , 207–216 (2020)..

Luo, X. H. et al. Metasurface-enabled on-chip multiplexed diffractive neural networks in the visible. Light Sci. Appl. 11 , 158 (2022)..

Bernstein, L. et al. Single-shot optical neural network. Sci. Adv. 9 , eadg7904 (2023)..

Hagen, N. A. & Kudenov, M. W. Review of snapshot spectral imaging technologies. Opt. Eng. 52 , 090901 (2013)..

Lin, X. et al. Spatial-spectral encoded compressive hyperspectral imaging. ACM Trans. Graph. 33 , 233 (2014)..

Baek, S. H. et al. Single-shot hyperspectral-depth imaging with learned diffractive optics. Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. 2651–2660(IEEE, 2021).

Hua, X. et al. Ultra-compact snapshot spectral light-field imaging. Nat. Commun. 13 , 2732 (2022)..

Mias, S. & Camon, H. A review of active optical devices: Ⅰ. Amplitude modulation. J. Micromech. Microeng. 18 , 083001 (2008)..

Brady, D. J. et al. Multiscale gigapixel photography. Nature 486 , 386–389 (2012)..

Akbulut, D. et al. Focusing light through random photonic media by binary amplitude modulation. Opt. Express 19 , 4017–4029 (2011)..

Tao, X. D. et al. High-speed scanning interferometric focusing by fast measurement of binary transmission matrix for channel demixing. Opt. Express 23 , 14168–14187 (2015)..

Rabut, C. et al. Ultrafast amplitude modulation for molecular and hemodynamic ultrasound imaging. Appl. Phys. Lett. 118 , 244102 (2021)..

Widrow, B. A study of rough amplitude quantization by means of Nyquist sampling theory. IRE Trans. Circuit Theory 3 , 266–276 (1956)..

Zisselman, E., Adler, A. & Elad, M. Compressed learning for image classification: A deep neural network approach. Handb. Numer. Anal. 19 , 3–17 (2018)..

LeCun, Y. et al. Gradient-based learning applied to document recognition. Proc. IEEE 86 , 2278–2324 (1998)..

Higham, C. F. et al. Deep learning for real-time single-pix el video. Sci. Rep. 8 , 2369 (2018)..

Yang, Z. H. et al. Image-free real-time target tracking by single-pixel detection. Opt. Express 30 , 864–873 (2022)..

Deng, Y. et al. High-efficiency terahertz single-pixel imaging based on a physics-enhanced network. Opt. Express 31 , 10273–10286 (2023)..

Hornbeck, L. J. 128 × 128 deformable mirror device. IEEE Trans. Electron Devices 30 , 539–545 (1983)..

Gattinger, P. et al. Spectral-coding-based compressive single-pixel NIR spectroscopy in the sub-millisecond regime. Sensors 21 , 5563 (2021)..

Ebner, A. et al. Diffraction-limited hyperspectral mid-infrared single-pixel microscopy. Sci. Rep. 13 , 281 (2023)..

Johnson, S. D. et al. Single-pixel RIDAR with deep learning optimised sampling. In Proc. of the 2020 Conference on Lasers and Electro-Optics . 1–2 (IEEE, 2020).

Hoshi, I. et al. Single-pixel imaging using a recurrent neural network combined with convolutional layers. Opt. Express 28 , 34069–34078 (2020)..

Jiang, W. et al. Imaging high-speed moving targets with a single-pixel detector. Opt. Express 28 , 7889–7897 (2020)..

Xu, M. et al. Deep compressed super-resolution imaging with DMD alignment error correction. Photonics 10 , 581 (2023)..

Liu, H. Y. et al. Masked autoencoder for highly compressed single-pixel imaging. Opt. Lett. 48 , 4392–4395 (2023)..

Zhan, X. R. et al. Scattering-induced entropy boost for highly-compressed optical sensing and encryption. Print at https://arxiv.org/abs/2301.06084 https://arxiv.org/abs/2301.06084 (2022).

Liu, H. Y., Bian, L. H. & Zhang, J. Image-free single-pixel segmentation. Opt. Laser Technol. 157 , 108600 (2023)..

Peng, L. T. et al. Image-free single-pixel object detection. Opt. Lett. 48 , 2527–2530 (2023)..

Zhan, X. R. et al. Ultrahigh-security single-pixel semantic encryption. Opt. Lett. 47 , 6169–6172 (2022)..

Zhan, X. R. et al. Weighted sampling-adaptive single-pixel sensing. Opt. Lett. 47 , 2838–2841 (2022)..

Jiao, S. M. et al. Optical machine learning with incoherent light and a single-pixel detector. Opt. Lett. 44 , 5186–5189 (2019)..

Zhan, X. et al. Global-optimal semi-supervised learning for single-pixel image-free sensing. Opt. Lett. 49 , 682–685 (2024)..

Wan, S. B., Chang, X. Y.&Bian, L. H. Image-free classification via few-shot learning. Proceedings of SPIE 12767, Optoelectronic Imaging and Multimedia Technology X. 229-238 (SPIE, 2023).

Zhang, Z. B. et al. Lensless single-pixel imaging by using LCD: application to small-size and multi-functional scanner. Opt. Express 27 , 3731–3745 (2019)..

Bleha, Jr. W. P.&Lei, L. A. Advances in liquid crystal on silicon (LCoS) spatial light modulator technology. Proceedings of SPIE 8736, Display Technologies and Applications for Defense, Security, and Avionics Ⅶ. 47–54 (SPIE, 2013).

Janssen, P. et al. Design aspects of a scrolling color lcos display. Displays 23 , 99–108 (2002)..

Kner, P. et al. Super-resolution video microscopy of live cells by structured illumination. Nat. Methods 6 , 339–342 (2009)..

Shao, L. et al. Super-resolution 3D microscopy of live whole cells using structured illumination. Nat. Methods 8 , 1044–1046 (2011)..

Lu-Walther, H. W. et al. fastSIM: a practical implementation of fast structured illumination microscopy. Methods Appl. Fluoresc. 3 , 014001 (2015)..

Hannebelle, M. T. M. et al. Open-source microscope add-on for structured illumination microscopy. Nat. Commun. 15 , 1550 (2024)..

Demmerle, J. et al. Strategic and practical guidelines for successful structured illumination microscopy. Nat. Protoc. 12 , 988–1010 (2017)..

Andr eev, A. L. et al. Novel FLC-materials open new possibilities for FLCoS based microdisplays and video projectors. Phys. Procedia 73 , 87–94 (2015)..

Yang, D. K.&Wu, S. T. Fundamentals of Liquid Crystal Devices. 2nd edn. (Chichester: John Wiley&Sons, 2015).

Pershan, P. S. Structure of Liquid Crystal Phases (World Scientific, 1988).

Mizuno, T. & Iwata, T. Hadamard-transform fluorescence-lifetime imaging. Opt. Express 24 , 8202–8213 (2016)..

Xu, Z. H. et al. 1000 fps computational ghost imaging using LED-based structured illumination. Opt. Express 26 , 2427–2434 (2018)..

Salvador-Balaguer, E. et al. Low-cost single-pixel 3D imaging by using an LED array. Opt. Express 26 , 15623–15631 (2018)..

Hahamovich, E. et al. Single pixel imaging at megahertz switching rates via cyclic Hadamard masks. Nat. Commun. 12 , 4516 (2021)..

He, Y. H. et al. Single-pixel imaging with neutrons. Sci. Bull. 66 , 133–138 (2021)..

Goodman, J. W. Introduction to Fourier Optics. 3rd edn. (Englewood: Roberts and Company publishers, 2005).

Park, Y., Depeursinge, C. & Popescu, G. Quantitative phase imaging in biomedicine. Nat. Photonics 12 , 578–589 (2018)..

Chang, X. Y., Bian, L. H. & Zhang, J. Large-scale phase retrieval. eLight 1 , 4 (2021)..

Yang, W. J. & Yuste, R. In vivo imaging of neural activity. Nat. Methods 14 , 349–359 (2017)..

Booth, M. J. Adaptive optics in microscopy. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 365 , 2829–2843 (2007)..

Yu, H. et al. Recent advances in wavefront shaping techniques for biomedical applications. Curr. Appl. Phys. 15 , 632–641 (2015)..

Lin, X. et al. All-optical machine learning using diffractive deep neural networks. Science 361 , 1004–1008 (2018)..

Sui, X. et al. A review of optical neural networks. IEEE Access 8 , 70773–70783 (2020)..

Piechulla, P. M. et al. Antireflective Huygens' metasurface with correlated disorder made from high-index disks implemented into silicon heterojunction solar cells. ACS Photonics 8 , 3476–3485 (2021)..

Chang, N. Y. & Kuo, C. J. Algorithm based on rigorous coupled-wave analysis for diffractive optical element design. J. Opt. Soc. Am. A 18 , 2491–2501 (2001)..

Prather, D. W. & Shi, S. Formulation and application of the finite-difference time-domain method for the analysis of axially symmetric diffractive optical elements. J. Opt. Soc. Am. 16 , 1131–1142 (1999)..

Taghizadeh, M. R. et al. Design and fabrication of diffractive optical elements. Microelectron. Eng. 34 , 219–242 (1997)..

Khonina, S. N., Kazanskiy, N. L. & Butt, M. A. Exploring diffractive optical elements and their potential in free space optics and imaging-a comprehensive review. Laser Photonics Rev. 18 , 2400377 (2024)..

Li, Y. Q., Fu, Q.&Heidrich, W. Extended depth-of-field projector using learned diffractive optics. Proceedings of 2023 IEEE Conference Virtual Reality and 3D User Interfaces. 449-459 (IEEE, 2023).

Shi, L. et al. Towards real-time photorealistic 3D holography with deep neural networks. Nature 591 , 234–239 (2021)..

Bawart, M., Bernet, S. & Ritsch-Marte, M. Programmable freeform optical elements. Opt. Express 25 , 4898–4906 (2017)..

Swanson, G. J. Binary optics technology: theoretical limits on the diffraction efficiency of multilevel diffractive optical elements. (Massachusetts Institute of Technology, 1991).

Bauchert, K. A., Serati, S. A.&Furman, A. Advances in liquid crystal spatial light modulators. Proceedings of SPIE 4734, Optical Pattern Recognition XIII. 35-43 (SPIE, 2002).

Takaki, Y. & Fujii, K. Viewing-zone scanning holographic display using a mems spatial light modulator. Opt. Express 22 , 24713–24721 (2014)..

Igasaki, Y. et al. High efficiency electrically-addressable phase-only spatial light modulator. Opt. Rev. 6 , 339–344 (1999)..

Hashimoto, N., Morokawa, S.&Kitamura, K. Real-time holography using the high-resolution LCTV-SLM. Preceedings of SPIE 1461, Practical Holography V. 291-302 (SPIE, 1991).

Salter, P. S. & Booth, M. J. Adaptive optics in laser processing. Light Sci. Appl. 8 , 110 (2019)..

Li, S. S. et al. High-quality near-field beam achieved in a high-power laser based on SLM adaptive beam-shaping system. Opt. Express 23 , 681–689 (2015)..

Chen, R. Y.et al. Physics-aware machine learning and adversarial attack in complex-valued reconfigurable diffractive all-optical neural network. Laser Photonics Rev. 16 , 2200348 (2022)..

Wang, H. et al. Intelligent optoelectronic processor for orbital angular momentum spectrum measurement. PhotoniX 4 , 9 (2023)..

Pinilla, S. et al. Hybrid diffractive optics design via hardware-in-the-loop methodology for achromatic extended-depth-of-field imaging. Opt. Express 30 , 32633–32649 (2022)..

Lazarev, G. et al. LCOS spatial light modulators: trends and applications. in Optical Imaging and Metrology: Advanced Technologies (eds Osten, W.&Reingand, N. ) 1-29 (Wiley-VCH, 2012).

Bowman, R. et al. Optimisation of a low cost SLM for diffraction efficiency and ghost order suppression. Eur. Phys. J. Spec. Top. 199 , 149–158 (2011)..

Gao, Y. H., Li, R. J. & Cao, L. C. Self-referenced multiple-beam interferometric method for robust phase calibration of spatial light modulator. Opt. Express 27 , 34463–34471 (2019)..

Chen, H. T., Taylor, A. J. & Yu, N. F. A review of metasurfaces: physics and applications. Rep. Prog. Phys. 79 , 076401 (2016)..

Hu, J. et al. A review on metasurface: from principle to smart metadevices. Front. Phys. 8 , 586087 (2021)..

Zheng, G. X. et al. Metasurface holograms reaching 80% efficiency. Nat. Nanotechnol. 10 , 308–312 (2015)..

Ye, W. M. et al. Spin and wavelength multiplexed nonlinear metasurface holography. Nat. Commun. 7 , 11930 (2016)..

Li, L. L. et al. Electromagnetic reprogrammable coding-metasurface holograms. Nat. Commun. 8 , 197 (2017)..

Huang, L. L., Zhang, S. & Zentgraf, T. Metasurface holography: from fundamentals to applications. Nanophotonics 7 , 1169–1190 (2018)..

Tseng, E. et al. Neural nano-optics for high-quality thin lens imaging. Nat. Commun. 12 , 6493 (2021)..

Liu, C. et al. A programmable diffractive deep neural network based on a digital-coding metasurface array. Nat. Electron. 5 , 113–122 (2022)..

Su, V. C. et al. Advances in optical metasurfaces: fabrication and applications [Invited ] . Opt. Express 26 , 13148–13182 (2018)..

He, Q. et al. High-efficiency metasurfaces: principles, realizations, and applications. Adv. Opt. Mater. 6 , 1800415 (2018)..

Moitra, P. et al. Large-scale all-dielectric metamaterial perfect reflectors. ACS Photonics 2 , 692–698 (2015)..

Ledwig, P. & Robles, F. E. Epi-mode tomographic quantitative phase imaging in thick scattering samples. Biomed. Opt. 10 , 3605–3621 (2019)..

Georgieva, A., Belashov, A. V. & Petrov, N. V. Optimization of DMD-based independent amplitude and phase modulation by analysis of target complex wavefront. Sci. Rep. 12 , 7754 (2022)..

Goorden, S. A., Bertolotti, J. & Mosk, A. P. Superpixel-based spatial amplitude and phase modulation using a digital micromirror device. Opt. Express 22 , 17999–18009 (2014)..

Hou, H. Y. et al. Complex-amplitude single-pixel imaging using coherent structured illumination. Opt. Express 29 , 41827–41841 (2021)..

Newton, I. A letter of Mr. Isaac Newton, professor of the mathematicks in the university of Cambridge; containing his new theory about light and colors: sent by the author to the publisher from Cambridge, Febr. $$ 6.16\frac{71}{72}$$ in order to be communicated to the R. s ociety. Philos. Trans. 6 , 3075–3087 (1672)..

Vojtíšek, P., Květoň, M. & Richter, I. Effective spectral dispersion of refractive index modulation. J. Opt. 19 , 045603 (2017)..

Huang, L. Q. et al. Spectral imaging with deep learning. Light Sci. Appl. 11 , 61 (2022)..

Garini, Y., Young, I. T. & McNamara, G. Spectral imaging: Principles and applications. Cytom. Part A 69A , 735–747 (2006)..

Arguello, H. et al. Deep optical coding design in computational imaging: a data-driven framework. IEEE Signal Process. Mag. 40 , 75–88 (2023)..

Manolakis, D. & Shaw, G. Detection algorithms for hyperspectral imaging applications. IEEE Signal Process. Mag. 19 , 29–43 (2002)..

Khan, M. J. et al. Modern trends in hyperspectral image analysis: a review. IEEE Access 6 , 14118–14129 (2018)..

Li, T. et al. Revolutionary meta-ima ging: from superlens to metalens. Photonics Insights 2 , R01 (2023)..

Anzagira, L. & Fossum, E. R. Color filter array patterns for small-pixel image sensors with substantial cross talk. J. Opt. Soc. Am. A 32 , 28–34 (2015)..

Chou Chau, Y. F. et al. A multichannel color filter with the functions of optical sensor and switch. Sci. Rep. 11 , 22910 (2021)..

Wang, S. W. et al. Concept of a high-resolution miniature spectrometer using an integrated filter array. Opt. Lett. 32 , 632–634 (2007)..

Mihoubi, S. et al. Multispectral demosaicing using pseudo-panchromatic image. IEEE Trans. Comput. Imaging 3 , 982–995 (2017)..

Zhang, T., Liang, Z. Y. & Fu, Y. Joint spatial-spectral pattern optimization and hyperspectral image reconstruction. IEEE J. Sel. Top. Signal Process. 16 , 636–648 (2022)..

Zhang, W. Y. et al. Deeply learned broadband encoding stochastic hyperspectral imaging. Light Sci. Appl. 10 , 108 (2021)..

Song, H. Y. et al. Deep-learned broadband encoding stochastic filters for computational spectroscopic instruments. Adv. Theory Simul. 4 , 2000299 (2021)..

Wu, R. J. et al. Optimized multi-spectral filter arrays for spectral reconstruction. Sensors 19 , 2905 (2019)..

Arguello, H. et al. Shift-variant color-coded diffractive spectral imaging system. Optica 8 , 1424–1434 (2021)..

Chen, C. C., Gu, H. G. & Liu, S. Y. Ultra-simplified diffraction-based computational spectrometer. Light Sci. Appl. 13 , 9 (2024)..

Engelberg, J. & Levy, U. The advantages of metalenses over diffractive lenses. Nat. Commun. 11 , 1991 (2020)..

Lv, T. et al. Aperture diffraction for compact snapshot spectral imaging. Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision. 10574-10584 (IEEE, 2023).

Mengu, D. et al. Snapshot multispectral imaging using a diffractive optical network. Light Sci. Appl. 12 , 86 (2023)..

Shi, Z. et al. Compact self-adaptive coding for spectral compressive sensing. Proceeding of 2023 IEEE International Conference on Computational Photography. 1-12 (IEEE, 2023).

Li, Z. W. et al. Efficient single-pixel multispectral imaging via non-mechanical spatio-spectral modulation. Sci. Rep. 7 , 41435 (2017)..

Arce, G. R. et al. Compressive coded aperture spectral imaging: an introduction. IEEE Signal Process. Mag. 31 , 105–115 (2014)..

Wang, L. Z. et al. HyperReconNet: joint coded aperture optimization and image reconstruction for compressive hyperspectral imaging. IEEE Trans. Image Process. 28 , 2257–2270 (2019)..

Zhang, W. H. et al. Handheld snapshot multi-spectral camera at tens-of-megapixel resolution. Nat. Commun. 14 , 5043 (2023)..

McClung, A. et al. Snapshot spectral imaging with parallel metasystems. Sci. Adv. 6 , eabc7646 (2020)..

Makarenko, M. et al. Real-time hyperspectral imaging in hardware via trained metasurface encoders. In Proc. of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition . 12692-12702 (IEEE, 2022).

Zhang, Q. B. et al. End-to-end joint optimization of metasurface and image processing for compact snapshot hyperspectral imaging. Opt. Commun. 530 , 129154 (2023)..

Hardeberg, J. Y., Schmitt, F. J. M. & Brettel, H. Multispectral color image capture using a liquid crystal tunable filter. Opt. Eng. 41 , 2532–2548 (2002)..

Chang, I. C. Acousto-optic tunable filters. Opt. Eng. 20 , 206824 (1981)..

Neumann, N. et al. Tunable infrared detector with integrated micromachined Fabry-Perot filter. J. Micro/Nanolithogr. MEMS, MOEMS 7 , 021004 (2008)..

López-Álvarez, M. A., Hernández-Andrés, J. & Romero, J. Developing an optimum computer-designed multispectral system comprising a monochrome ccd camera and a liquid-crystal tunable filter. Appl. Opt. 47 , 4381–4390 (2008)..

Wang, X. et al. Compressive spectral imaging system based on liquid crystal tunable filter. Opt. Express 26 , 25226–25243 (2018)..

Gebhart, S. C., Thompson, R. C. & Mahadevan-Jansen, A. Liquid-crystal tunable filter spectral imaging for brain tumor demarcation. Appl. Opt. 46 , 1896–1910 (2007)..

Xie, Z. W. et al. Liquid-crystal tunable color filters based on aluminum metasurfaces. Opt. Express 25 , 30764–30770 (2017)..

He, Q. et al. Dynamically tunable transmissive color filters using ultra-thin phase change materials. Opt. Express 28 , 39841–39849 (2020)..

Monga, V., Li, Y. L. & Eldar, Y. C. Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing. IEEE Signal Process. Mag. 38 , 18–44 (2021)..

Zhang, J. et al. Physics-inspired compressive sensing: Beyond deep unrolling. IEEE Signal Process. Mag. 40 , 58–72 (2023)..

Phillips, D. B. et al. Adaptive foveated single-pixel imaging with dynamic supersampling. Sci. Adv. 3 , e1601782 (2017). .

Fredrikson, M., Jha, S. & Ristenpart, T. Model inversion attacks that exploit confidence information and basic countermeasures. In Proc. of the 22nd ACM SIGSAC Conference on Computer and Communications Security . 1322-1333 (ACM, 2015).

Shi, Z. J. et al. Super-resolution orbital angular momentum holography. Nat. Commun. 14 , 1869 (2023)..

Martel, J. N. P. et al. Neural sensors: Learning pixel exposures for HDR imaging and video compressive sensing with programmable sensors. IEEE Trans. Pattern Anal. Mach. Intell. 42 , 1642–1653 (2020)..

Yang, Z. Y. et al. Miniaturization of optical spectrometers. Science 371 , eabe0722 (2021)..

Ma, M. C. et al. Physics-informed and simulation-driven optimization for binary Fourier single-pixel imaging. Opt. Lett. 49 , 218–221 (2024)..

Horstmeyer, R. et al. Convolutional neural networks that teach microscopes how to image. Print at https://arxiv.org/abs/1709.07223 https://arxiv.org/abs/1709.07223 (2017).

Kellman, M. R. et al. Physics-based learned design: optimized coded-illumination for quantitative phase imaging. IEEE Trans. Comput. Imaging 5 , 344–353 (2019)..

Muthumbi, A. et al. Learned sensing: jointly optimized microscope hardware for accurate image classification. Biomed. Opt. Express 10 , 6351–6369 (2019)..

Ghorbani, F. et al. Deep neural network-based automatic metasurface design with a wide frequency range. Sci. Rep. 11 , 7102 (2021)..

Banerji, S. et al. Imaging with flat optics: metalenses or diffractive lenses? Optica 6 , 805–810 (2019)..

Fu, Y. et al. Coded hyperspectral image reconstruction using deep external and internal learning. IEEE Trans. Pattern Anal. Mach. Intell. 44 , 3404–3420 (2022)..

Meng, Z. Y., Ma, J. W. & Yuan, X. End-to-end low cost compressive spectral imaging with spatial-spectral self-attention. In Proc. of the 16th European Conference on Computer Vision . 187-204 (Springer, 2020).

Lin, X. et al. Dual-coded compressive hyperspectral imaging. Opt. Lett. 39 , 2044–2047 (2014)..

Zhang, S. P. et al. Learning tensor low-rank prior for hyperspectral image reconstruction. In Proc. of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . 12006–12015 (IEEE, 2021).

Işíl, Ç. et al. All-optical image denoising using a diffractive visual processor. Light Sci. Appl. 13 , 43 (2024)..

Nie, S. J. et al. Deeply learned filter response functions for hyperspectral reconstruction. In Proc. of the 2018 IEEE Conference on Computer Vision and Pattern Recognition . (IEEE, 2018).

Gharbi, M. et al. Deep joint demosaicking and denoising. ACM Trans. Graph. (TOG) 35 , 191 (2016)..

Wikipedia. Computational imaging (2019). at https://en.wikipedia.org/wiki/Computationalcimaging https://en.wikipedia.org/wiki/Computationalcimaging .

Sinha, A. et al. Lensless computational imaging through deep learning. Optica 4 , 1117–1125 (2017)..

Bai, B. J. et al. Pyramid diffractive optical networks for unidirectional image magnification and demagnification. Light Sci. Appl. 13 , 178 (2024)..

Huang, Y. G. et al. Liquid-crystal-on-silicon for augmented reality displays. Appl. Sci. 8 , 2366 (2018)..

Sun, M. J. et al. Lensless imaging via LED array based computational ghost imaging. Opt. Laser Technol. 180 , 111401 (2025)..

Allen, J. Application of patterned illumination using a DMD for optogenetic control of signaling. Nat. Methods 14 , 1114 (2017)..

Li, R. J. & Cao, L. C. Progress in phase calibration for liquid crystal spatial light modulators. Appl. Sci. 9 , 2012 (2019)..

Tian, P. F. et al. Fabrication, characterization and applications of flexible vertical InGaN micro-light emitting diode arrays. Opt. Express 24 , 699–707 (2016)..

Shen, C. Y. et al. All-optical phase conjugation using diffractive wavefront processing. Nat. Commun. 15 , 4989 (2024)..

Hyman, R. M. et al. Polarization-independent phase modulation using a blue-phase liquid crystal over silicon device. Appl. Opt. 15 , 6925–6929 (2024)..

Li, J. et al. Mechanisms of 2π phase control in dielectric metasurface and transmission enhancement effect. Opt. Express 27 , 23186–23196 (2019)..

Kittle, D. et al. Multiframe image estimation for coded aperture snapshot spectral imagers. Appl. Opt. 49 , 6824–6833 (2010)..

Xu, N. et al. Snapshot hyperspectral imaging based on equalization designed DOE. Opt. Express 31 , 20489–20504 (2023)..

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰