A modular hierarchical array camera

Xiaoyun Yuan; Mengqi Ji; Jiamin Wu; David J. Brady; Qionghai Dai; Lu Fang

doi:10.1038/s41377-021-00485-x

Go to Nature Website

Your Location：

Home >

Browse articles >

A modular hierarchical array camera

Articles

- A modular hierarchical array camera
- A modular hierarchical array camera
- LSA 2021年10卷第4期页码：542-557
- Affiliations：
  
  1.Department of Electronic Engineering, Tsinghua University, Beijing 100084, China
  2.Department of Automation, Tsinghua University, Beijing 100084, China
  3.College of Optical Sciences, University of Arizona, Tucson, AZ 85721, USA
  4.Institute of Brain and Cognitive Science, Tsinghua University, Beijing 100084, China
  5.Beijing National Research Center for Information Science and Technology, Beijing 100084, China
- Author bio：
  
  Lu Fang (fanglu@tsinghua.edu.cn)
- Funds：
- DOI：10.1038/s41377-021-00485-x
  中图分类号：
- 纸质出版日期：2021-04-30，
  
  网络出版日期：2021-02-18，
  
  收稿日期：2020-11-10，
  
  修回日期：2021-01-30，
  
  录用日期：2021-02-01
- Accepted：
Scan QR Code
A modular hierarchical array camera[J]. LSA, 2021,10(4):542-557.

Yuan, X. Y. et al. A modular hierarchical array camera. Light: Science & Applications, 10, 542-550 (2021).
A modular hierarchical array camera[J]. LSA, 2021,10(4):542-557. DOI： 10.1038/s41377-021-00485-x.

Yuan, X. Y. et al. A modular hierarchical array camera. Light: Science & Applications, 10, 542-550 (2021). DOI： 10.1038/s41377-021-00485-x.

摘要

Abstract

Array cameras removed the optical limitations of a single camera and paved the way for high-performance imaging via the combination of micro-cameras and computation to fuse multiple aperture images. However

existing solutions use dense arrays of cameras that require laborious calibration and lack flexibility and practicality. Inspired by the cognition function principle of the human brain

we develop an unstructured array camera system that adopts a hierarchical modular design with multiscale hybrid cameras composing different modules. Intelligent computations are designed to collaboratively operate along both intra- and intermodule pathways. This system can adaptively allocate imagery resources to dramatically reduce the hardware cost and possesses unprecedented flexibility

robustness

and versatility. Large scenes of real-world data were acquired to perform human-centric studies for the assessment of human behaviours at the individual level and crowd behaviours at the population level requiring high-resolution long-term monitoring of dynamic wide-area scenes.

关键词

Keywords

references

Lohmann, A. W. Scaling laws for lens systems.Appl. Opt.28, 4996-4998 (1989)..

Cossairt, O. S., Miau, D.&Nayar, S. K. Scaling law for computational imaging using spherical optics.J. Optical Soc. Am. A28, 2540-2553 (2011)..

Brady, D. J. et al. Multiscale gigapixel photography.Nature486, 386-389 (2012)..

Fan, J. T. et al. Video-rate imaging of biological dynamics at centimetre scale and micrometre resolution.Nat. Photonics13, 809-816 (2019)..

Wilburn, B. et al. High performance imaging using large camera arrays.ACM Trans. Graph.24, 765-776 (2005)..

Zhao, Y. et al. Heterogeneous camera array for multispectral light field imaging.Opt. Express25, 14008-14022 (2017)..

Kittle, D. S. et al. A testbed for wide-field, high-resolution, gigapixel-class cameras.Rev. Sci. Instrum.84, 053107 (2013)..

Park, H. J.&Friston, K. Structural and functional brain networks: from connections to cognition.Science342, 1238411 (2013)..

Sporns, O.&Betzel, R. F. Modular brain networks.Annu. Rev. Psychol.67, 613-640 (2016)..

Bullmore, E.&Sporns, O. Complex brain networks: graph theoretical analysis of structural and functional systems.Nat. Rev. Neurosci.10, 186-198 (2009)..

Sporns, O. Network attributes for segregation and integration in the human brain.Curr. Opin. Neurobiol.23, 162-171 (2013)..

Lynn, C. W.&Bassett, D. S. The physics of brain network structure, function and control.Nat. Rev. Phys.1, 318-332 (2019)..

Strogatz, S. H. Exploring complex networks.Nature410, 268-276 (2001)..

Seshadrinathan, K.&Nestares, O. High dynamic range imaging using camera arrays. In: Lin, X., Vetro, A.&Wu, M. (eds)Proc. 2017 IEEE International Conference on Image Processing (ICIP). (IEEE, Beijing, China, 2017).http://www.2017.ieeeicip.org/Committee.htmlhttp://www.2017.ieeeicip.org/Committee.html..

Schuster, G. M. et al. Panoramic single-aperture multi-sensor light field camera.Opt. Express27, 37257-37273 (2019)..

Cossairt, O. S., Miau, D.&Nayar, S. K. Gigapixel computational imaging. In: Lensch, H., Narasimhan, S.&Testorf, M. (eds)Proc. 2011 IEEE International Conference on Computational Photography (ICCP). (IEEE, Pittsburgh, PA, 2011).http://www.cs.cmu.edu/~ICCP2011/committee.htmlhttp://www.cs.cmu.edu/~ICCP2011/committee.html..

Perazzi, F. et al. Panoramic video from unstructured camera arrays.Computer Graph. Forum34, 57-68 (2015)..

Kopf, J. et al. Capturing and viewing gigapixel images. In: Levoy, M (ed. )ACM SIGGRAPH 2007. (ACM, New York, NY, 2007).https://dl.acm.org/doi/proceedings/10.1145/1275808https://dl.acm.org/doi/proceedings/10.1145/1275808..

Wang, X. Y. et al. PANDA: a gigapixel-level human-centric video dataset. In: Boult, T., Medioni, G.&Zabih, R. (eds)Proc. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. (IEEE, Seattle, WA, 2020).http://cvpr2020.thecvf.com/index.php/organizershttp://cvpr2020.thecvf.com/index.php/organizers..

Anderson, R. et al. Jump: virtual reality video.ACM Trans. Graph.35, 198 (2016)..

Pozo, A. P. et al. An integrated 6DoF video camera and system design.ACM Trans. Graph.38, 216 (2019)..

Bain, N.&Bartolo, D. Dynamic response and hydrodynamics of polarized crowds.Science363, 46-49 (2019)..

Hall, E. T.The Hidden Dimension(Doubleday, 1966).

Zhou, C. et al. A social interaction field model accurately identifies static and dynamic social groupings.Nat. Hum. Behav.3, 847-855 (2019)..

Rio, K. W., Dachner, G. C.&Warren, W. H. Local interactions underlying collective motion in human crowds.Proc. R. Soc. B: Biol. Sci.285, 20180611 (2018)..

Helbing, D., Farkas, I.&Vicsek, T. Simulating dynamical features of escape panic.Nature407, 487-490 (2000)..

Hu, P. Y.&Ramanan, D. Finding tiny faces. In: Chairs, G., Chellappa, R., Zhang, Z.&Hoogs, A. (eds)Proc. 2017 IEEE Conference on Computer Vision and Pattern Recognition. (IEEE, Honolulu, HI, 2017).https://cvpr2017.thecvf.com/organizershttps://cvpr2017.thecvf.com/organizers..

Yuan, X. Y. et al. Multiscale gigapixel video: a cross resolution image matching and warping approach. In: Wetzstein, G., Waller, L.&Karl, C. (eds)Proc. 2017 IEEE International Conference on Computational Photography (ICCP). (IEEE, Stanford, CA, 2017).http://iccp2017.stanford.edu/index.php/people/http://iccp2017.stanford.edu/index.php/people/..

Henriques, J. F. et al. High-speed tracking with kernelized correlation filters.IEEE Trans. Pattern Anal. Mach. Intell.37, 583-596 (2015)..

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

暂无数据