A modular hierarchical array camera

Xiaoyun Yuan; Mengqi Ji; Jiamin Wu; David J. Brady; Qionghai Dai; Lu Fang

doi:10.1038/s41377-021-00485-x

Go to Nature Website

Your Location：

Home >

Browse articles >

A modular hierarchical array camera

Articles

- A modular hierarchical array camera
- Light: Science & Applications Vol. 10, Issue 4, Pages: 542-557(2021)
- Affiliations：
  
  1.Department of Electronic Engineering, Tsinghua University, Beijing 100084, China
  2.Department of Automation, Tsinghua University, Beijing 100084, China
  3.College of Optical Sciences, University of Arizona, Tucson, AZ 85721, USA
  4.Institute of Brain and Cognitive Science, Tsinghua University, Beijing 100084, China
  5.Beijing National Research Center for Information Science and Technology, Beijing 100084, China
- Author bio：
  
  Lu Fang (fanglu@tsinghua.edu.cn)
- Funds：
- DOI：10.1038/s41377-021-00485-x
  CLC：
- Published：30 April 2021，
  
  Published Online：18 February 2021，
  
  Received：10 November 2020，
  
  Revised：30 January 2021，
  
  Accepted：01 February 2021
- Accepted：
Scan QR Code
Yuan, X. Y. et al. A modular hierarchical array camera. Light: Science & Applications, 10, 542-550 (2021).
DOI：

Yuan, X. Y. et al. A modular hierarchical array camera. Light: Science & Applications, 10, 542-550 (2021). DOI： 10.1038/s41377-021-00485-x.

摘要

Abstract

Array cameras removed the optical limitations of a single camera and paved the way for high-performance imaging via the combination of micro-cameras and computation to fuse multiple aperture images. However

existing solutions use dense arrays of cameras that require laborious calibration and lack flexibility and practicality. Inspired by the cognition function principle of the human brain

we develop an unstructured array camera system that adopts a hierarchical modular design with multiscale hybrid cameras composing different modules. Intelligent computations are designed to collaboratively operate along both intra- and intermodule pathways. This system can adaptively allocate imagery resources to dramatically reduce the hardware cost and possesses unprecedented flexibility

robustness

and versatility. Large scenes of real-world data were acquired to perform human-centric studies for the assessment of human behaviours at the individual level and crowd behaviours at the population level requiring high-resolution long-term monitoring of dynamic wide-area scenes.

关键词

Keywords

references

Lohmann, A. W. Scaling laws for lens systems.Appl. Opt.28, 4996-4998 (1989)..

Cossairt, O. S., Miau, D.&Nayar, S. K. Scaling law for computational imaging using spherical optics.J. Optical Soc. Am. A28, 2540-2553 (2011)..

Brady, D. J. et al. Multiscale gigapixel photography.Nature486, 386-389 (2012)..

Fan, J. T. et al. Video-rate imaging of biological dynamics at centimetre scale and micrometre resolution.Nat. Photonics13, 809-816 (2019)..

Wilburn, B. et al. High performance imaging using large camera arrays.ACM Trans. Graph.24, 765-776 (2005)..

Zhao, Y. et al. Heterogeneous camera array for multispectral light field imaging.Opt. Express25, 14008-14022 (2017)..

Kittle, D. S. et al. A testbed for wide-field, high-resolution, gigapixel-class cameras.Rev. Sci. Instrum.84, 053107 (2013)..

Park, H. J.&Friston, K. Structural and functional brain networks: from connections to cognition.Science342, 1238411 (2013)..

Sporns, O.&Betzel, R. F. Modular brain networks.Annu. Rev. Psychol.67, 613-640 (2016)..

Bullmore, E.&Sporns, O. Complex brain networks: graph theoretical analysis of structural and functional systems.Nat. Rev. Neurosci.10, 186-198 (2009)..

Sporns, O. Network attributes for segregation and integration in the human brain.Curr. Opin. Neurobiol.23, 162-171 (2013)..

Lynn, C. W.&Bassett, D. S. The physics of brain network structure, function and control.Nat. Rev. Phys.1, 318-332 (2019)..

Strogatz, S. H. Exploring complex networks.Nature410, 268-276 (2001)..

Seshadrinathan, K.&Nestares, O. High dynamic range imaging using camera arrays. In: Lin, X., Vetro, A.&Wu, M. (eds)Proc. 2017 IEEE International Conference on Image Processing (ICIP). (IEEE, Beijing, China, 2017).http://www.2017.ieeeicip.org/Committee.htmlhttp://www.2017.ieeeicip.org/Committee.html..

Schuster, G. M. et al. Panoramic single-aperture multi-sensor light field camera.Opt. Express27, 37257-37273 (2019)..

Cossairt, O. S., Miau, D.&Nayar, S. K. Gigapixel computational imaging. In: Lensch, H., Narasimhan, S.&Testorf, M. (eds)Proc. 2011 IEEE International Conference on Computational Photography (ICCP). (IEEE, Pittsburgh, PA, 2011).http://www.cs.cmu.edu/~ICCP2011/committee.htmlhttp://www.cs.cmu.edu/~ICCP2011/committee.html..

Perazzi, F. et al. Panoramic video from unstructured camera arrays.Computer Graph. Forum34, 57-68 (2015)..

Kopf, J. et al. Capturing and viewing gigapixel images. In: Levoy, M (ed. )ACM SIGGRAPH 2007. (ACM, New York, NY, 2007).https://dl.acm.org/doi/proceedings/10.1145/1275808https://dl.acm.org/doi/proceedings/10.1145/1275808..

Wang, X. Y. et al. PANDA: a gigapixel-level human-centric video dataset. In: Boult, T., Medioni, G.&Zabih, R. (eds)Proc. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. (IEEE, Seattle, WA, 2020).http://cvpr2020.thecvf.com/index.php/organizershttp://cvpr2020.thecvf.com/index.php/organizers..

Anderson, R. et al. Jump: virtual reality video.ACM Trans. Graph.35, 198 (2016)..

Pozo, A. P. et al. An integrated 6DoF video camera and system design.ACM Trans. Graph.38, 216 (2019)..

Bain, N.&Bartolo, D. Dynamic response and hydrodynamics of polarized crowds.Science363, 46-49 (2019)..

Hall, E. T.The Hidden Dimension(Doubleday, 1966).

Zhou, C. et al. A social interaction field model accurately identifies static and dynamic social groupings.Nat. Hum. Behav.3, 847-855 (2019)..

Rio, K. W., Dachner, G. C.&Warren, W. H. Local interactions underlying collective motion in human crowds.Proc. R. Soc. B: Biol. Sci.285, 20180611 (2018)..

Helbing, D., Farkas, I.&Vicsek, T. Simulating dynamical features of escape panic.Nature407, 487-490 (2000)..

Hu, P. Y.&Ramanan, D. Finding tiny faces. In: Chairs, G., Chellappa, R., Zhang, Z.&Hoogs, A. (eds)Proc. 2017 IEEE Conference on Computer Vision and Pattern Recognition. (IEEE, Honolulu, HI, 2017).https://cvpr2017.thecvf.com/organizershttps://cvpr2017.thecvf.com/organizers..

Yuan, X. Y. et al. Multiscale gigapixel video: a cross resolution image matching and warping approach. In: Wetzstein, G., Waller, L.&Karl, C. (eds)Proc. 2017 IEEE International Conference on Computational Photography (ICCP). (IEEE, Stanford, CA, 2017).http://iccp2017.stanford.edu/index.php/people/http://iccp2017.stanford.edu/index.php/people/..

Henriques, J. F. et al. High-speed tracking with kernelized correlation filters.IEEE Trans. Pattern Anal. Mach. Intell.37, 583-596 (2015)..

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

⁰