June 2019 - Nicolas Mellado

Real-time 3D reconstruction from single-photon lidar data using plug-and-play point cloud denoisers

Julián Tachella (1), Yoann Altmann (1), Nicolas Mellado (2), Aongus McCarthy (1), Rachael Tobin (1), Gerald S. Buller (1), Jean-Yves Tourneret (3), Stephen McLaughlin (1)
(1) School of Engineering and Physical Sciences, Heriot-Watt University, UK
(2) CNRS, IRIT, Université de Toulouse, France.
(3) ENSEEIHT- IRIT-TeSA, Université de Toulouse, France.

Nature Communications

DOI
(Open access)

Data and code

Single-photon lidar has emerged as a prime candidate technology for depth imaging through challenging environments. Until now, a major limitation has been the significant amount of time required for the analysis of the recorded data. Here we show a new computational framework for real-time three-dimensional (3D) scene reconstruction from single-photon data. By combining statistical models with highly scalable computational tools from the computer graphics community, we demonstrate 3D reconstruction of complex outdoor scenes with processing times of the order of 20 ms, where the lidar data was acquired in broad daylight from distances up to 320 metres. The proposed method can handle an unknown number of surfaces in each pixel, allowing for target detection and imaging through cluttered scenes. This enables robust, real-time target reconstruction of complex moving scenes, paving the way for single-photon lidar at video rates for practical 3D imaging applications.

Illustration of a single-photon lidar dataset. The dataset consists of a man behind a camouflage net¹⁵. The graph on the left shows the histogram of a given pixel with two surfaces. The limited number of collected photons and the high background level makes the reconstruction task very challenging. In this case, processing the pixels independently yields poor results, but they can be improved by considering a priori knowledge about the scene’s structure.

Bibtex

@article{tachella:hal-02306826,
  TITLE = {{Real-time 3D reconstruction from single-photon lidar data using plug-and-play point cloud denoisers}},
  AUTHOR = {Tachella, Julian and Altmann, Yoann and Mellado, Nicolas and Mccarthy, Aongus and Tobin, Rachel and Stuart Buller, Gerald and Tourneret, Jean-Yves and Mclaughlin, Stephen},
  JOURNAL = {{Nature Communications}},
  PUBLISHER = {{Nature Publishing Group}},
  VOLUME = {10},
  PAGES = {4984},
  YEAR = {2019},
  DOI = {10.1038/s41467-019-12943-7},
}