Context
Example of an office acquisition with an iPad Pro (IRIT1-214).
Societal context: With the fast improvement and cost reduction of their technology, 3D acquisition devices are becoming as versatile and widespread as photographic 2D cameras. Their main advantage over 2D acquisition devices is the capture of the environment as a full 3D representation composed of millions or even billions of unorganized points. Lots of industries are thus investing in these acquisition devices for acquiring a huge variety of models or locations – e.g. the earth seen from space, forests, cities, streets, buildings, monuments, factories, manufactured objects, mobile device surrounding, etc – with an even larger variety of applications – e.g. BIM (Building Information Modeling), building and mechanical inspection, cultural heritage, communication, numerical twins, autonomous vehicles, robotics, urbanism, biological diversity analysis to name just a few. Besides these industrial developments, lidars are also proposed on recent iphones and ipads, thus generating the need for exploiting these 3D data in large public applications such as furnishing, virtual sales, sport training, body re-education, etc. 3D acquisition devices are now largely deployed and terabits of points are to be processed, however their efficient processing remains an open and challenging problem.There are thus important expectations of the research carried out on unstructured point cloud analysis by the academic community.
Scientific context: Digital 2D images have been widely used for decades, and during the years, a well identified and structured scientific community has been built around these data. The increase of 3D unstructured data acquisition is more recent and during the last two decades, 3D point cloud processing challenges have stimulated different scientific communities such as computer vision, computer graphics, remote sensing, signal processing, artificial intelligence and robotics. Each of these communities proposed different strong advances often based on the modalities they are aware of, and these modalities may significantly differ from a community to another. Nowadays, even though the efficient processing of unorganized point clouds is a very active field of research with a huge number of new publications every years, we believe that research advances on unorganized point clouds suffer from the lack of transversal structuration and communication between the different scientific communities as achieved with the image processing scientific community.
In order to explore new original and promising research directions, this project includes researchers from the IRIT, the IMT and the LAAS.
Goal
As done at the international level with the image processing scientific community, our goal is to group researchers of the IMT, the IRIT et the LAAS around the challenges of processing and analyzing unstructured data, with a first focus on 3D data. This project team would thus gather the expertise of the different project members, from 3D point cloud processing, image processing, computer vision, signal processing and machine learning, to the geometry and topology of manifolds, varifolds and metrics, with additional expertise on signal processing and robotics. Our aim is to share expertise and knowledge from these different communities in order to identify and explore new original and promising research directions.
Scientific positioning
3D acquisition devices produce data as hundreds of millions, or billions of unorganized 3D points stored in very large files. While petabytes of point clouds have been acquired, their efficient processing for an effective use in different application domains remains an open, very challenging research problem. Despite a wide range of application domains (vision and robotics, augmented reality, processes digitalization, etc), most standard point cloud processing pipelines share several of their first steps: points have to be filtered, denoised, and both low level geometrical and topological information have to be reconstructed / inferred / recognized. The lack of native topology, the content variety and the massive nature of point clouds avoid the practical use of existing heuristics or learning-based approaches, and more scalable and tractable solutions have to be proposed. Our project team will focus on these low level processing steps.
First directions of research
During 2022, we have organized six seminars IMT/IRIT (6 to 7 attendees), two IRIT/LAAS (3 to 5 attendees), three internal to IRIT (5 attendees) and a local 1-day workshop (12 attendees). We identified the following directions of research:
Geometric measures theory techniques for characterizing local structures/features (corners, edges, surfaces, etc)
Our goal is to explore the use of tools coming from Geometric Measure Theory to efficiently characterize surfaces, corners and edges in point clouds. A large set of techniques already exist to study the local features of a submanifold or a rectifiable set: measure densities, approximate tangent plane and cones, curvature information via the second fundamental form, etc. When dealing with discrete data, these tools must be adapted to extract information from a weak representation with the additional difficulty of dealing with noise. We also want to provide robust theoretical guarantees (e.g. error bounds) when performing local computations on point clouds. First attempts would explore basic approaches as edge detection with measure density estimates, tangent plane extraction with spectral decomposition of correlation matrices and more complex techniques such as curvature computation via a weak second fundamental form from a varifold point of view [BR22].
Topological signatures and topological persistence for the detection of representative structures at different scales
We will study the computation of different topological signatures at different scales. This would enable the study of the link between the topological analysis and the identification of the representative topological structures in point clouds. The central part of this axis is the proposition of topological signatures that are both efficient for characterizing features such as edges (or more complex primitives) and reconstructing their topology, while being computationally efficient and adapted to point clouds [ELZ02,EH09].
Lightweight and scalable machine learning approaches for identifying structures in unstructured data
We want to explore lightweight neural networks architectures and dedicated parameterization for the efficient processing of unstructured data [HLP22]. We want to focus on solutions and architectures that can learn and process hundreds of thousands of points in seconds while remaining scalable and energy-efficient. Interactive training and easy network specializations/adaptation without the need for large annotated datasets is also a central concern. Finally, we would like to explore the control of the latent space properties, as it may lead to more interpretable and fair machine learning models. The development of interactive learning techniques would also allow us to explore training interactions between non-expert users and machine learning systems in the context of dedicating a learning system to a specific unstructured data processing.
Other directions of investigation
We are also interested in investing other promising research directions such as the study of:
- fast neighborhood queries in massive point clouds for collision detections, descriptor computations and surface evaluation,
- the use of unstructured data geometric descriptors for characterizing features in sounds and images,
- the development of other multiscale analysis techniques such as wavelets on unstructured data [BDC19],
- the scalability of content characterization in very high dimension unstructured data.
Bibliography
[BDC19] Y. Béarzi, J. Digne and R. Chaine. Wavejets: A Local Frequency Framework for Shape Details Amplification. Computer Graphics Forum, 2018.
[EH09] H. Edelsbrunner, J. Harer. Computational Topology : An Introduction. American Mathematical Society. 2009.
[BR22] B. Buet, M. Rumpf. Mean curvature motion of point cloud varifolds. ESAIM: Mathematical Modelling and Numerical Analysis, EDP Sciences, 2022, 56 (5), pp.1773 – 1808.
[ELZ02] H. Edelsbrunner, D. Letscher, A. Zomorodian. Topological persistence and simplification. Disc. Compu. Geom., 2002.
[HLP22] C.E. Himeur, T. Lejemble, T. Pellegrini, M. Paulin, L. Barthe, N. Mellado. PCEDNet : A Lightweight Neural Network for Fast and Interactive Edge Detection in 3D Point Clouds. ACM Transactions on Graphics, 41(1), 2022.