Hi! I’m Yushiro Yamashita, a data scientist at HACARUS. Today, I will like to introduce the application of sparse modeling to point cloud data.
Point Cloud
I want to first start by introducing what point cloud is. Point cloud is a three-dimensional data in which the coordinate values of each point are recorded in a format such as (x, y, z). Unlike image and video data, point cloud has a unique characteristic that the data are not arranged regularly at all points on the coordinates (Fig. 1), meaning subspaces do not contain any points.
Fig. 1. Point Cloud image
Point cloud is obtained from LiDAR [1], which is a technology that measures the distance to an object by measuring the scattering of pulsed laser light (Fig. 2) [2].
Fig. 2. LiDAR image [2]
Since LiDAR has been attracting attention for autonomous robots and vehicles [3], the demand has been increasing in recent years and is expected that the market will expand to 6 Billion USD by 2024 (Fig. 3) [4]. With this trend, data processing method for point cloud is strongly desired.
Fig. 3. Expected market growth for LiDAR [4]
Point Cloud and Sparse Modeling
At HACARUS, we tried noise reduction (denoise) with the purpose of “applying sparse modeling to point cloud data processing”.
It has already been confirmed that denoise methods based on the sparse modeling are effective in the field of image and video processing (Fig. 4).
Fig. 4. Example of denoising an image using sparse modeling
Sparse modeling can be used in the same way to denoise point cloud data as well. Therefore, we experimented with this approach.
Why we denoise
Let’s take LiDAR as an example to see why denoising is necessary in the first place.
Data acquired by sensors in the real world usually contains noise. In terms of LiDAR, ambient light such as sunlight would be the main noise (Fig. 5) [5].
Fig. 5 Image of data with noise [5]
In this figure, the left image is the data with noise and the right image is the filtered data (close to the original data).
If you tried to use this data as is, for example in object detection, you can’t expect good detection results. Therefore, denoising is essential in order to utilize point cloud data.
Denoise technique and results
The method we used was based on the paper, “Cloud Dictionary: Sparse Coding and Modeling for Point Clouds” [6] published in December 2016. The main processes were “split into patches”, “principal component analysis”, “reconstruction with dictionaries”, and “combine reconstructed patches”. Sparse modeling was used in the “reconstruction with dictionaries” step (Fig. 6). For a detailed explanation, we offer an AI consulting service or training program where you can learn how to understand and manage data and how to develop the algorithm in detail. Feel free to contact us if you’re interested to learn more.
Fig. 6 Overall flow of denoising
We used a rabbit-shaped point cloud data (129,551 points on coordinate values of (x, y, z) for each point) as a ground truth (Fig. 7) [7]. We added Gaussian noise to obtain a noisy point cloud (Fig. 8). We aim to denoise this point cloud and make it closer to the ground truth.
Fig. 7 The ground truth
Fig.8 Noisy point cloud
The results after denoising on this rabbit-shaped point cloud data are shown below:
Fig. 9 Denoised point cloud
Fig. 10 Comparison
Compared to the noisy point cloud (orange dots), you can see that the noises are reduced properly (Fig. 10). We’ll compare the results with the ground truth data (Fig. 11). From the left, the figure shows the denoised, noisy, and ground truth point clouds. The results viewed from different angles are arranged vertically.
Fig. 11 16 Point cloud comparison of denoised, noisy, and ground truth results from various angles
Summary
We tried sparse modeling denoising on 3D point cloud data. As a result, we were able to confirm that sparse modeling was indeed effective for denoising point cloud data. If you are if you’re interested in learning more about how HACARUS can use its technology to help you with your AI needs, feel free to contact us using the form below.
References
[1] ‘Lidar’ Wikipedia,https://en.wikipedia.org/wiki/Lidar
[2] ‘Precise load carrying through target recognition with 3D LiDAR sensors’ SICK, https://www.sick.com/ag/en/industries/industrial-vehicles/mobile-platforms/automated-guided-vehicles/load-handling/precise-load-carrying-through-target-recognition-with-3d-lidar-sensors/c/p613537
[3] ‘Automated guided vehicle’ Wikipedia, ttps://en.wikipedia.org/wiki/Automated_guided_vehicle
[4] ‘LiDAR for Automotive and Industrial Applications 2019’ Yole Development
http://www.yole.fr/iso_upload/News/2019/PR_LIDAR_IndustryUpdate_YOLE_April2019.pdf
[5] ‘Rapid, High-Resolution Forest Structure and Terrain Mapping over Large Areas using Single Photon Lidar’ nature scientific reports, https://www.nature.com/articles/srep28277
[6] Cloud Dictionary: Sparse Coding and Modeling for Point Clouds https://arxiv.org/abs/1612.04956
[7] ‘Chapter 5: Meshes’ Visual Computing: Geometry, Graphics, and Vision
https://www2.sonycsl.co.jp/person/nielsen/visualcomputing/
