One of the main characteristics of a laser scanner, regardless of type, is “noise.” When I discuss this with clients, I notice there can sometimes be some misconceptions. I will take up this week’s article with an explanation of how the laser scanning industry characterizes and measures noise.
There are a number of characteristics we measure when it comes to system accuracy. For this article I will review 3 of these and discuss one in a bit of detail:
- Network Accuracy (often erroneously called “absolute” accuracy) – how well does my point cloud match a reference system such as a National Geodetic Survey (NGS) monument/benchmark? We generally test this using surveyed marks on the ground that can be measured in the point cloud data. The difference between the check points and the point cloud is expressed by a metric called the Root Mean Squared Error (RMSE).
- Local (relative) Accuracy – This metric tells us how accurate the point cloud data are in measurements that are independent of an external reference. An example of this is a ruler. The ruler gives us the distance between two marks, regardless of their position in their domain. An example where you need good local accuracy but do not care about network accuracy is measuring volumetrics using a toe drawn into the data set.
- Precision – This is the repeatability of a measurement. For example, if I were to repeatedly measure the height of a door in exactly the same spot using the same device and technique, I would expect the same answer for every measurement. Of course, this does not happen – there is a spread in my answers. The bigger this spread, the lower the precision (and conversely, the higher the “noise”).
It is this third element, precision, that I will discuss in this note.
If we image a perfectly flat surface (say a concrete slab) with an unmanned aerial system (UAS) LIDAR, we would love to get a perfectly flat point cloud over this slab. In reality, we get something that looks more like the profile of Figure 1. These are data points from a DJI L1 LIDAR (which uses a Livox Viva scanner) over a very smooth parking lot. You can see that we have a “noise envelope” of around 30 cm with occasional excursions up to 40 cm or so (circled in red in the figure).
Figure 1: Hard Surface Noise (each grid is 10cm x 10cm )
How do we apply an appropriate metric to hard surface noise such that we can compare the performance of different sensors? You might think the range between the highest point and the lowest point (called “peak to peak” or PTP) would be a reasonable metric. However, if you closely examine Figure 1, you will note that the data seem to follow a dense band (clustering) with the occasional extra high or low point. Using PTP looks only at these outlying points and does not give us a good feel for the location of the bulk of the points. Secondly, you will note a gradual sloping trend in the profile that follows, of course, the drainage slope of the lot.
An approach more indicative of the true “noise” of the system is depicted in Figure 2. The idea is to:
- Select a planar region within the point cloud. This plane need not be horizontal.
- Construct the best fit (in a least squares sense) plane to these points. There are a variety of algorithms to accomplish this. We use (in our EVO software) an algorithm called Principal Component Analysis (PCA).
- Measure the distance from each point to its perpendicular intersection with the plane. This is effectively measuring how far each point deviates from the mean.
- Compute the standard deviation of these measurements
Figure 2: a formal approach to measuring precision
Since we are repeating the same measurement multiple times (distance to the plane), the expected statistical distribution will be Gaussian (a Normal distribution). This allows us to draw conclusions such as “we expect 68% of the points to be within 1 standard deviation (1 ÃÆ’) of the mean.
We have this tool built right in to our True View EVO software, supplied with every True View sensor. Since we compensate for the tilt in the plane you select, it need not be horizontal. For example, you do not need to worry if a selected road surface is horizontal, so long as it is flat. The result of executing this tool over the same area as our profile in the parking lot yields the results of Figure 3.
Figure 3: Planar statistics for parking lot area (DJI L1)
The measured standard deviation for this patch area is 6.6 cm, about what we would expect from the high noise Livox sensor employed in the DJI L1. Note that we also measure the peak to peak (PTP) excursion of the data over the test area. We term this Range. For this particular sample, the range is 51 cm.
If we apply Gaussian statistics, we expect to find 68% of the points in our sample area within a ±1 ÃÆ’ band of the mean. For our example, we expect to see 68% of the points in a band 13.2 cm wide, centered on the mean. Again, we provide these statistics in our “Planar Statistics” tool. There are 10,060 total points in our sample, 6,962 which are in the ±1 ÃÆ’ band of 13.2 cm (the column labeled “In1Sig” in Figure 3). If we do the math, this is 69.2%. Thus you can see that we are in excellent agreement with our expectation of 68% from Normal distribution theory!
The takeaway notes from this discussion are:
- Precision is measured as standard deviation (usually with respect to the mean), not peak to peak.
- You should expect to see, in data profile views, a hard surface band that is about 6 times the standard deviation (3 ÃÆ’ on each side of the center) so do not be shocked by how wide the noise looks in a profile view.
- There is no free lunch! The higher the precision (e.g. lower the noise), the more expensive the scanner. The DJI L1 is very inexpensive but it is very noisy. The RIEGL scanners are very expensive but have impressive low noise characteristics.
In a future note, I will discuss the types of projects where high precision matters and review some tools we have in EVO for dealing with noise.