Accuracy of LiDAR Deliverables a Complex Concept

by Mike Tully
4 days ago

Tags

A client told me the other day that some vendors are promising vertical accuracy of LiDAR deliverables that exceed the LiDAR manufacturer specifications for that system and asked: Is it legitimate for a service provider to promise better accuracy than manufacturer specifications? Or, are service providers overselling their LiDAR services by promising high vertical accuracies?

The short answer is “Yes,” it is possible to exceed these specifications, and “No,” service providers may not be overselling their services by promising such accuracies. However, the answer is easily over-simplified and it is not a simple “yes.” Further, even though it is possible to exceed manufacturer specifications for a system, it does not follow that it is common. So let me try to put a little context on this complex concept.

The first problem one runs into when reading LiDAR manufacturer “accuracy specifications” is that they are normally reporting the "relative accuracies” not “absolute

Raw LiDAR data after it has been properly aligned and IMU inaccuracies have been removed.

accuracies” of their systems. Second, these accuracy specifications are usually quite conservative. For example, Aerial Services uses the Riegl VQ-480 Corridor Mapping LiDAR System. Riegl’s spec sheet for the system says that the accuracy for the system is 2.5 cm (1 sigma at 150m above ground). But this is 2.5 cm “relative accuracy” not an “absolute accuracy”. Relative accuracies will generally always be better (more accurate) than absolute accuracies for a given acquisition for reasons discussed below. Further, project deliverables are usually delivered with “absolute” vertical accuracies that meet certain ASPRS or NSSDA thresholds. If the specification was written using NSSDA guidelines it would be expressed at 2 sigmas or “5.0 cm with a 95% confidence level.”

Further, this Riegl specification is expressed for a point cloud acquired at an unusually low altitude (150 meters above ground). Most projects are not flown this low and the absolute & relative accuracies degrade as the system is flown higher. How much less? This is not published with their specifications but it does degrade as altitude increases.

The final accuracy of a point cloud (or the bare earth elevation model derived from the point cloud) is a product of the LiDAR “system” and a “production process,” and not simply the ranging from the laser. A modern LiDAR system has many other components (like a GPS sensor, inertial system (IMU), mirror, aircraft, etc.) that when integrated, calibrated, and operated together all contribute error to the ultimate accuracy of the point cloud. So to get to the bottom of the question of error we have to approach it from a systems and operational level and not only from an overly simplistic “laser” perspective.

As it turns out, the accuracy of the laser ranging is actually one of the most accurate components in the whole system. The International Earth Rotation and Reference Systems Services (IERS) use laser ranging and achieve less than 1 cm accuracy when measuring the distance to geodetic satellites in orbit around the earth over 3500 miles away. (Manual of Airborne Topographic LiDAR, Michael S. Renslow, 2012, p. 246.) Although airborne LiDAR systems may not be this accurate, as it turns out the error contribution from the laser ranging is by far one of the least significant sources of overall error in terrestrial or airborne applications. Other sources of error like those introduced by the GNSS or inertial systems are far more important and are often the largest contributors to vertical error.

But “accuracy” only starts with the LiDAR system. Once the aircraft is on the ground, the point cloud must be processed. The processing of the point cloud can also contribute significantly to the achievable accuracy. For example, transformations from WGS to a local projection can be fraught with errors in the hands of a novice. The distance of the GNSS base station(s) to the LiDAR system during acquisition has a major effect on final accuracy. How well the system is calibrated plays an important role. Whether the data is aligned after flight to minimize IMU errors contributes to total error. The accuracy of the surveyed control points is important, but also, how the point cloud is processed using those control points. All of these factors (and others) in total contribute to the actual measurable error in the final deliverable.

On top of these factors, we have the practical consideration as to how (or if) the vertical accuracy of the point cloud will be assessed. ASPRS specifications and NSSDA guidelines state that base mapping accuracy must be assessed using at least 20 surveyed ground control that are at least 3 times more accurate than the mapping being tested. This can be quite difficult and expensive. For these reasons absolute accuracy is commonly not statistically estimated in this manner. In addition, only those portions of the point cloud classified as “bare earth” are required to meet these higher accuracy specifications (“fundamental accuracy”). Those portions of the cloud obscured by vegetation and other objects are required to meet less stringent accuracies (“supplemental accuracy”). Many projects may have only a small portion of the project area modeled by “bare earth” classifications. For these reasons there is no way to really know how accurate the deliverable is unless resources are spent that are necessary to statistically measure (estimate) its accuracy in the different point classifications. Fortunately, there are now commercial applications that enable sound statistical sampling and objective error assessment of point clouds (Spatial Information Solutions TopoAnalyst) and do not rely on outdated manual or visual-only assessment methods.

In conversations with experts at Riegl, I learned two important things. First, the published specifications are actually quite conservative. Second, they have LiDAR sensor operators that do deliver bare earth point clouds with vertical accuracies that exceed the printed specifications. So it has been established in practice that it is possible. However, because the operation of these systems and the processing of LiDAR data is fraught with error in unskilled hands, it is far easier and, unfortunately, more common that LiDAR data is delivered that does not meet project specifications (whether they are tested or not). For those purchasing professional LiDAR services, you are well advised to know your provider and “trust but verify” that their deliverables meet your specifications. Depending on the intended use of the data, it is often advisable to statistically verify the accuracy of the deliverables to specific standards.

Texas River, Woolpert

 

Mike Tully

Mike Tully is the president and CEO of Aerial Services, Inc. He is a Certified Photogrammetrist (CP) and Geographic Information Systems Professional (GISP), as well as a member of the American Society for Photogrammetry & Remote Sensing (ASPRS), Management Association for Private Photogrammetric Surveyors (MAPPS), Society of American Foresters (SAF), and National States Geographic Information Council (NSGIC).


RELATED ARTICLES

To leave a comment on this story, please log in:

COMMENTS