Expanding Access to LiDAR Through Compression

by Christine Grahl
02 August 2012

What happens when a computer scientist with an uncanny knack for geometry compression takes an interest in LiDAR data? In the case of Martin Isenburg, a self-described “massive data processor and common sense extractor” based in Germany, the result is an award-winning open-source software solution that is making it easier for government agencies and organizations to store and transmit increasingly larger LiDAR datasets by compressing data from the ASPRS LAS format data into much smaller LAZ files.

Developer of the LAStools software for processing LiDAR data, Isenburg worked extensively on compressing the connectivity and geometry of polygonal meshes—first as a student at UNC Chapel Hill, then as a postdoctoral researcher at UC Berkeley, and finally on his own as a consultant. When he introduced LASzip as part of LAStools in July 2007, it was mostly an experiment in his free time; he had no idea how the compression tool would be received in the commercial sector.

“I had been working on compressing large meshes for many years without anybody actually using this technology because there was simply no widespread demand for such data,” Isenburg said. “But for LiDAR points there was so much content available in an already established format, and more and more was being generated, so I believed having a compressed version of LAS would be useful.”

As word began to spread about LASzip, a number of government agencies and private organizations became interested in testing this technology. A sponsor soon stepped forward to fund improvements in the code and ensure that the tool would be released open source under an LGPL license. From there, “it sort of took off,” Isenburg said.

The LASzip compression is lossless, meaning that the LiDAR data is retained exactly: point coordinates, intensities, classification—even the order. The amount of compression that can be achieved on a single dataset therefore varies depending on the order of the points in the file—points in random order achieve much less compression than points that are still in the order in which they were acquired, which is typically the case. Files can often be compressed to as little as 7 to 25 percent of their original size. For LiDAR power users such as NOAA, the USGS, the Minnesota Department of Natural Resources and a number of private organizations, such a high level of compression saves on storage and transmission bandwidth; for example, moving Digital Coast LiDAR data to LASzip recently saved NOAA 15 terabytes of expensive disk space. More importantly it makes it easier to share datasets both internally and externally.

In April 2012, LAStools received a Technology Innovation Award at the Geospatial World Forum in Amsterdam for the LASzip compression tool. Popular commercial software packages such as Certainty 3D’s TopoDOT, Safe Software’s FME and Blue Marble Geographics’ Global Mapper already have native LAZ support. Isenburg said that QT Modeler and Fugroviewer are expected to follow in their next release, and RIEGL, a major manufacturer of scanner hardware, plans to have RiProcess output directly to LAZ files.

According to Isenburg, a major breakthrough came when the Land Survey of Finland decided to publish its entire national LiDAR holdings exclusively in LAZ format as part of their new open data policy. Since then, Isenburg has been in contact with many other state and federal agencies across Europe and elsewhere about the use of LASzip for storing LiDAR.

For Isenburg, who long approached his work on compression as mostly a hobby, the growing interest in LASzip—and LAStools in general—is opening new doors of opportunity. Isenburg recently added a LiDAR processing toolbox for Esri's ArcGIS that allows users to easily access LAStools within ArcGIS 9.3, 10.0 and 10.1. Through his newly formed company rapidlasso, Isenburg intends to push LAStools even further by adding more tools, better tutorials and user training and, importantly, a more visible presence in the market. “Right now I have a software that in many ways is on equal footing or even better than some of the big players that sell a lot of seats at a high price,” he said. “What I don’t have is the glossy brochures, the PR campaigns, the nice website and the tradeshow booths that make you feel like you’re getting a fancy product.” Step by step that will be coming, he said.

The change means that more users of LAStools may choose to pay for a commercial-grade license that includes the complete collection of scriptable tools with multi-core batching that processes standard LAS, compressed LAZ, Terrasolid BIN, ESRI Shapefiles and ASCII files. However, several popular tools, including the LASzip compressor, will remain available as a free, open-source, standalone software.

“At this point, I see a reasonably good chance for LASzip to become an official recognized industry standard for LiDAR compression,” Isenburg said.

-------------

Image: False-color elevations, standard deviations of elevation, highest intensity, hill-shaded elevations and point densities from the Minnesota state LiDAR collection (ftp://ftp.lmic.state.mn.us/pub/data/elevation/lidar/), generated with lasgrid and blast2dem in LAStools.

Christine Grahl

Christine Grahl is the editor of GeoDataPoint. She can be reached at geodatapoint@bnpmedia.com.

Print Article

COMMENTS

On August 02, 2012 Martin Isenburg wrote:

Hello,

I want to acknowledge those folks who have helped tremendously to make LASzip successful. Although I may have developed the compression technology, the quick adaption of LAZ as a “de-facto standard” for LiDAR compression by more and more agencies and industries would not have been possible without the support of others.

LASzip had been available since 2007 on my web page in form of an API (header files and library) and as a binary executable (but without any license). At that time laszip.exe was nothing but an academic proof-of-concept prototype implementation to accompany a planned paper on LiDAR compression. I was surprised to later find out that it had already been used by some commercial data vendors for data deliveries ... (-:

I was first encouraged to “open source” my laszip.exe compressor in the fall of 2009 by two companies and two federal agencies who wanted LiDAR compression but did not want to buy into the only other LiDAR compression technology available at that time - the proprietary LiDAR compressor TM from LizardTech (R) - that - as a commercial solution - came with certain licensing cost structure.

But the open sourcing of LASzip 1.0 did not actually happen until Howard Butler (Hobu Inc) and Dave Finnegan (USACE) convinced me of the benefits and promised me help with integrating and promoting LAZ via their widely popular libLAS library that had forked off LAStools in November 2007. They also motivated me to add random-access to the decompressor. In particular Howard kept nagging me about adding “chunking” - how he coined this functionality - to the LASzip algorithm.

This gave me reason to completely re-factor the LASzip code base into a more modular and more extensible industry-strength implementation that would allow to incorporate changes to the evolving LAS specification in the future. During the redesign to LASzip 2.0 a few more ideas on how to better predict the next point from the previous ones were also integrated to further improve compression ratio and speed.

At this point Michael Gerlek (Flaxen consulting) joined the effort and helped to properly interface LASzip with libLAS (and now PDAL). Via this interface, LAZ support is now available, for example, in FME2012, TopoDOT, Global Mapper, QT Modeler, GRASS, and any other software that uses libLAS or PDAL under the hood.

Finally, Tim Loesch (DNR Minnesota) and Chris Crosby (OpenTopography) were the earliest adopters to provide the LAZ format as an option for downloading large amounts of LiDAR. This gave LAZ the first real exposure to a wider audience.

I want to extend kudos to all of them. Thank’s guys!

Martin @lastools