wv3 accuracy study with histogram

The spark that ignited the PhotoSat accuracy studies

wv3 accuracy study with histogram

2016 PhotoSat WorldView-3 satellite surveying accuracy study, Asmara, Eritrea.

“Everything that the last speaker just told you is wrong”.   This shocked me since I was the last speaker.  I was just rejoining the audience after my presentation at a satellite data distributors’ conference in San Diego in 2008.

I had given a presentation on PhotoSat’s experience using satellite photos for elevation mapping.  I had shown comparisons between PhotoSat stereo IKONOS satellite elevation mapping and hundreds of mining exploration drill  collar elevations.  Our results suggested an IKONOS mapping accuracy of better than 1.0m in elevation.

The speaker who followed me showed the published specifications of the IKONOS satellite.  He declared that this proved that the results I had just shown were impossible. Then he went on to talk about his own stereo IKONOS mapping results.  His results showed 5m to 10m in elevation mapping accuracy.

 

Looking for a way to unambiguously measure our accuracy

The speaker who challenged PhotoSat’s results in the San Diego meeting actually did us a huge favor.  Although it did not feel like that as I sat fuming in my chair.  His comments provided the motivation for me to find a way to prove we were right.  After this meeting we set about looking for a way to unambiguously demonstrate the accuracy of the PhotoSat stereo satellite elevation mapping.

 

Searching for a detailed, high quality, ground survey data set

We concluded that to prove our accuracy we needed to find a highly accurate ground survey data set covering hundreds of square kilometers. But where to find it?

About two months later, a light came on.  I realized that we might find the elevation survey data that we needed from a large, regional, mining exploration gravity survey.  The topographic surveys associated with mining exploration gravity surveys are among the most accurate and carefully checked topographic surveys in the world.

 

An old friend tells me about an existing ground survey data set

I phoned Kevin MacNabb, president of MWH Geo-Surveys International.  Kevin did gravity survey contracts for me when I was a geophysicist at BP.  I said “Kevin, I am looking for a large regional gravity survey with thousands of accurate topographic survey points.  I want to use the topographic survey data to measure the accuracy of PhotoSat’s stereo IKONOS satellite mapping.”

I added “it would be great if the data is in an area of sparse vegetation in a remote region of the world.  This way we can prove the accuracy of our stereo satellite topographic mapping and show that we can do this anywhere in the world.”

Kevin replied “how would 45,000 ground survey points covering over a thousand square kilometers just west of Asmara, Eritrea do?”  For us this was the perfect data set. Eritrea was a challenging place to work.  It had just emerged from a civil war.  A perfect place to be mapping from satellites. For a fuller description, click this link.

 

Eritrea differential GPS survey crew and equiptment

Asmara Project, Eritrea. MWH Geo-Surveys differential GPS survey crew and equipment. Over 45,000 ground points were surveyed between 2004 and 2008. The Magellan RTK base with a ProMarkTM 500 GPS rover are shown in this photo.

 

The Eritea ground survey data is owned by an existing PhotoSat customer

It turned out that Kevin’s customer for the Eritrea gravity survey, Sunridge Gold, was also a PhotoSat customer for stereo IKONOS mapping.  We negotiated the right to use the 45,000 ground survey points for accuracy studies.  In return, we did some additional stereo IKONOS mapping for the company.

 

PhotoSat’s first comprehensive accuracy study

We were immediately able to use 10,000 of the Eritrea ground survey points to measure the elevation accuracy of 200 km² of stereo IKONOS elevation grid.  PhotoSat had already produced this elevation grid for Sunridge Gold.  We measured the accuracy of the PhotoSat elevation grid as 48cm Root Mean Square Error (RMSE).  The full 2008 IKONOS Eritrea Accuracy study is available to review.

 

PhotoSat accuracy measurement and improvement

Since 2008, PhotoSat has been using the 45,000 Eritrea ground survey points as a test bed to measure accuracy improvements in the PhotoSat processing.  This gives us a quantitative measure of accuracy improvements.  We have shown the results in many conferences and published them.  If there are still disbelievers they are certainly not challenging us publicly.

 

Satellite companies start to provide stereo test data over the Eritrea site.

In 2009, two commercial high resolution satellite companies, GeoEye and DigitaGlobe, provided stereo satellite photos over the PhotoSat Eritrea test area.   The DigitalGlobe data was from the 50cm ground resolution WorldView-1 satellite launched in September 18, 2007.  The GeoEye data was from the 50cm ground resolution GeoEye-1 satellite launched on September 6, 2008.

PhotoSat published elevation mapping accuracy reports for both new satellite systems.  The stereo GeoEye-1 PhotoSat elevation grid had an accuracy of 31cm RMSE, determined by 8,893 ground survey points.  The stereo WorldView-1 PhotoSat elevation grid had an accuracy of 35cm RMSE, determined by over 15,000 ground survey points.

 

WorldView-2 joint DigitalGlobe and PhotoSat news release on Eritrea accuracy study

Soon after the commissioning of the WorldView-2 satellite in early 2010 DigitalGlobe asked PhotoSat to use its new processing system to conduct an accuracy study over the Eritrea test area using stereo WorldView-2 photos.  The PhotoSat Eritrea Accuracy study showed WorldView-2 accuracy of better than 30cm RMSE.  These results were issued on March 16, 2010.  The news release is available here.

 

With accuracy improvement PhotoSat mapping becomes PhotoSat surveying

Once the PhotoSat elevation grids achieved an accuracy of better than 30cm many of our customers began using them in place of ground surveying.  We consequently renamed our products that have accuracy better than 30cm to surveying products.

 

The Eritrea ground survey data set has been used for hundreds of PhotoSat accuracy tests and studies

Since PhotoSat first acquired the 45,000 ground survey points in Eritrea, we have used the data for hundreds of accuracy test and studies.

 

2016 PhotoSat Eritrea accuracy studies

In 2016, we used the current version of the PhotoSat Geophysical Processing System to process a full range a stereo satellite photos over the Eritrea test area.  Some of these results are published on our website on the links below.  The link names include the satellite name, the number of ground control points used in the processing and the RMSE accuracy.

 

WorldView-3, Eritrea, 21 GCP, RMSE 15cm

WorldView-2, Eritrea, 21 GCP, RMSE 14cm

WorldView-3, Eritrea, 2 GCP, RMSE 19cm

WorldView-2, Eritrea, 2 GCP, RMSE 19cm

WorldView-1, Eritrea, 21 GCP, RMSE 19cm

WorldView-1, Eritrea, 9 GCP, RMSE 23cm

Pleiades-1B, Eritrea, 74 GCP, RMSE 26cm

Pleiades-1B, Eritrea, 1 GCP, RMSE 28cm

Kompsat-3A, Eritrea, 11 GCP, RMSE 48cm

Kompsat-3A, Eritrea, 1 GCP, RMSE 53cm

ALOS PRISM, Eritrea, 3 GCP, RMSE 1.4m 

SPOT 7, Eritrea, 1 GCP, RMSE 1.4m

ALOS PRISM, Eritrea, 1 GCP, RMSE 2.4m 

 

If you would like more information on PhotoSat surveying you can visit the following links.

Satellite surveying

Mining Industry Applications

Oil and Gas Industry Applications

Case histories

Satellite survey of Libya with ground survey points

Are PhotoSat satellite surveys really more reliable than ground surveys?

By Gerry Mitchell, P.Geo, President PhotoSat

Satellite survey of Libya with ground survey points

World View 2 satellite survey in Libya with ground survey points for 3D oil and gas seismic survey.

 

In 2008, I would get a hollow feeling in the pit of my stomach whenever customers phoned or emailed to tell us that the PhotoSat surveys did not match their ground surveys. Back then, I was sure there was a problem with the satellite photos, or that we had made some terrible mistake in our processing.

However, by 2013 our customers were using PhotoSat satellite surveys to check and adjust their ground surveys.

Ground surveys right, PhotoSat surveys wrong?

Back in 2008 it was clear to everyone, including me, that ground survey data was right and satellite survey data wrong. After all, ground survey data was collected by someone who had stood on the ground. It was the “ground truth”.

In comparison, the PhotoSat surveys were produced from satellite photos taken from 750 kilometers above the earth. Of course the ground surveys were right and the satellite photo surveys were wrong. Or so we thought. As we did more and more satellite surveying projects, we began finding obvious ground survey errors. Some of these projects had thousands of ground survey points.

ground target at mining project drill hole

Ground target at a mining project drill hole with arms 20cm wide and 1m long. The image to the right show how this target appears on a 50cm ground resolution GeoEye satellite photo.

 

Some ground surveys wrong, PhotoSat right

We began finding projects with two sets of ground survey data, one set matching the PhotoSat surveying perfectly and the other set mismatching. Investigating these cases with customers was eye-opening and often entertaining.

On one project in southern Mexico, all of the ground survey points on or near roads matched the PhotoSat surveying perfectly. The points in remote areas, particularly on the tops of hills, had differences of two to five meters in elevation.

So what happened? The contract surveyor had used his high quality, bulky, GPS surveying equipment for the survey points that he could drive to. But he had sent his young assistant with a hiker’s GPS to all of the survey points on the hilltops. The surveyor was confident that the client would never check those remote points.

By 2010, we had become much more confident in the accuracy and reliability of our PhotoSat surveying. When there was a mismatch between PhotoSat surveying and ground GPS surveying we began suggesting that the PhotoSat surveying was “usually right”.  As you can well imagine, many of our customers, and all of their surveyors, were sure that I was delusional.

We had to find another strategy. We began saying, “Thanks for telling us about our mistake. Please send us a copy of your survey data and we will see if we can identify and fix our problem”.

Drill hole with 40cm by 40cm white concrete block

Drill hole with a 40cm by 40cm white concrete block on a mining project in central Mexico. The drill holes on this project were surveyed three times by three different GPS survey contractors. The coordinates of this drill hole on the three surveys are shown on an extreme zoom of the PhotoSat WorldView satellite survey. The PhotoSat survey has more reliable coordinates than any of the 3 GPS surveys.

 

Not all GPS surveys created equal

We also learned that not all ground GPS surveys are created equal. All good quality GPS systems record the GPS signals for later processing, so we began asking for copies of these GPS recordings along with the GPS ground survey coordinates.

The effect of this simple request on the quality of the survey data was remarkable. Some surveyors would immediately go and resurvey the ground points as soon as we asked for the GPS recordings.

By processing the GPS recordings we could see how long the surveyors had been at each point. In many of the significant mismatches, we discovered that the GPS recording times were much too short for ten centimeter accuracy. At first we had pushback from many surveyors when we suggested that their coordinates were probably inaccurate.

Then in June, 2011, the International Association of Oil and Gas producers published a thick report of guidelines for GPS surveying, freely available as a PDF file. For us this was a godsend.

Whenever there was debate about GPS accuracy, we would email a copy of the report saying, “These are the guidelines that we are relying on.  Please tell us where they are wrong”. All the discussions about recording times for GPS accuracy stopped.

Most ground surveys are good quality

Of course, it’s worth noting that most of the ground surveys that we receive are very good. Only occasionally are there serious problems that we cannot easily resolve. And of course PhotoSat also makes occasional processing errors and mistakes.

We always investigate whenever the PhotoSat surveying does not match the ground surveying. When we find it to be our mistake, we fix the PhotoSat survey data and resend it to the customer at no cost.

Correcting multiple mismatching ground surveys

By 2013, many of our repeat customers no longer assumed that the PhotoSat surveys were wrong when they did not match their ground surveys. In just five years there had been a 180 degree shift. In 2008, ground surveying always proved that PhotoSat surveying was wrong. By 2013 the PhotoSat surveying was being used to quality check and fix ground surveying.

This is great for projects with several different ground surveys. These are often surveyed by different contractors. For example, we have one case of an oil and gas project with five different ground GPS surveys performed by five different contractors. We proved that none of the ground surveys matched any of the other four ground surveys.

The key ground survey was for an oil well that discovered several hundred million barrels of oil. We matched the PhotoSat survey to the discovery well. All of the other four surveys mismatched the PhotoSat survey, each by different horizontal and vertical distances.

We used the PhotoSat survey to measure the offsets of each of the ground surveys from the oil well. Then we adjusted the other four surveys to match the oil well. This gave the project a consistent set of ground surveys all matched to the oil well.

This case history is described in more detail here.

Give GPS surveys on an oil and gas poject

Five GPS surveys on an oil and gas project. Each was done by a different survey contractor. None of the surveys matched each other. PhotoSat detected the mismatches and adjusted the survey data to produce a coherent survey data set.

 

Please see the experience section of the PhotoSat website for additional case histories and accuracy studies.

LiDAR and satellite elevation data

PhotoSat verifies accuracy of DigitalGlobe’s 30cm WorldView-3 satellite elevation data to within 15cm

PhotoSat has recently completed a study to measure the accuracy of the elevation grid produced from the new 30cm resolution WorldView-3 (WV3) satellite. We measured the accuracy of our topographic mapping by comparing it to a highly accurate LiDAR elevation grid. The study was carried out over an 88 km2 area in Southeast California that overlaps an Open Topography LiDAR survey.

Read the full elevation accuracy report here (PDF)

For the study, PhotoSat produced a 50cm grid of elevations using our proprietary geophysical processing technology with stereo satellite images taken by WV3. Our resulting elevations were then compared to a 50cm LiDAR elevation grid, which is accurate to about 5cm. The resulting 15cm RMSE elevation accuracy was impressively achieved using a single ground reference point.

Below are some images of the elevation surveys and the differences between the datasets. You can also view the full WorldView-3 elevation accuracy study (PDF) on our website.

For more information on our highly accurate satellite topography, contact us at info@photosat.ca or 1-604-681-9770.

WV3 30cm resolution satellite ortho photo
Figure 1: WV3 30cm resolution satellite ortho photo created from WV3 stereo photos, for the area of the LiDAR survey used in this study.

LiDAR elevation grid
Figure 2: An image showing a portion of the LiDAR elevation grid. Lower elevations are blue, and higher elevations are red.

PhotoSat’s WV3 elevation grid image
Figure 3: PhotoSat’s WV3 elevation grid image covering the area of the LiDAR image. The grid has an elevation point every 50cm. At this scale, the LiDAR and WV3 images are identical. Lower elevations are blue, and higher elevations are red.

PhotoSat’s WV3 elevation grid clipped to the LiDAR extents
Figure 4: PhotoSat’s WV3 elevation grid clipped to the LiDAR extents, for areas with slopes less than 20% grade. Areas where development occurred since the 2008 LiDAR survey were removed for the accuracy analysis.

Differences between our WV3 elevation grid and the LiDAR elevation grid
Figure 5: The differences between our WV3 elevation grid and the LiDAR elevation grid, for areas with slopes less than 20% grade, are shown in a standard histogram on the left and a cumulative histogram on the right. If we assume that the LiDAR is perfect, the WV3 elevations have a Root Mean Square Error (RMSE) of 15cm. Ninety percent of the WV3 elevations are within 22cm of the LiDAR elevations giving a 90% Linear Error (LE90) of 22cm.

Comparison of the LiDAR and WV3 elevation grids for 1000m wide area
Figure 6: Comparison of the LiDAR and WV3 elevation grids for 1000m wide area. Minor differences between the elevation grids are visible at this scale.

Continue reading PhotoSat verifies accuracy of DigitalGlobe’s 30cm WorldView-3 satellite elevation data to within 15cm