3000 Photo Map, and Metashape report

[jmason702]

Here is the errors, Thoughts? 3000 photos, nadir, and cross at 70 degrees pitch, 15 MPH, 175 AGL, (i could fly no higher, jets overhead) 75/65 overlap

 

[jmason702]

26 markers on the control point, emlid RS2, also, RS2 sending corrections to the drone

 

[Dave Pitman]

26 markers on the control point

Huh?

 

[jaja6009]

Huh?

I think he means he moved the flag on 26 images on that control point.

 

[jmason702]

I think he means he moved the flag on 26 images on that control point.

yes

 

[jmason702]

My question is really, by comparing your CPs to your photo errors how is this justifying the accuracy of your RTK flight and photos? I'm just trying to understand it...
This go around my CP error was greater than my photo error ( I did set my base up too quickly I think this time )
Also, I know I should have 3 or 4 CPs.
So to help me understand, how do we read the data? The CPs vs the Photo RTK errors

Thanks

 

[Dave Pitman]

My question is really, by comparing your CPs to your photo errors how is this justifying the accuracy of your RTK flight and photos? I'm just trying to understand it...
This go around my CP error was greater than my photo error ( I did set my base up too quickly I think this time )
Also, I know I should have 3 or 4 CPs.
So to help me understand, how do we read the data? The CPs vs the Photo RTK errors

Thanks

I don't know the math on where the error #s on the photos come from exactly. But consider them just an estimate based upon what the drone wrote to the image exif and what the processor thinks after it triangulates tie points. The values are probably good to use relatively between the photo sets of different projects, but without outside auditing via checkpoints, they have very little weight.

If you turn that check point into a control point, and re-optimize, the accuracy of the surface will most likely improve, especially the vertical, but the error values on the images themselves may get worse. But that doesn't matter. With an RTK flight if you set 6 points, and use 1 as a control point and 5 as checkpoints, you will get the benefit of tightning up the accuracy, especially the vertical, and getting a better average of real error expected on the surface.

The #s you got are not bad. The vertical is off quite a bit, but that is to be expected. That would be helped out with a control point or 2. I believe @elphtrooper's workflow is to take the checkpoint error value and adjust the whole surface by that amount, rather than add in control points, which gets you to close to the same place in the end.

But to your question, the error values on the photo set is good to know, but in the end, it is not significant in "prooving" the accuracy of the surface. Hopefully that makes sense?

 

[jmason702]

I don't know the math on where the error #s on the photos come from exactly. But consider them just an estimate based upon what the drone wrote to the image exif and what the processor thinks after it triangulates tie points. The values are probably good to use relatively between the photo sets of different projects, but without outside auditing via checkpoints, they have very little weight.

If you turn that check point into a control point, and re-optimize, the accuracy of the surface will most likely improve, especially the vertical, but the error values on the images themselves may get worse. But that doesn't matter. With an RTK flight if you set 6 points, and use 1 as a control point and 5 as checkpoints, you will get the benefit of tightning up the accuracy, especially the vertical, and getting a better average of real error expected on the surface.

The #s you got are not bad. The vertical is off quite a bit, but that is to be expected. That would be helped out with a control point or 2. I believe @elphtrooper's workflow is to take the checkpoint error value and adjust the whole surface by that amount, rather than add in control points, which gets you to close to the same place in the end.

But to your question, the error values on the photo set is good to know, but in the end, it is not significant in "prooving" the accuracy of the surface. Hopefully that makes sense?

Yes it does make sense, thank you, just trying to better understand how the checkpoints verify the map

 

[Dave Pitman]

Yes it does make sense, thank you, just trying to better understand how the checkpoints verify the map

If the check points are good, the error reported by them IS the error. All the other metrics are secondary. If your image set error is reported as 0 and your checkpoints error is 5, the error for your surface is 5.

 

[jmason702]

If the check points are good, the error reported by them IS the error. All the other metrics are secondary. If your image set error is reported as 0 and your checkpoints error is 5, the error for your surface is 5.

10-4 Got it!

 

[jaja6009]

Also you can try some non-default parameter changes in Metashape.

If using all obliques or a large percentage of obliques, use the Reference Preselection Source. Then in the Reference pane in Capture Distance put in your average relative flight height.

When using no GCPs use the Fit Additional Corrections option on Optimize.

This is what I do for all obliques or a large number of them and no GCPs.


Did you purchase Metashape? If so check out the tiled models, they are excellent!

 

[Dave Pitman]

If using all obliques or a large percentage of obliques,

Have you found that including obliques with a nadir image set improves the vertical reconstruction? Both the P4RTK and Evo2 RTK have the option to include a string of photos at the end of a mission to reduce vertical error. I have not tried it myself (yet). I am never asked for pretty models, or models at all actually, so I am always focused on 2D accuracy.

 

[jaja6009]

I missed your Check Point error in the initial post.
You are having the dreaded vertical shift with your X and Y being spot on and then a consistent vertical shift.

On my Baam Tech PPK kit this shift was always present and consistent. Their explanation was the Camera Calibration by the software was incorrect.
It was fixed by a mandatory GCP. Once the GCP was introduced, the error dropped to a more expected 3 times GSD or under.

One of the reasons I chose my Phantom 4 RTK was a calibrated camera file in the XMP that Agisoft imports in along with the images. There is debate on whether to use this or to let the software calibrate the camera when ran.
My workflow uses this info but I still Optimize, but before optimize I still have pretty good low RMSE error on the Check Points.

On large maps I will always use GCPs. I might try just one but when I did my 200 acre project I used 5.

I have have good success though on small maps not using any GCPs.

As Dave Pitman stated, when asked the error or accuracy of your product, you would give the RMSE from the Check Points and not the Camera's error.

Further elaboration on my previous post's processing choices.

Reference Preselection Source on the Align Images pane.


Fit Additional Corrections on the Optimize pane.



For the Fit additional corrections, I am assuming that even though it says Phantom 4 RTK, it is also referring to other RTK aircraft, but I might ask this on the Agisoft forum, or perhaps Dave Pitman knows.

 

[jaja6009]

Have you found that including obliques with a nadir image set improves the vertical reconstruction? Both the P4RTK and Evo2 RTK have the option to include a string of photos at the end of a mission to reduce vertical error. I have not tried it myself (yet). I am never asked for pretty models, or models at all actually, so I am always focused on 2D accuracy.

Yes and yes.

On the Phantom 4 RTK using only nadir images, I found a vertical shift. But then adding what DJI calls Altitude Optimization which is what you said with a flight line of nadir images going from the outside area of interest to the center, the vertical shift was corrected.

I am writing a White paper for Public Safety on the use of common Public Safety themed drone used for mapping. I tested as is, or out of the box with no GCPs and using the onboard GPS as well as then adding properly spaced GCPs.
I then compared these outputs to a Phantom 4 Pro V2 with PPK kit (1 GCP used per the PPK manufacturer's directions) and then a Phantom 4 RTK (No GCPs used for the RTK bird).

The Phantom 4 RTK with no GCPs using just nadir with Altitude Optimization (1 line of obliques from the outside corner to the center) was had very low RMSE.

The Phantom 4 RTK with all obliques and no GCPs was the most accurate of all tested aircraft and workflows. It beat out:

Phantom 4 RTK nadir with Altitude Optimization
Phantom 4 Pro V2 PPK with 1 GCP
Phantom 4 Pro V2 with 9 GCPs
Parrot Anafi with 9 GCPs
Mavic Enterprise (Zoom) 9 GCPs
Mavic Enterprise Advanced 9 GCPs

Snip from the paper giving the various RMSE.



For the Phantom 4 RTK, 1 Grid means lawnmower pattern Nadir and 2 Grid means two grids flown with one being rotated 90 degrees and all images being obliques.
The Phantom 4 Pro V2 PPK is a Phantom 4 Pro V2 with Baamtech PPK kit and I obtained the RINEX from the Reach RS2 located on a NGS monument 100 to 200 yards away from the test area.

Off topic, how about the Anafi!
With no GCPs it was surprisingly good even with the RMSE in meters, but when you add in GCPs it did great!

 

[Dave Pitman]

One of the reasons I chose my Phantom 4 RTK was a calibrated camera file in the XMP that Agisoft imports in along with the images.

I've run a few sets from a P4RTK through MS and after importing the images, when I open TOOLS > CAMERA CALIBRATION there are no values. I had assumed they would be populated with the xmp data but it doesn't look like it. I asked about it on the AS forum and got no input.

The Phantom 4 RTK with all obliques and no GCPs was the most accurate of all tested aircraft and workflows.

Very interesting. What angle did you use?

Your paper looks very nice.

 

[jmason702]

WOW thanks for all the info guys!!

 

[jmason702]

Im still bouncing around trials in meta shape, as I still have 6 months on my DD sub.
I love DD for orthos, and I love meta shape because it can do anything!

 

[jmason702]

When using no GCPs use the Fit Additional Corrections option on Optimize.

this is what I do

 

[jaja6009]

I would screen shot it, but Metashape is busy processing and this is gonna be a long one.

I will screen shot it for you by tomorrow.

I think it is in Tools, Preferences and then Advanced and is named appropriately.


My paper of course is not any kind of end all assessment. It is only stating results for the area flown using a pretty much default workflow. It is to help introduce agencies that discover mapping to the problems they will run into the moment they realize that their pretty maps have bad to atrocious elevation values mostly due to them not understanding what elevations the drone is reporting (relative or height above ellipsoid) and what the software is expecting. This is for either accident/scene reconstruction or even natural disasters with flooding being something that would need to have correct elevations using the correct datum and/or geoid.

It will also help to show that they need to decide what type of accuracy they are looking for and resist the urge to force a Public Safety type drone to perform mapping when there are low cost mapping solutions available that will do it better, faster and even safer (No GCP collecting in the hazard area).

I wish I had jmason's Autel to throw in the mix too as I really love the images it takes and the RTK on it is legit.
If I already didn't have too many drones, I would add one to the fleet.

And I used 60 degrees on the obliques.

I've run a few sets from a P4RTK through MS and after importing the images, when I open TOOLS > CAMERA CALIBRATION there are no values. I had assumed they would be populated with the xmp data but it doesn't look like it. I asked about it on the AS forum and got no input.


Very interesting. What angle did you use?

Your paper looks very nice.

Ok after some testing.

Go to Tools, Preferences and Advanced tab and then make sure this is checked off.




If I import images from my Phantom 4 Pro V2 I get this in Tools, Camera Calibration



Notice Type is Auto
And this is Phantom 4 Pro V2 Type is Precalibrated.



No calibration parameters.

Here is the Phantom 4 RTK



On the Phantom 4 RTK is Type is Auto it is empty except for the f: Focal Length

Alexy on Agisoft has stated in the forum that he thinks letting the software perform the camera calibration is the better option.
I have not really tested this, as my workflow with the DJI camera calibration has worked well for me.

 

[jmason702]

You are awesome!!