Analysing what camera roll looks like in equirectangular photos and how to account for it.
As I have mentioned on the blog many times, almost all 360 cameras include the following GPS tags in the metadata of photos they take;
GPS:GPSLatitude: decimal Latitude
GPS:GPSLongitude: decimal Longitude
GPS:GPSAltitude: decimal Altitude
0(Above Sea Level) or
1(Below Sea Level)
GPS:GPSTimeStamp: UTC time of GPS fix
GPS:GPSDateStamp: Date of GPS fix
XMP-GPano:PoseHeadingDegrees: Heading in degrees
XMP-GPano:PosePitchDegrees: Pitch in degrees
XMP-GPano:PoseRollDegrees: Roll in degrees
GPS:GPSImgDirection: Heading in degrees
T(True direction) or
GPS:GPSPitch: Pitch angle in degrees
GPS:GPSRoll: Roll angle in degrees
EXIF:CameraElevationAngle: Pitch angle in degrees with positive pitch upwards
Note, it is important to note the difference between
XMP-GPano:InitialViewHeadingDegrees: Heading offset in degrees
XMP-GPano:InitialViewPitchDegrees: Pitch offset in degrees
XMP-GPano:InitialViewRollDegrees: Roll offset in degrees
As touched on in the last few posts,
XMP-GPano:InitialView..., is only concerned with the viewer. Any value (in degrees) set for these fields will offset the viewer from corresponding
XMP-GPano:Pose... values, but have no relation to the actual heading, pitch, or roll in the real world.
XMP-GPano:Pose... considers the real world heading, pitch, or roll (and not the viewer).
XMP-GPano:Pose... values are not set, the
XMP-GPano:InitialView... values entered will be offset from 0 (its default value if none set). If
XMP-GPano:Pose... values are set, the respective
XMP-GPano:InitialView... values will be offset from the
XMP-GPano:Pose... values set.
All that being said, GoPro 360 Cameras do not include any information in photos beyond GPS positional information (
I’ve talked previously about calculating rough values for heading and pitch when other sensor telemetry is not available by working out the heading and the pitch to the next photo in a photo sequence (using latitude, longitude, and altitude).
In an attempt to try an level the horizon of my 360 photos I need to be able to determine the camera roll, however,
alt values cannot be used to do this.
So down the rabbit whole I went.
The following posts detail my attempt to try and estimate roll in photos taken on GoPro 360 cameras (or any 360 camera where limited metadata is written into the photos).
What roll looks like in equirectangular projections
When you have look at enough equirectangular photos outside of a 360 viewer you begin to identify roll by eye.
Let me demonstrate…
Using my example roll video from a few weeks ago;
I have extracted frames at roughly (done by eye) at 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees, and 315 degrees.
You can see the timestamps I used to capture the frames in ffmpeg command;
ffmpeg -ss HH:MM:SS -i GS010011.mp4 -frames:v 1 -q:v 40 GS018421_HHMMSS.jpg
I’ve also annotated the horizon in red in each frame.
0 degrees (
ffmpeg -ss 00:00:01 -i GS010011.mp4 -frames:v 1 -q:v 40 GS018421_000001.jpg
45 degrees (
ffmpeg -ss 00:00:03 -i GS010011.mp4 -frames:v 1 -q:v 40 GS018421_000003.jpg
90 degrees (
ffmpeg -ss 00:00:06 -i GS010011.mp4 -frames:v 1 -q:v 40 GS018421_000006.jpg
135 degrees (
ffmpeg -ss 00:00:08 -i GS010011.mp4 -frames:v 1 -q:v 40 GS018421_000008.jpg
180 degrees (
ffmpeg -ss 00:00:10 -i GS010011.mp4 -frames:v 1 -q:v 40 GS018421_000010.jpg
225 degrees (
ffmpeg -ss 00:00:12 -i GS010011.mp4 -frames:v 1 -q:v 40 GS018421_000012.jpg
270 degrees (
ffmpeg -ss 00:00:14 -i GS010011.mp4 -frames:v 1 -q:v 40 GS018421_000014.jpg
315 degrees (
ffmpeg -ss 00:00:16 -i GS010011.mp4 -frames:v 1 -q:v 40 GS018421_000016.jpg
Reviewing the wave (with a clear horizon)
You can see clear patterns as the camera rolls around the
As the roll increases to 45 degrees the sine wave gets increasingly distorted. As it reaches 90 degrees the curve gets closer to a square wave. As it gets closer to 135 degrees, the sine wave flattens again (and is a horizontal mirror of the 45 degree wave).
As a human, this pattern is fairly easy to spot assuming a good proportion of a horizon is visible and fairly level. It gets a bit more complicated when there are obstructions to the horizon or the photos is in an enclosed space.
Dealing with obstructions to the horizon
As almost all our photos are taken outdoors, I won’t consider photos taken indoors in these posts.
Regarding obstructions, here is a photo taken along a ridge line.
The camera is fairly level, as you can see from the annotated horizon. However, without the annotation, the rising ridge in front of the camera makes it a little harder to spot as a human at first glance.
But what about a computer? I will take a look at that next week…