North, South, East or, West?
Me? North (in my home office).
A heading of about 5 degrees, if you want me to be more accurate.
Now look at the 50th photo (roughly) you took on your phone. Any idea of the heading?
What about the pitch?
And it is very unlikely your phone will have the answer either.
Usually you will find it tagged with where the photo was taken using GPS (latitude, longitude and elevation), but I’m not aware of a phone that logs the heading (prove me wrong!).
There’s good reason for this; rarely is it important.
Though in some case it’s life or death. This is no exaggeration.
Take Bellingcat. Their crowdsourced work often involves taking a photograph and determining the location, heading, and even time (using shadows) it was taken to aid their investigations (like the downing of MH17).
Because the Trek Pack camera (GoPro Fusion) does not support heading or pitch when shooting timelapse tours, we needed to manually calculate this information when building Tourer.
Now you might be wondering why pitch or heading is important. “Surely a user can define where they want to be looking using a 360-degree photo viewer?”, you might be asking. This is true. Though when trying to link 360-degree photos in a tour (e.g what direction is the next photo?), this information is vital.
Using latitude and longitude captured by the GoPro Fusion GPS receiver we have all the information we need to make an fairly accurate calculation of these values.
Before we begin, take a moment to visualise this as a 3D map. The latitude (
x) and longitude (
y) gives the horizontal position. The elevation (
z) gives us the vertical position.
To calculate distance, we can use the Haversine formula to find the distance between two points on a sphere (Earth) given their longitudes and latitudes.
Two things to note: 1) this is probably overkill as the curvature of the earth is minuscule at short distances (e.g. 3 metres) and, 2) it does not account for elevation change between points, although again given distances in question, this will be small.
The best way to think about this calculation is to visualise the photos as points in a line from left to right (first photo to last).
Now we can turn to trigonometry (“sohcahtoa”), to calculate the missing values for these right angled triangles we’ve visualised;
Sine = Opposite / Hypotenuse
Cosine = Adjacent / Hypotenuse
Tangent = Opposite / Adjacent
pitch(θ) = tan(θ) = opposite / adjacent
We have the adjacent measurement (distance between photos), we just need the opposite value (elevation change between photos).
The GPS logs the elevation, so we can work this out;
source photo elevation - destination photo elevation = elevation change (note, can be positive or negative).
So now we have;
pitch(θ) = tan(θ) = opposite / adjacent = elevation change / distance.
You’ll notice in the diagram Tourer generates a forward and backward connection (P1 - P2 and P2 - P1) for each connected photo. This is so you can move forward and backwards between photos.
Heading or orientation is generally measured from North (0°), and is often referred to as the azimuth.
If I’m facing North my heading is 0°. If I’m facing South my heading is 180°. And so on.
All our tours are currently filmed in time order. The start / end of the tour is the first / last photo.
Given photos are generally less than 3m apart and our Trek Pack cameras are always facing forward / backwards in the same direction, we can make an assumption that the camera is facing in the direction of the next photo (by time). Note, this will not always be correct, for example, if Trekker turns 90° between start and destination photo (but this is rare, so we’re overlooking it for now in favour of simplicity).
Now we can calculate the heading of the current photo by measuring the heading (angle) from North to the next photo (by time), as shown in the diagram.
There is a slight problem with this approach, we cannot get the heading of the last photo (because there is not a next photo by time). As a simple fix, we built Tourer to assign the previous photo heading (by time) to the last photo in the tour.
Now we have an estimate for what direction the camera was facing when the photo was taken.
In some case we have multiple connections. For example, at a fork where two paths intersect – one going forward, two others off to each side.
So we not only need to know the heading to the next photo based on time, but also the direction to all connected photos so that a user can move through a tour virtually (think Google Street View).
First we need to consider what photos are connected. In Tourer photos are considered connected if the horizontal distance between photos is <=10 metres of the elevation change is <=+/-5 (both these values calculated earlier).
So we can use the same logic as before to work out heading for all connected photos.
As with pitch, you’ll notice in the diagram Tourer generates a forward and backward connection heading (P1 - P2 and P2 - P1) for each connected photo. And like with pitch, it is so you can move forward and backwards between photos.
One final calculation (I promise); adjusted heading.
This is useful for some software, including Explorer, where the 360-degree photos are rendered visually, as shown above, and where the software has no concept of North. That is the software simply assumes the centre of the 360-degree photo is always facing North (0°).
As such we cannot use the heading (degrees from North) calculated previously on its own.
We must make therefore adjust the heading to connected photos to the centre of the source 360-degree photo (it’s heading from North) in these scenarios.
This an easy calculation to estimate using existing heading fields;
adjusted heading = heading of source photo - heading of connection photo.
You will see there is no adjusted heading for the next photo by time (e.g P2 to P3). It does exist, but is not shown in the diagram because the adjusted photo heading for the next photo by time (heading) will always be 0. For example, heading from P2 to P3 = x, and adjusted heading for P2 to P3 = x (heading) - x (connection heading) = 0.
Jump into the code
We are not geospacial experts and know these calculations can probably be improved in place of estimates. You can help us out…
All the code for Tourer is open source. You can see how we calculate these values in the Tourer Gitlab repo – and point out any mistakes we’ve made.