Equirectangular, non-equalangle cubemap, equi-angular cubemap, pyramid…
Recently I’ve been doing a lot of research on 360 video encoding.
There are a lots of ways 360 videos are encoded and displayed, including the projection type used.
In 360 photography, a projection type is a way to map a spherical field of view to a flat image.
Different projection types have different benefits. In the world of 360 videos (which can often be very large), video sharing services are always looking to optimise the data size of content they serve to a viewer.
Equirectangular projections
I talked about equirectangular projections in my post last year about adding a custom nadir to a 360 photo.
To recap…
The worlds most famous equirectangular projection – the world map.
If you’ve every looked at a equirectangular projected 360 photo in a “flat” view you will have clearly seen how distorted it is at the top and bottom of the image, but less around the center. You’ll see why when you look at how an equirectangular projection is formed…
The diagram below might help you visualise this better (see how the top circles in right equirectangular projection are distorted from spherical globe):
Many of you will be familiar with equirectangular projection types. Most 360 cameras use this projection type for the final output of image and videos. This is largely because most 360 image software understands how to render equirectangular projection types (when this projection type is present in the metadata).
Example Equirectangular projection
Cubemap projections (non-equalangle cubemaps)
In the world of video games and virtual reality cubemaps are generally the default projection type.
In large part this is because equirectangular images contain a lot of redundant pixels (whereas cubemap can deliver the same image as smaller files, thus saving bandwidth costs) and distortion (around the poles, as shown above, which can be made worse when images are edited).
Cubemaps (or cubic format) uses six cube faces to fill the whole sphere around the viewer.
This is conceptually simple: deform a sphere into a cube, then unfold the cube’s six faces and lay them flat.
This is an improvement over equirectangular projections, but it still causes substantial variation in pixel density as is evident looking at the transformation from globe, to cube, to unraveled cube. You can see the edges of the cubes (shown with circles) are more deformed on the left, right, forward and back sides of the cube faces.
Example Cubemap projection (non-equalangle)
A note about cubemap formats
Download this diagram from Google Drawings here
You should be aware there are a variety of cubemap layouts used by different software, as shown above. Ultimately the result of each is the same, a cube.
Equirectangular To Cubemap conversions
There are lots of free web apps online that convert images using equirectangular projections to cubemaps, for example, this one created by Lucas Crane.
Equi-Angular Cubemap projection (EAC)
In light of the above problems, the Equi-Angular Cubemap (EAC) projection was created by Google’s engineering team (see this Google blog post for the announcement).
EAC is not too different to the non-equalangle cubemap projection, except you’ll see the traditional cubemap has samples of varying length depending on the sample’s location on the cube face (left side of diagram above). Whereas, for EAC, the sample sizes are the same.
You can see in the Equi-Angular Cubemap transformation that the corners of each cube are not as distorted as they were in the non-equalangle cubemap projection (although it should be noted they are still slightly distorted in EAC).
360 videos uploaded to YouTube are converted to EAC and served using this projection type. Paul Bourke also has a good write up about the YouTube 360 format.
Example Equi-Angular projection
Example non-equalangle vs Equi-Angular projection
The non-equalangle and Equi-Angular projections might look identical in the example images used above.
In this final example image above image I’ve overlaid the Equi-Angular cubemap example image with 50% transparency over the non-equalangle example image.
You can now see the images are indeed quite different, although the edges still match perfectly. It can be seen that the EAC version has a “zoom” effect on the interior of the cube faces, this results in a higher resolution there compared to the standard non-equalangle cube map.
Pyramid projection
The Pyramid projection was announced by Facebook in 2016 to support VR video.
When a 360 video is uploaded, we transform it from an equirectangular layout to the pyramid format for each of the 30 viewports, and we create five different resolutions for each stream, for a total of 150 different versions of the same video.
Source: Facebook engineering blog
I won’t attempt to explain this projection type in this blog post as it is fairly complex. This video from the Facebook team gives a very good explanation:
Facebook claims a massive 80% reduction in bandwidth using the Pyramid projection (against the original equirectangular projected video).
A note on Google Street View
Google Street View loads images with an equirectangular projection, and they optimise the way they load based on your device and available bandwidth.
Part of this optimisation involves loading different parts of the image as quadrants, versus the whole image at once. You can see an example of this in the image above. If you have a fast connection and moderately powerful machine, you have probably never seen this because all quadrants have downloaded very quickly.
This optimisation technique is not to be confused with a cubemap projection, the projection type is still equirectangular.
Acknowledgements
Image credits for this post:
- Google Blog post; Bringing pixels front and center in VR video
- Paul Bourke; Converting to/from cubemaps
Never miss an update
Sign up to receive new articles in your inbox as they published.
Discuss this post
