I continue taking apart a .360 file, and trying to rebuild it with a equirectangular projection (without GoPro software).
After extracting the video tracks from GoPro’s .360 format in last weeks post, this week I wanted to better understand the structure of each frame.
I suspected there might be duplicate pixels written into the frames after reading GoPro’s post about their .360 format, and looking at the dimensions of the frames extracted last week.
Looking at the widths of each of the 3 squares in each frame in track 0 and track 5, you can see the two outer squares, where the stitch lines are present, are wider (1376px) than that in the middle (1344px). The middle being a true square (1344x1344).
The outer squares are 32 px wider (1376-1344). And measure 688 pixels to the stitch line.
Using imagemagick I decided to break the two tracks into the 6 cubefaces used to take a closer look.
Track 0:
$ convert -crop 1376x1344+0+0 track0/img1.jpg pre_cubefaces/left_img1.jpg
$ convert -crop 1344x1344+1376+0 track0/img1.jpg final_cubefaces/forward_img1.jpg
$ convert -crop 1376x1344+2720+0 track0/img1.jpg pre_cubefaces/right_img1.jpg
Track 5:
$ convert -crop 1376x1344+0+0 track5/img1.jpg -rotate pre_cubefaces/bottom_img1.jpg
$ convert -crop 1344x1344+1376+0 track5/img1.jpg -rotate final_cubefaces/back_img1.jpg
$ convert -crop 1376x1344+2720+0 track5/img1.jpg pre_cubefaces/top_img1.jpg
Take the image below taken from the right frame of a video shot in my backgarden.
Here, we can clearly see the overlapping area, and it in fact measures 64 pixels (in a video shot at 5.6k). So we have identical sets of pixels on each side of the center (32px either side) of the image.
It is this 64 pixel strip that needs to be blended. Which makes sense. The 64 overlap, gives 2 strips of 32 pixel overlap each side of the center. 32 pixels being the extra width of the overlapping frame (1376 vs 1344). So after blending, 32 pixels are removed, and the squares of these cubefaces reduced by 32 pixels to make a square, as required.
You might be thinking; why can’t we just cut one strip of duplicated 32 pixels? It’s not that simple.
Let’s look at an example. First by splitting the left cubeface into two at the stitch line:
$ convert -crop 688x1344+0+0 pre_cubefaces/left_img1.jpg precut_cubefaces/left_l_img1.jpg
$ convert -crop 688x1344+688+0 pre_cubefaces/left_img1.jpg precut_cubefaces/left_r_img1.jpg
Then by placing the right side on-top of the left side (over the duplicate pixel area):
$ convert -size 1344x1344 xc:transparent PNG32:precut_cubefaces/left_l_r_img1.png
$ convert precut_cubefaces/left_l_r_img1.png precut_cubefaces/left_l_img1.jpg -geometry +0+0 -composite PNG32:precut_cubefaces/left_l_r_img1.png
$ convert precut_cubefaces/left_l_r_img1.png precut_cubefaces/left_r_img1.jpg -geometry +640+0 -composite PNG32:precut_cubefaces/left_l_r_img1.png
If you look closely you can still see duplicate pixels, and a clear stitch line.
Therefore we need to apply a blend.
At a high-level the process works like this:
- Overlap the two images by 32 pixels, as above
- Then blend using alpha compositing (alpha + (1-alpha)
This will leave us with 6 perfect square cubefaces (1344x1344).
Once we blend the left, right, bottom, and top images, we can then rebuild as the cubemap with the new dimensions (4032x2688).
First we create the image
convert -size 4032x2688 xc:transparent PNG32:img1_cubemap.png
And then by placing each cube within it:
# top row
$ convert img1_cubemap.png cubefaces/left_img1.jpg -geometry +0+0 -composite PNG32:img1_cubemap.png
$ convert img1_cubemap.png cubefaces/forward_img1.jpg -geometry +1344+0 -composite PNG32:img1_cubemap.png
$ convert img1_cubemap.png cubefaces/right_img1.jpg -geometry +2688+0 -composite PNG32:img1_cubemap.png
# bottom row
$ convert img1_cubemap.png cubefaces/bottom_img1.jpg -geometry +0+1344 -composite PNG32:img1_cubemap.png
$ convert img1_cubemap.png cubefaces/back_img1.jpg -geometry +1344+1344 -composite PNG32:img1_cubemap.png
$ convert img1_cubemap.png cubefaces/top_img1.jpg -geometry +2688+1344 -composite PNG32:img1_cubemap.png
Now we have a blended EAC cubemap that is ready to be converted to an equirectangular projection.
Never miss an update
Sign up to receive new articles in your inbox as they published.
Discuss this post
