POST my_beautiful_panoramas.mp4

Those who use our Map the Paths Uploader will be familiar with its ability to upload your sequences to Google Street View.

Those of you who are even more techie might know we use the Google Street View Publish API’s photo resource to upload each image (developer docs here).

Those of you who like to take things apart might enjoy a previous post of mine introducing exactly how the Street View API and photo resource works.

In short, currently the Map the Paths Uploader works with the Google Street View Publish API like so:

  1. The Uploader creates a sequence of photo files from videos/photos selected
  2. User selects to the Street View integration, and assigns a placeID to the sequence.
  3. The Uploader uploads all photos in a sequence 1-by-1 to Street View using the photo resource.
  4. The Street View API returns information about the photo on Google (including its photoId. The Uploader stores the returned photoId for each image in the sequence.
  5. Around 72 hours later the mapsPublishStatus of each photo moves to PUBLISHED or REJECTED_UNKNOWN.
  6. The Uploader syncs this information with Map the Paths Web.

This process has worked fairly well to date, but it starts to cause issues for tours of 30+ images. There are a few reasons for this:

As more Map the Paths trekkers upload longer tours, it became clear we needed to take a look at alternative methods, and discovered Google offered an alternative resource named photoSequence on the Street View Publish API.

Officially this is still in alpha phase (you will need to request access), but there are some good examples of how it’s used in the wild, including this example on GitHub.

That gopro_fusion_timelapse_uploader.py script converts a series of timelapse photos into a photoSequence (packaging all the photos into a single .mp4 file for uploading).

Not only does the photoSequence resource save the number of requests to the Google API to upload images, it is useful because it natively supports auto-connections and blurring not available for the photo resource (no more rejected blue lines!).

Looking at the script in more detail, the overall flow works like so;

  1. Extract extract_geodata from images (gps timestamp, latitude, longitude, altitude) to create a rawGpsTimeline.
  2. Then convert_video which takes images and creates an .mp4 file from the photos using ffmpeg.
  3. Optionally blur images if the --blur flag is selected. Note, this is done Street View server side and supports both blurFaces and blurLicensePlates.
  4. Then upload_video which uploads the .mp4 video created earlier
  5. And finally, publish_video which uploads the geodata (the rawGpsTimeline created in step 1).

The full photoSequence resource looks like this.

{
  "id": string,
  "photos": [
    {
      object (Photo)
    }
  ],
  "uploadReference": {
    object (UploadRef)
  },
  "captureTimeOverride": string,
  "rawGpsTimeline": [
    {
      object (Pose)
    }
  ],
  "gpsSource": enum (GpsSource),
  "imu": {
    object (Imu)
  },
  "processingState": enum (ProcessingState),
  "failureReason": enum (ProcessingFailureReason),
  "blurringOptions": {
    object (BlurringOptions)
  }
}

After successful upload a full photoSequence blob will be returned with an id and nested photos object information (including photoId’s) for each image in your sequence.

The photoSequence then enters ProcessingState: PENDING or PROCESSING. This indicates it is going through all the backend Street View checks and processing steps before being published to Google Maps.

After the Street View processing completes (usually 72 hours, but potentially longer), the photoSequence should enter ProcessingState: PROCESSED.

However, processing can, and often does fail, producing one of the following potential ProcessingFailureReason error

`ProcessingFailureReason` valueError Description
PROCESSING_FAILURE_REASON_UNSPECIFIEDThe failure reason is unspecified, this is the default value.
LOW_RESOLUTIONVideo frame's resolution is too small.
DUPLICATEThis video has been uploaded before.
INSUFFICIENT_GPSToo few GPS points.
NO_OVERLAP_GPSNo overlap between the time frame of GPS track and the time frame of video.
INVALID_GPSGPS is invalid (e.x. all GPS points are at (0,0))
FAILED_TO_REFINE_POSITIONSThe sequence of photos could not be accurately located in the world.
TAKEDOWNThe sequence was taken down for policy reasons.
CORRUPT_VIDEOThe video file was corrupt.
INTERNALA permanent failure in the underlying system occurred.
INVALID_VIDEO_FORMATThe video format is invalid or unsupported.

One downside of using the photoSequence upload is that if one image in the uploaded video fails the Street View server side checks, the entire photoSequence will fail. As opposed to photo upload, where a single photo failure will not result in an entire failure for all photos uploaded in the sequence.

One final thing to be aware of; the photoSequence upload does not allow you to directly assign a placeID to photos. Thus, this needs to be done manually using the photo resource. Once the photoSequence upload is complete and photoId’s are returned, you can update each photo 1-by-1 with the place information.

Ultimately, if you’re working with a sequence with more than 30 images, go with the photoSequence method (and make sure any bad images are removed before submission to reduce the likelihood of failed checks), otherwise the photo approach described here will work just fine.

Coming soon to Map the Paths Uploader…

Map the Paths Uploader image spacing

We’re planning to migrate from photo to photoSequence when uploading final sequences to Google Street View in the Map the Paths Uploader after making any desired edits or deletions.

As noted, the Uploader supports the photo upload resource at present, but if you’ve got a slightly longer sequence it is worth holding out for a few more weeks when we’ll ship the more stable photoSequence approach in the next version (v0.4) of the Map the Paths Uploader.

Stay tuned for the announcement by signing up for Trek View updates!


Posted by:

David Greenwood

David Greenwood, Trek View Chief Explorer



Off-Road Street View

Map the Paths Tours

Explore trails from all over the world captured on foot, by bike, on skis, and using many other adventurous forms of transport. Fill in the gaps by requesting new coverage or capturing your own.