Google’s Motion Stills turns moving photos into gifs

Google has made an application for iOS that glues the photos from Apple Live Photos together in such a way that a smoothly moving gif is created. For that, it uses an algorithm locally on the iPhone to stabilize the photos, eliminating the need for an internet connection.

To achieve a technique where the calculation is done on the device itself, the algorithm uses linear programming to calculate a virtual path that the camera would have traveled as if a stabilization system had been used during filming. In addition, the image in the background should be quieter than in the foreground, so that the image as a whole appears calmer.

Google’s challenge was to run techniques that are normally applied via distributed computing on a mobile phone. In addition, the company managed to achieve a speed gain of forty times through optimizations. The app also uses the GPU as if real-time textures are to be rendered for a game.

To make loops more beautiful, where the foreground does move, Motion Stills differentiates between foreground and background, something that according to Google’s Research Blog is difficult if the foreground takes up a large part of the image. The method classifies motion vectors between foreground and background, after which different motion models estimate the most important results

The app is available through the App Store for iOS devices.

Left: original with virtual camera path and movement classification: foreground red, background green. Right: the final result.