Artificial intelligence Samsung creates animations based on images

Samsung has designed a system using artificial intelligence to create animations based on images. For example, a person can be animated based on characteristics extracted from photographs.

The software was created by a Samsung development center in Moscow, in collaboration with the Skolkovo Institute of Science and Technology. The makers have put a scientific paper online on Arxiv, but in a video they briefly explain how the system works. Three neural networks are created that, based on a few images, or frames from videos, can build a model that can be used to animate images. The system has been tested with images of several people, as demonstrated in the video.

In an example, the makers show how an animation of a person can be made on the basis of only eight frames. To do this, one of the neural networks converts the frames used into vectors that can be used to create animations. Another neural network looks at facial features in the images to be analysed, while the third network looks at the ‘realism’ and pose of the animated person.

It is also possible to build an animated model with just one frame, although the makers point out that more images result in an animation that is more realistic. In addition, the software also works with paintings to a limited extent; thus the developers managed to generate moving images of the Mona Lisa.

Traditionally, to create animations, a 3d model is built of the image to be animated. That is not necessary with this software: Samsung’s AI itself extracts the relevant information from images and uses it to build animations. There is more software that builds animations based on artificial intelligence; such software is used, among other things, to create so-called deepfakes.