Nvidia shows deep learning method to convert 30fps image into slow-motion video

Nvidia researchers have developed a method to use neural networks to interpolate video images. This makes it possible to convert a standard recording in, for example, 30 fps into a slow-motion video of, for example, 240 or 480 fps.

Nvidia writes that it is already possible to record images with a high frame density, but that this is impractical in many situations. With the deep learning method, called Super SloMo, it would be possible to add a slow motion effect to an existing standard recording, for example at 30 fps. The researchers claim that their method can be used to generate any number of frames between two existing frames.

In the paper, presented this week at a conference in the US, the researchers write that they use a neural network to predict the content of a frame between two other frames. One network, a cnn, predicts movement between two frames, while a second network is deployed to reduce artifacts.

The training of their model was based on approximately 11,000 YouTube videos at 240fps, where the model had to predict seven intermediate frames. The researchers state that their method performed better than other existing variants. They show, among other things, images from the YouTube channel The Slow Mo Guys, which they then further slowed down.

Nvidia’s research is not an isolated one, there are also other possibilities to slow down existing videos afterwards, such as Twixtor.