Nvidia shows deep learning method to fill gaps in images afbeeldingen

Nvidia has presented its own research into a way to perform image inpainting, which is to eliminate holes or damaged areas in an image. It uses deep learning for this with its own technology that would ensure better results.

Comparison by Nvidia

According to Nvidia, the method can not only be used to fill in missing parts of an image, but also to remove elements of the image. The company has published a video showing how its method works. This could be used in image editing software, according to Nvidia. The company claims its model can handle “holes of any shape, size and distance from the edge of the image.” The details of the research are described in a paper.

In it, the researchers write that their research differs from previous work, which would mainly focus on rectangular image parts in the center of the image. However, the new method would also work with irregular shapes. Moreover, the methods proposed so far would often lead to visual artifacts in the final image. Nvidia wants to counteract this by not making output for missing pixels dependent on the input for those pixels, according to the company.

The researchers used a dataset of about 55,000 masks for training purposes. This concerns missing parts in images. The images used came from other datasets, namely ImageNet, Places2, and CelebA-HQ. The training took place on an Nvidia V100 accelerator with 16GB of memory.