Researchers show AI tool that can manipulate images with two mouse clicks

Researchers have shown an experimental AI tool that can fully manipulate images in no time. For example, it must be possible to rotate a face and adjust facial expressions within two mouse clicks.

The tool, DragGAN, is intended to make specific changes to certain parts of images without changing the entire image. This requires users to pick two points on the photo: the part to be manipulated and the spot on the photo where that part should go. The tool should do the rest, say the researchers.

DragGAN allows anyone to distort an image with precise control over where pixels go, manipulating the pose, shape, expression and layout of diverse categories such as animals, cars, people and landscapes. Unlike, for example, Photoshop’s Warp tool, existing pixels are not changed, but the neural net creates completely new pixels.

The examples shown show, among other things, that the tool can open a lion’s mouth and then make the animal look in a different direction by rotating its head. People’s clothing, such as sleeves, are also lengthened, and arms and legs can be seen being moved. Another example shows how entire landscapes are modified.

Most of the images in the examples are AI-generated, but according to researchers, the tool should also be able to handle real photos. This is demonstrated by a smiling photo of US President Joe Biden, who is changed in a few minutes to a shocked expression, with his head also rotated.

According to the researchers, the tool is able to achieve “realistic results that consistently follow the rigidity of the object, even for challenging scenarios, such as hidden parts.” The team promises to post the tool’s code to GitHub next month.

Six researchers, who work at German and American universities, have contributed to the project. Abhimitra Meka, research scientist at Google’s AR department, also contributed.