One of the companies behind Stable Diffusion presents text-to-video AI

Runway, one of the makers of the plate generator Stable Diffusion, has presented Gen-2. That’s an AI tool that can turn a written prompt into a three-second silent video. Gen-1 couldn’t do that yet.

Gen-2 comes out as a product for customers of Runway ML, an American company that makes AI tools. Researchers at the company also have one paper put on Arxiv. The company trained the model on images and on video.

There are restrictions on the videos. These are short clips of three seconds long that also contain no sound. The company is investigating possibilities to add audio, reports Bloomberg. Gen-2 videos, like other AI-made footage, also contain weird errors, so that people will often recognize the difference with real footage.

Gen-1 came out earlier this year, but still required users to upload a photo or video preview. That is no longer necessary. Users can adjust details afterwards, such as removing elements from the video. Runway has trained the model not to generate unwanted results through, among other things, human corrections to the output. It is unknown when Gen-2 will be available to the general public.