Facebook to generate photo captions with artificial intelligence

Facebook will automatically generate captions based on the content of photos via artificial intelligence. The function is called ‘automatic alternative text’ and is intended to ensure that visually impaired users can view the content of a photo.

The generated text can then be read out again by screen reader software, Facebook reports. The system recognizes the content of photos based on a neural network, which has learned to recognize photos from millions of examples.

For example, it can describe a photo with terms such as ‘contains three people, smiling, outside’. Until now, screen reader users only heard when a photo was shown, and who shared it. Facebook’s new feature is the result of a longer-term study that focuses on how visually impaired people use the social network.

The feature will be available first to users with English language screen readers on iOS. Later, Facebook also wants to implement the new capabilities on other platforms. Support should also be extended to additional languages.