Facebook makes artificial intelligence for image recognition open source
The Facebook Artificial Intelligence Research lab, or FAIR, has made DeepMask, SharpMask and MultiPathNet open source. The three pieces of software can determine from a photo what appears in it, where those parts are located and where exactly its outline is.
Facebook will announce the opening on Thursday in an extensive blog post. The three software components work together to divide a photo into multiple identified segments. A demo of the software in action is also available for those interested. DeepMask turns an image into a low-res version and determines what to see based on the rough shapes, SharpMask sharpens the photo as expected and then actually points out the objects within the frame and finally MultiPathNet makes accurate masks that exactly fit over the objects, to indicate exactly where the objects begin and end.
The AI works with machine learning, which means that it is not programmed how to recognize objects, but is presented with large series of images and then told what is in those photos to see. That way the AI ’learns’ how different objects in a photo can look like and, after seeing enough examples, she can do this work herself without a human having to ‘give it up’.
The blind and partially sighted could be told by the AI what can be seen in a photo, regardless of whether the photo has a description and whether it is correct. In the even longer term, blind users could even touch parts of the photo and be told exactly what they are putting their finger on, as if it were some kind of photo-Braille. In the future, Facebook hopes to have refined the technology so that it can also be released on video for the same purpose. Applications in augmented reality are also possible.
It is not unique in itself that Facebook gives a peek behind the scenes when it comes to AI routines. Over the past year, the company has written extensive papers on all three strands and published them for everyone. The motivation for making the three routines open source has been done “with the hope that the public will help dramatically accelerate advances in machine vision.” After all, the Facebook AI also has to compete with Google’s TensorFlow software, which is also open source. DeepMask and SharpMask are together on GitHub and MultiPathNet is separate.