Google makes SyntaxNet and Parsey McParseface open source for language comprehension

Google has released the code of its natural language understanding platform under the name SyntaxNet. The framework can be used to analyze language and to give systems some understanding of what texts are about through machine learning.

The open source release contains all the code developers need to train new models. In addition, Parsey McParseface is part of the publication. This parser has already been trained by Google with its self-learning TensorFlow software and users can use it to analyze English texts. According to Google, Parsey McParseface is currently the most accurate model in the world. Google is releasing its parser so that researchers and developers can use it for translations, among other things. Ultimately, Google hopes to develop new methods of knowledge acquisition through systems and understanding of all languages.

Google is one of the organizations that has made significant strides in the development of language comprehension through systems. That development has been going on for decades, but has accelerated in recent years with advances in machine learning.

One of the difficulties with parsing is that the meaning of human expressions is often ambiguous. This is not a problem for humans, but for computers the number of combinations of possible structures is increasing exponentially. “The majority of these structures are unlikely, but they are nevertheless possible and must somehow be excluded by the parser,” Google writes. The company uses neural networks for this, in which sentences are analyzed from left to right. Each time, SyntaxNet scores decisions based on probability. Rather than simply making the decision with the highest score, parts of hypotheses are considered at each step until multiple alternative hypotheses are ranked higher.

Natural language understanding plays a central role in personal assistants such as Google Now, Microsoft’s Cortana, Apple’s Siri and Amazon’s Echo. In addition, the language comprehension techniques are being used for chatbots, which many developers are working on, and which can tap into chat conversations by offering relevant services based on conversations.