Bing chatbot made mistakes in demonstration Microsoft
The chatbot of Microsoft’s search engine Bing made gross mistakes in the demonstration Microsoft gave of it last week. That’s what a researcher has figured out. Among other things, the AI produced financial results based on a company’s quarterly figures.
Microsoft compared the quarterly figures of Gap and Lululemon in a demonstration, with the AI incorrectly displaying several values, reports Dmitri Brereton. These values can be found in tables in reports of those quarterly figures. Some of the figures were correct, but others were numbers that did not appear in the reports.
Other Bing demonstrations also had dubious results, according to Brereton. For example, Bing reported that a bar in Mexico City has a website to make reservations, but that is incorrect. Nothing new has appeared online about another bar in recent years and the bar is also not visible at the given address, making it appear that it no longer exists. At several bars, Bing mentions wrong opening hours and the search engine forgets to mention that it is a gay bar.
When comparing vacuum cleaners, Bing lists the url of a different, cordless version of the vacuum cleaner in question. As a result, it is unclear to which version the allegedly mentioned flaws would belong.
Microsoft admits to The Verge the mistakes. “We expect the system to make a lot of mistakes during this preview period, and the feedback is critical to learning where things are going wrong so we can learn from them and make the system better.” Meanwhile, the first regular users have been added to the waiting list gained access to the new version of Bing.
Bing wasn’t the only one with errors in the demonstration. Google’s chatbot Bard said the new James Webb telescope took the first picture of an exoplanet, but in fact the first picture of an exoplanet dates back to 2004.
The chatbots collect information and summarize it with the help of algorithms. This works with a language model that has been trained on a lot of text in order to generate new texts. In this way, Microsoft and Google want the software to be able to answer user questions in detail. The software has no built-in function to distinguish fact from fabrication. Microsoft has the technology for OpenAI’s new Bing, which has released GPT-3 and ChatGPT, among others.
Bing: flaws in demonstration AI