Group of researchers discovers ChatGPT training data
Researchers from Google DeepMind and several universities have discovered a simple way to get training data used by ChatGPT. By having the chatbot repeat a certain word indefinitely, it shows personal information, among other things.
The scientists have Tuesday a paper published in which they show how they obtained the training data. “With a budget of just $200 for ChatGPT (gpt-3.5-turbo), we were able to retrieve over 10,000 unique training examples,” write the researchers.
They made the chatbot repeat certain words, such as ‘poem’, indefinitely. This forced the language model to deviate from its training procedures and ‘revert to its original language modeling objective’ and make use of details in its training data.
The data includes data that comes from the public Internet, but not necessarily from public sites. This includes names, telephone numbers, e-mail addresses and home addresses. The researchers say it’s concerning that ChatGPT can be so easily abused and say their research should serve as “a cautionary tale for those training future models.”
The group of scientists shared their paper with OpenAI on August 30 and then waited 90 days before publishing. The specific attack would no longer work, but the underlying vulnerability has not yet been resolved, the researchers write.