OpenAI has banned the accounts of a group of Chinese users who attempted to use ChatGPT for debugging and editing code for an AI social media surveillance tool, according to the company . This initiative, known as the Peer Review campaign by OpenAI, involved the group prompting ChatGPT to create sales pitches for a program indicated by documents to be aimed at overseeing anti-Chinese sentiment on platforms like X, Facebook, YouTube, Instagram, and others. The operation appeared particularly focused on identifying calls to protest against human rights violations in China, intending to provide these insights to the country’s authorities.
"This network of ChatGPT accounts operated in a time pattern consistent with mainland Chinese business hours, prompted our models in Chinese, and used our tools in a manner and volume indicative of manual prompting rather than automation," said OpenAI. "The operators used our models to proofread claims that their insights had been delivered to Chinese embassies abroad and to intelligence agents monitoring protests in countries including the United States, Germany, and the United Kingdom."
Ben Nimmo, a principal investigator with OpenAI, stated this was the first instance of the company uncovering an AI tool of this nature. "Threat actors sometimes provide us with a glimpse of their activities on other parts of the internet through their use of our AI models," Nimmo told .
Much of the surveillance tool’s code seems to be based on an open-source version of a model from Meta's . The group also appears to have utilized ChatGPT to generate an end-of-year performance review, claiming to have composed phishing emails on behalf of clients in China.
"Evaluating the impact of this activity would necessitate contributions from multiple stakeholders, including operators of any open-source models who can illuminate this activity," OpenAI stated concerning the operation’s attempts to utilize ChatGPT for editing code related to the AI social media surveillance tool.
In a separate incident, OpenAI mentioned it had recently banned an account using ChatGPT to generate social media posts critical of , a Chinese political scientist and dissident now in exile in the US. The same group also used the chatbot to generate articles in Spanish critical of the US, which were published by "mainstream" news organizations in Latin America, often credited to either an individual or a Chinese company.
This article originally appeared on Engadget at https://www.engadget.com/ai/openai-bans-chinese-accounts-using-chatgpt-to-edit-code-for-social-media-surveillance-230451036.html?src=rss
