Viral ChatGPT poses propaganda and hacking risks, researchers warn
Ever since OpenAI’s viral chatbot was unveiled late last year, detractors have lined up to flag potential misuse of ChatGPT by email scammers, bots, stalkers and hackers.
The latest warning is particularly eye-catching: It comes from OpenAI itself. Two of its policy researchers were among the six authors of a new report that investigates the threat of AI-enabled influence operations. (One of them has since left OpenAI.)
“Our bottom-line judgment is that language models will be useful for propagandists and will likely transform online influence operations,” according to a blog accompanying the report, which was published Wednesday morning.
Concerns about advanced chatbots don’t stop at influence operations. Cybersecurity experts warn that ChatGPT and similar AI models could lower the bar for hackers to write malicious code to target existing or newly discovered vulnerabilities. Check Point Software Technologies Ltd., an Israel-based cybersecurity company, said attackers were already musing on hacking forums how to re-create malware strains or dark web marketplaces using the chatbot.
Several cybersecurity experts stressed that any malicious code provided by the model is only as good as the user and the questions asked of it. Still, they said it could help less sophisticated hackers with such things as developing better lures or automating post-exploitation actions. Another concern is if hackers develop their own AI models.
We interviewed ChatGPT, a chatbot that has garnered widespread attention for its ability to mimic human conversation. Then we brought in experts in artificial intelligence and the arts to analyze ChatGPT’s responses.
WithSecure, a cybersecurity company based in Helsinki, contends in a new report also out Wednesday that bad actors will soon learn how to game ChatGPT by figuring out how to ask malicious prompts that could feed into phishing attempts, harassment and fake news.
“It’s now reasonable to assume any new communication you receive may have been written with the help of a robot,” Andy Patel, intelligence researcher at WithSecure, said in a statement.
A representative for OpenAI didn’t respond to a request for comment, nor did the researchers for OpenAI who worked on the report on influence operations. The FBI, National Security Agency and National Security Council declined to comment on the risks of such AI-generated models.
Kyle Hanslovan, who used to create offensive cyber exploits for the U.S. government before he set up his own defensive company, Huntress, based in Ellicott City, Md., was among those who said there are limits to what ChatGPT could deliver. He told Bloomberg News that it was unlikely to create sophisticated new exploits of the sort a nation-state attacker can generate “because it lacks a lot of creativity and finesse.” But like several other security experts, he said it would help non-English speakers craft markedly better phishing emails.
Hanslovan said that ChatGPT is ultimately likely to give defenders “a little bit better of an upper hand” than the attackers.
Juan Andres Guerrero-Saade, senior director of SentinelLabs at the cybersecurity company SentinelOne, said ChatGPT knows code better than him when it comes to the painstaking world of reverse engineering and “deobfuscation” — the effort to uncover the secrets and sorcerers behind malicious source code.
Guerrero-Saade was so astounded by the ChatGPT’s capabilities that he’s thrown out his teaching syllabus for delving into nation-state hackers. Next week, he said more than two dozen students in his class at the Johns Hopkins School of Advanced International Studies will hear his belief that ChatGPT can be a force for good.
It can make the building blocks of code legible quicker than he can manually, and more cheaply than expensive software, he said. Guerrero-Saade said he has been asking it to go back and reanalyze CaddyWiper malware that targeted Ukraine and find errors in his and others’ initial analysis.
“There’s really not that many malware analysts in the world right now,” he said. “So this is a sizable force multiplier.”
In the study on AI-enabled influence operations, the researchers said their main worries were that the campaigns could be cheaper, easier to scale, instant, more persuasive and harder to identify using the AI tools. The report is an effort by Georgetown University’s Center for Security and Emerging Technology, OpenAI and the Stanford Internet Observatory.
The authors also “outline steps that can be taken before language models are used for influence operations at scale,” such as teaching AI models how to be “more fact sensitive,” imposing stricter restrictions on usage of models and developing AI technology that can identify the work of other AI machines, according to the report and the blog.
But the risks are clear from the report, which was started well before the release of ChatGPT. “There are no silver bullets for minimizing the risk of AI-generated disinformation,” it concludes.
