Meta said on Wednesday it had discovered “likely AI-generated” content being used fraudulently on its Facebook and Instagram platforms, including comments praising Israel’s handling of the Gaza war posted under posts by global news organizations and US lawmakers.
The social media company said in a quarterly security report that the accounts are posing as Jewish students, African-Americans and other concerned citizens, targeting audiences in the United States and Canada. She attributed the campaign to Tel Aviv-based political marketing firm STOIC.
STOIC did not immediately respond to a request for comment on the allegations.
Why it matters
While Meta has found basic AI-generated profile photos in influence operations since 2019, the report is the first to reveal the use of text-based generative AI technology since it emerged in late 2022.
Researchers are concerned that generative artificial intelligence, which can quickly and cheaply produce human-like text, images and sound, could lead to more effective disinformation campaigns and influence elections.
In a press call, Meta security executives said they took down the Israeli campaign early and did not think new AI technologies impeded their ability to disrupt influence networks, which are coordinated attempts to push messages.
Executives said they had not seen such networks using AI-generated images of politicians realistic enough to be mistaken for authentic photos.
Key quote
“There are several examples in these networks of them using probably generative AI tools to create content. Maybe that gives them the ability to do it faster or to do it with more volume. But it hasn’t really affected our ability to detect them,” he said. Meta’s Director of Threat Investigation Mike Dvilyanski.
By the numbers
The report highlighted six covert influence operations that Meta disrupted in the first quarter.
In addition to the STOIC network, Meta shut down an Iran-based network that focused on the Israel-Hamas conflict, although it did not identify any use of generative artificial intelligence in that campaign.
Context
Meta and other tech giants have grappled with how to address the potential misuse of new AI technologies, particularly in elections.
The researchers found examples of image generators from companies including OpenAI and Microsoft producing photos with voting-related disinformation, despite those companies having policies against such content.
Companies have highlighted digital tagging systems for tagging AI-generated content at the time of its creation, though the tools don’t work on text and researchers doubt their effectiveness.
What’s next
Meta faces key tests of its defenses with elections in the European Union in early June and in the United States in November.
© Thomson Reuters 2024
