In a recent expose, Wired magazine revealed a troubling trend in artificial intelligence. An investigation conducted on the messaging platform Telegram created numerous AI-powered chatbots capable of generating explicit deepfake images of real individuals upon request.
This recent review of Telegram communities that share at least one piece of explicit nonconsensual content has revealed at least 50 bots that say they can create explicit photos or videos of individuals with the click of a button. These bots differ in functionality; while many of them claim they can ‘undress’ photos, others boast that they can generate images of people making different sexual poses.
WIRED’s research shows that the 50 bots studied here have over 4 million monthly active users. Two of the bots said they had more than 400,000 users each, while 14 others said they had more than 100,000 members each. These findings show that explicit deepfake creation tools are easily accessible and reaffirm that Telegram is an important platform for such materials. However, this snapshot is limited to English-language bots and probably does not encompass even 10% of all deepfake bots in Telegram.
“We’re talking about a significant, orders-of-magnitude increase in the number of people who are clearly actively using and creating this kind of content,” Ajder states regarding the Telegram bots. “It is really concerning that these tools—which are really ruining lives and creating a very nightmarish scenario primarily for young girls and for women—are still so easy to access and to find on the surface web, on one of the biggest apps in the world.”
Specifically, non-consensual deepfake content, referred to as nonconsensual intimate image abuse (NCII), has increased since late 2017 due to generative AI. Many ‘nudify’ and ‘undress’ sites and Telegram bots work with thousands of women and girls around the globe, including Italy’s PM and schoolgirls in South Korea. A survey conducted earlier this year revealed that 40% of US students knew of deepfakes associated with their K-12 schools in the previous year.
The bots listed by WIRED are connected to at least 25 channels with more than 3 million members total, which share information about new bot features and special ‘tokens’ offering access to its usage. When WIRED asked about its policies on the use of sexually explicit deepfake content, Telegram deleted 75 identified bots and channels but declined to comment on the deletions.
More channels and bots mentioned by WIRED show the extent of the issue; several owners said they would rebuild their bots, saying, “We will make another bot tomorrow.” Those accounts were also deleted.
