San Francisco city attorney David Chiu sent cease-and-desist letters to Apple and Google on Thursday, demanding the companies remove 13 apps from their app stores that allow users to create AI-generated nonconsensual nude images, according to letters seen by WIRED.
Legal Action Against Deepfake Nudification Apps
The letters accuse Apple and Google of “aiding and abetting” the sale of explicit deepfake images and call on the Silicon Valley giants to “sever” business relationships with the app developers. Chiu told WIRED that the companies have likely “made millions of dollars in fees” from apps that offer “nudification.” The legal notices cite California laws prohibiting supporting services that create deepfake pornography.
“Generating non-consensual intimate images is illegal, harmful, and completely unacceptable,” Chiu said. “These companies have responsibility to ensure that apps on their platforms do not facilitate sexual abuse.”
The city attorney’s office previously took legal action against 16 popular deepfake websites, and the latest letters target eight apps on Apple’s App Store and five on Google Play.
Industry Response
Google spokesperson Dan Jackson told WIRED that the company has deleted “hundreds” of apps with nudifying features for policy violations, including the five Android apps flagged by Chiu’s office. “Google Play does not allow apps that contain sexual content, and we continually take proactive steps to detect and remove apps with harmful content,” Jackson said in a statement. He added that the company has also restricted search terms like “nudify” on its store.
Apple did not provide comment ahead of publication. Both companies have developer policies that prohibit pornography, abuse, and harassment, and have previously removed dozens of such apps after reports by researchers and journalists.
| Entity | Action Taken |
|---|---|
| Apple | No comment; 8 apps flagged by Chiu |
| Removed “hundreds” of apps; restricted search terms; 5 Android apps flagged |
The Rise of Deepfake Nudification Technology
Over the last five years, a lucrative ecosystem of deepfake “nudification” tech has emerged online, most prominently with xAI’s Grok being used to create millions of sexualized images in January. A host of apps, websites, and bots allow people—overwhelmingly men—to upload pictures of women and girls and digitally “remove” clothing or place them into graphic sexual scenarios. Some results are available in seconds, and the technology has become more realistic as generative AI improves.
Previous reporting by WIRED and Indicator Media uncovered incidents in at least 90 schools where deepfake sexual abuse images were created of minors. Chiu emphasized the human toll: “These images are used to bully, humiliate, and threaten women and girls… There have been victims who’ve been suicidal.”
Implications for Enterprise Technology Leaders
The San Francisco action highlights growing regulatory scrutiny of platform liability for third-party AI applications. For enterprise technology leaders responsible for supply chains, logistics platforms, or digital trade systems, the case underscores the need for rigorous content moderation and compliance checks when integrating or hosting AI tools. Regulators are increasingly holding platform operators accountable for facilitating harmful uses of AI, even when the technology is provided by external developers. Companies must ensure their own AI deployments and marketplaces do not expose them to similar legal risks, particularly as laws like California’s set precedents for holding tech firms responsible for the misuse of their platforms.
Sources:
