Apple tightens App Store safeguards against nonconsensual AI intimate images

Apple has long barred apps designed to generate sexually explicit imagery. Its updated App Store rules now spell out developers’ responsibility to stop nonconsensual intimate images, including AI-generated ones, from circulating through their apps. The change makes an existing prohibition more explicit while clarifying penalties for developers who hide nudification features from reviewers.

The revised moderation requirements cover apps that host user-generated content, not just tools built to create sexual images. As Engadget explains, developers must prevent uploads of nonconsensual intimate imagery and remove material that gets through their filters. An app’s stated purpose is therefore not the only issue: how it handles content supplied by users also falls within the rules.

Filtering is only one requirement

• Filter objectionable material, including nonconsensual intimate imagery generated with AI tools.
• Give users a way to report offensive content and block abusive accounts.
• Respond to complaints and remove prohibited images that slip through.

Apps risk removal for failing to implement sufficient protections, or for not remedying a violation once notified. The obligation is not limited to taking down a particular image after someone complains; prevention is part of the requirement too.

Approval does not authorize a hidden feature

Apple also clarified its longstanding prohibition on hidden, dormant or undocumented features. That includes an app introducing nudification capabilities only after it has passed review. Here, nudification means turning an ordinary image of a real person into sexually explicit imagery. The clarification addresses functionality concealed from the review process, rather than only material users later upload.

The update follows pressure on the app marketplaces. In July, San Francisco City Attorney David Chiu sent cease-and-desist letters to Apple and Google demanding the removal of apps capable of generating nonconsensual intimate imagery, according to Digital Trends. The letters followed investigations into AI tools that could digitally undress people.