The sites allow users to create AI-generated images of real people, swapping their faces onto nude images in a violation of state and federal laws prohibiting deepfake pornography, revenge pornography and child pornography, Chiu's office said.
"This investigation has taken us to the darkest corners of the internet, and I am absolutely horrified for the women and girls who have had to endure this exploitation," Chiu said in a statement. "Generative AI has enormous promise, but as with all new technologies, there are unintended consequences and criminals seeking to exploit the new technology."
Chief Deputy City Attorney Yvonne Meré said at a news conference Thursday that, as the mother of a 15-year-old girl, she was appalled to learn about the capabilities of some of these websites and put together a team after talking with Chiu to investigate.
"How can it be that this pernicious practice can go on without consequence?" she said, adding that "individual victims of this conduct have little practical and legal recourse."
Leading AI companies such as OpenAI and Anthropic place limits on the kind of violent or explicit imagery their models will generate. But in an as yet unregulated space like AI, companies like those those Chiu is going after train AI image models on pornographic material and depictions of child sexual abuse, his office said, "so that they can be used to generate nude images of real, identifiable women and girls without their consent."
Deputy City Attorney Karun Tilak said during the news conference that it was not entirely clear which AI programs were being used to create the images. He said earlier versions of AI image programs such as Stable Diffusion that lacked more recent safeguards would probably be capable of creating them.
Tilak also said his office knows the identity of some of the website owners, but that some "have hidden in the shadows" and that "through this lawsuit we hope to obtain their identities."
Chiu's office said San Francisco Superior Court is still the proper jurisdiction because "defendants are engaging in unlawful and unfair business practices in San Francisco." He added during the news conference that because victims were based in California and state laws were being violated, his office is able to bring the suit.
The suit is seeking civil penalties and for the sites to be blocked by web hosts from continuing to post the alleged illegal content. So as not to push traffic to the companies' websites, their URLs were redacted in the legal complaint filed Thursday.
The sites collectively have been visited something like 200 million times in the first six months of this year, according to Chiu's office. They work by allowing users to pay to upload a clothed image of a person that is run through an AI image model to create a pornographic image.
Chiu's office pointed to an Los Angeles Times story from February in which nonconsensual nude images of 16 eighth-grade students were circulated among students at a Beverly Hills middle school.
Another example of the technology going viral was when deepfaked nude images of Taylor Swift began appearing on X this year.
Separately, the latest version of an AI model built by Elon Musk-owned company xAI began allowing users to generate violent and racy if not outright pornographic material this week. Those images users posted online and said were generated by the Grok-2 model included a scantily clad Kamala Harris along with past presidents appearing to do drugs and as Donald Trump and "Sesame Street" characters wielding weapons.
Deepfakes depicting politicians have become a particular area of concern with an election looming in November. California lawmakers are seeking to pass bills to limit their usage around elections.
Speaking about the websites his office is suing, Chiu said in a statement: "We have to be very clear that this is not innovation — this is sexual abuse. This is a big, multifaceted problem that we, as a society, need to solve as soon as possible. We all need to do our part to crack down on bad actors using AI to exploit and abuse real people, including children."
© 2024 the San Francisco Chronicle. Distributed by Tribune Content Agency, LLC.
