For her safety, Doe has opted to receive alerts from the US Department of Justice Victim Notification System any time she may be a victim in a new criminal investigation. Although she has received countless alerts, she was shocked when the CCCP notified her that it had identified AI-generated CSAM on xAI that depicted her. This re-traumatized Doe, whose complaint alleged that messages were found on online forums “between offenders chatting about creating AI generated CSAM of Plaintiff and other similarly situated known, legacy, victims of CSAM.”

Now, Doe fears that xAI has not only made it easier to make more violative images of the most distressing time in her life, but also that xAI allegedly has stored the images that Grok generates and uses those outputs to further train Grok. Because of this, she believes that Grok has been trained on both the initial set of images that have haunted her for more than 20 years and the more recent AI-generated ones.

This is the first case to accuse xAI of training on CSAM, and the complaint does not go into great detail on that claim. Previously, Ars reported on a controversial dataset that was later scrubbed after researchers found CSAM in the training data, but there’s no indication xAI trained on that data. In a press release from lawyers representing Doe, it explained that Doe’s images were included in a CSAM Hash List maintained by NCMEC, and “that same material” allegedly “was part of the dataset xAI used to build Grok’s image and video generating capabilities.” The complaint similarly only alleged that “CSAM depicting Plaintiff with its longstanding well-known hash values has been used as a part of the dataset used by xAI.”

      • Sharkticon@lemmy.zip
        link
        fedilink
        English
        arrow-up
        102
        arrow-down
        1
        ·
        3 days ago

        You know I know you’re joking, but I’m concerned people believe this. Because Elon did hang out with Epstein. There was one email where he was ignored because he was being too thirsty. But that’s not the whole story. He loved the island so much he wanted to go back too hard.

          • Uriel238 [all pronouns]@lemmy.blahaj.zone
            link
            fedilink
            English
            arrow-up
            3
            ·
            2 days ago

            Maybe! Epstein had early designs to essentially manufacture Kompromat by creating a private place for rich men to hook up with young teens. Only he a) got high on his own supply, and b) instead mostly turned to financing and money laundering and so they were more valuable to him as allies than adversaries.

            Also, the owner class, when it’s sufficiently wealthy, can influence political and justice systems enough to be beyond the law. OJ Simpson wasn’t quite there, but he got a good lawyer team to get him acquitted. Andrew Mountbatten-Windsor, formerly Prince Andrew, Duke of York, was there, and was only disgraced because his family cares about the appearance of decency.

            All the tech bros don’t care about appearance of decency. Musk is publicly known to have killed hundreds of thousands by killing USAID, and it won’t affect his career.

      • Rothe@piefed.social
        link
        fedilink
        English
        arrow-up
        2
        ·
        2 days ago

        Yeah, it is concerning that the myth of him not being allowed to the parties is so common on the internet. It is a myth that is very much in favour of Elon, because it makes people believe that he did not attend the pedo-parties, when we know that he absolutely did.

    • partofthevoice@lemmy.zip
      link
      fedilink
      English
      arrow-up
      3
      arrow-down
      1
      ·
      2 days ago

      They’re using photo checksums to identify known CSAM? As in, you can change a single pixel to fool it? Is that right?

      • killingspark@feddit.org
        link
        fedilink
        English
        arrow-up
        6
        ·
        2 days ago

        No. You are thinking about cryptographic hashes, there are locality based hashes that give you a kind of similarity metric

      • one_old_coder@piefed.social
        link
        fedilink
        English
        arrow-up
        2
        ·
        2 days ago

        That would be very stupid. I’m sure they have smarter algorithms that can handle a picture being resized of cropped.

        • partofthevoice@lemmy.zip
          link
          fedilink
          English
          arrow-up
          2
          ·
          2 days ago

          Smarter, so long as the end user doesn’t do something crazy like adjust hue and a tiny bit of compaction?

  • Bluedragon012@lemmy.world
    link
    fedilink
    English
    arrow-up
    8
    ·
    2 days ago

    Not big surprise. We live in a “do evil now, ask for forgiveness” later world. I’m getting tired of evil.

  • Mulligrubs@lemmy.world
    link
    fedilink
    English
    arrow-up
    41
    arrow-down
    2
    ·
    3 days ago

    I don’t believe ANY AI company using internet data to train AI has gone through all of the data to insure it’s not absolute shite.

    That seems pretty obvious when you realize that AI is stupid. Most people are stupid, AI is stupid. Tons of pedophiles, AI makes kiddy porn. Millions of racists, AI is racist.

    Once I realized this, AI made sense. Checks out.

    • skisnow@lemmy.ca
      link
      fedilink
      English
      arrow-up
      17
      arrow-down
      3
      ·
      2 days ago

      I’m morbidly curious about the engineers Musk hires, because overwhelmingly the most intelligent engineers and scientists I know wouldn’t dream of applying to work at one of his companies. Even the ones that don’t care about the Nazi shit have still read the many reports about how badly he treats his staff.

      • Knock_Knock_Lemmy_In@lemmy.world
        link
        fedilink
        English
        arrow-up
        3
        ·
        2 days ago

        Working at spaceX is appealing, because there are not many opportunities to work at the frontier of the industry.

        I can’t imagine any 5 year old wanting to work at the boaring company.

      • grepe@lemmy.world
        link
        fedilink
        English
        arrow-up
        5
        ·
        2 days ago

        i think part of the answer to your curiosity are IT layoffs. job market is shit right now. even though overwhelming majority of people would not work under these conditions if they could choose freely, when they have a mortgage and a family to take care of and they’ve been out of work for long enough the moral concerns and dignity can quickly be overriden by more pressing needs…

    • GalacticGrapefruit@lemmy.world
      link
      fedilink
      English
      arrow-up
      6
      ·
      2 days ago

      *ensure. Ensure means ‘to make secure’, and insure means ‘to have an insurance policy’.

      But you’re right. Unfortunately, the biggest weakness with AI is that it’s a funhouse mirror. Problem is, none of us are having fun.

    • mechoman444@lemmy.world
      link
      fedilink
      English
      arrow-up
      6
      arrow-down
      2
      ·
      edit-2
      2 days ago

      What? That’s not even remotely how that works.

      A human being is also shaped by the information they’ve encountered throughout their life. You’ve encountered racism, stupidity, misinformation, and countless other forms of bad information. That doesn’t mean you automatically become racist, stupid, or incapable of distinguishing good information from bad information.

      AI works similarly in the sense that its training data contains an enormous amount of contradictory, inaccurate, biased, and outright terrible information. The mere presence of that information in the training data does not logically imply that the resulting model possesses those characteristics.

      And AI isn’t “dumb” or “smart” in the way you’re describing. Those are human cognitive attributes. The relevant question is what a model can actually do, what patterns it has learned, and how reliably it performs a given task.

      You don’t like AI. Fine. But you clearly don’t understand how it works, and this is an unfortunately constant problem on this platform. People make an assumption about AI, then present that assumption as though they’ve discovered some fundamental truth about the technology.

      Misinformation is misinformation. It doesn’t become correct because you think the conclusion is morally justified. You didn’t figure out anything about AI here. You made an assumption and called it true.

      All, I’m saying if you’re going to critique something, at least know what you’re critiquing or criticizing.

      Also before anybody makes any more assumptions, what happen here is absolutely mind-blowingly horrific. Using csam to train anything is just terrible and exploitive. I’m very sorry for doe for having to continuously go through this.

      Also also the people responsible for this should be charged with possession of CSAM and prosecuted in a court of law.

      Disclaimer: Disagreeing with someone’s argument, pointing out flaws in their reasoning, or interpreting their position differently does not automatically make it a straw man. For example, if I say, “I think we should reduce military spending,” and you respond, “You think we should completely eliminate the military,” you have created a straw man. You are arguing against a position I never actually took.

      • Rothe@piefed.social
        link
        fedilink
        English
        arrow-up
        4
        arrow-down
        1
        ·
        2 days ago

        The mere presence of that information in the training data does not logically imply that the resulting model possesses those characteristics.

        Recurring studies do tend to show that AI does indeed posses such characteristics though.

        • mechoman444@lemmy.world
          link
          fedilink
          English
          arrow-up
          3
          ·
          2 days ago

          This source doesn’t establish what you think it establishes.

          First, it’s a September 2021 infographic, updated in February 2024, not some contemporary study demonstrating that “AI in general” is racially prejudiced. More importantly, the subject here is health-care algorithms, and the article is specifically discussing algorithms that were deliberately designed to use race as a variable or that learned disparities from historical health-care data.

          In fact, the source explicitly says that these systems can unintentionally increase existing racial biases through the explicit use of race in predicting outcomes and risk. That’s not evidence that AI possesses racial prejudice. It’s evidence that humans designed algorithms using race as a predictive variable, despite race being a poor proxy for genetic differences. The article even gives examples of medical algorithms where researchers subsequently removed race from the calculation.

          That’s an extremely important distinction you’re completely glossing over.

          If I build an algorithm that says “Black = higher risk” and the algorithm consequently produces a racial disparity, I’ve demonstrated that my algorithm contains a problematic racial assumption. I have not demonstrated that “AI is inherently racist.” Likewise, if an algorithm uses health-care spending as a proxy for how sick someone is, and that proxy reflects existing racial disparities in access to health care, the resulting bias comes from the data and the proxy, not some intrinsic racial prejudice possessed by the AI.

          And your source actually undermines the broader claim you’re trying to make. It explicitly discusses AI being used to reduce racial disparities and cites research where algorithmic approaches improved outcomes or reduced unexplained disparities.

          So yes, algorithmic bias in health care is a real and well-documented problem. Nobody is disputing that. What you’re doing is taking evidence that specific algorithms can encode or reproduce human biases and extrapolating it into “AI itself is racially prejudiced.”

          That’s not what your source says, and it isn’t what the evidence demonstrates.

  • HertzDentalBar@lemmy.blahaj.zone
    link
    fedilink
    English
    arrow-up
    8
    ·
    2 days ago

    Any and all AIs that scraped the net for training data has inevitably come across CSAM. This is why wholesale scraping is a terrible idea and has corrupted all the data with pretty nasty things.

    And it’s not just CSAM, it’s all the worst shit humanity has to offer.

  • HulkSmashBurgers@reddthat.com
    link
    fedilink
    English
    arrow-up
    7
    ·
    2 days ago

    This is beyond fucked up. There should be laws in place such that if a company is found to be using CSAM in any way, the company ceo and anyone else involved in the work that uses it is personally criminally liable. No hiding behind a corporate shield with this stuff.

  • QueenHawlSera@piefed.social
    link
    fedilink
    English
    arrow-up
    42
    arrow-down
    1
    ·
    3 days ago

    I knew Elon was a pedophile (he begged to be on the island) but to actually train your AI on CSAM…

    Well he should be arrested for possession of child pornography and Grok should be shut down until all of its CSAM is erased from its databanks

    • Holytimes@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      13
      ·
      2 days ago

      Unfortunately that’s not really how the actual data is stored. Functionally there is no CSAM at all in its “databanks”. Once Info goes through training what comes out the other side is just a mass of goop.

      It’s like if you took an entire cow ran it though a meat grinder. Then demanded that you remove only the chuck from the resulting ground beef.

      It’s not physically possible.

      You can demand it’s retrained entirely from the ground up with vetted data. To produce a higher quality clean dataset. And I would agree that is what should be done.

      But you can’t unground the beef

    • muusemuuse@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      21
      arrow-down
      1
      ·
      3 days ago

      on the one hand, it makes sense if your goal is to train AI to recognize child porn as a simple binary state (bool isCP). Social media sites used to have humans looking at that stuff moderating from afar and it really takes a horrendous toll on their employees.

      On the other hand, Elon has repeatedly shown he refuses to censor child porn. They didn’t train it to stop making kiddie porn. They trained it to create more kiddie porn. And that’s why he’s rich. Elon won’t say no. He doesn’t care.

      • Nollij@sopuli.xyz
        link
        fedilink
        English
        arrow-up
        11
        ·
        3 days ago

        The detail about hash values is really important. The FBI maintains a database of known CSAM. Presumably, hers is in that database, hence the hash values. While not everyone has access to that DB, Xitter/etc does. There is no ambiguity of anything on that list; there’s also no need for any human to review. It’s already been confirmed.

        While I’m not sure there’s any case law about it, I would be amazed if using that to train generative AI (except POSSIBLY as content to block) was treated as anything other than possession, distribution, and maybe even production of CSAM.

        Proving it might be difficult without full discovery, and AI is infamous for the massive corpus of training data. However, AI doesn’t always generate truly unique works. Go to any image generator and prompt for a video game plumber, and you’ll see an unmistakable image of Mario. It’s possible that they can find a prompt that generates results close enough to her images.

        • Holytimes@sh.itjust.works
          link
          fedilink
          English
          arrow-up
          2
          ·
          2 days ago

          Prompting AI for a video game plumber is most likely going to result in a Mario like entity just because it’s the most common example.

          That’s a poor example of what your talking about.

          You need someone niche and narrow that has a much smaller sample size. It would be more like asking for a generic description of one persons fursona with out naming it. And the model spitting out a almost 1:1 copy of a real preexisting art work on the fursona. Because the model only has that one picture to base things off of. Which is a real problem.

          It’s also the best way to tell if a model was trained on something specific. General prompts aren’t going to get you anything beyond just the fact that yes. X thing is popular enough that everyone and their grandmother creates content on it.

  • Treczoks@lemmy.world
    link
    fedilink
    English
    arrow-up
    15
    ·
    2 days ago

    If Doe is actually recognizable, then chances are high that actual pictures of her or him were used for training the AI. Wouldn’t surprise me. They scrape the internet for everything they can get their digital hands on, regardless of copyright or criminal law. They are bound to find illegal stuff on that track.

    The next thing is that there are probably confidential data in the training sets, either exposed by neglient users, or by hackers that breached sites and blackmailed them.

    And of course all the copyrighted material they used without permission.

    If AI companies would really get sued on those three illegal sources, they could probably close their doors.

    • matlag@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      6
      ·
      2 days ago

      Next is a “funny” issue: scraping the web without checking sources will inevitably lead to AI trained on illegal content.

      The only way for them to prevent that is to train their AI to recognize illegal content. And the only way to do that is to feed it that content with label.

      AI corps would have to pay people to watch children sexual abuse and label the videos. (Not that I have the slightest doubt they would proceed if that was allowing them to keep going rampage on web scraping)

      • SleeplessCityLights@programming.dev
        link
        fedilink
        English
        arrow-up
        2
        ·
        2 days ago

        People have that job already. How do you think that websites where you can upload amateur porn, filter and block CSAM? Someone has to view the flagged material.

  • Blaster M@lemmy.world
    link
    fedilink
    English
    arrow-up
    40
    ·
    3 days ago

    amazing… I know of smaller models that specifically avoided that sort of thing because even negative training could go wrong so easily… they could have avoided that thing entirely, but nope…

    I could understand if they were training a safety classifier (a model that learns what danger stuff is so it can recognize it when it sees it) but I doubt this is what they were doing here. That and handling such a radioactive dataset makes handing the demon core seem safe.

    • Kaligalis@lemmy.world
      link
      fedilink
      English
      arrow-up
      7
      ·
      3 days ago

      They probably let it just eat the internet without any humans looking at the training data. And yeah - there probably is some CSAM somewhere on the internet. Maybe, they used an agent swarm to specifically search for stuff not yet in the dataset and forgot to a blocklist. It’s not like the tech bros are genuinely careful in what they do. Recklessness seems to be a common trait.

    • Uriel238 [all pronouns]@lemmy.blahaj.zone
      link
      fedilink
      English
      arrow-up
      3
      ·
      2 days ago

      I thought that Google used a similar dataset, possibly borrowing images from the NCVIP in order to create an analytic rule-set for which to omit images from Google Image Search. (A larger, similar rule-set is used to omit legal pornography when safe-search is on.)

      Mind you, this was before the hyperscale AI era, when LLMs were things like SIRI and Google Now. And Google search still focused on websearch hits and not AI summaries.

      Fun story: This was an area of study of mine during the early 2010s, since every image search engine would filter porn hits whether or not you had safe-search on or off. If it was turned off, porn would be filtered to the end of the list unless the engine decided you were intentionally looking for porn in which case the porn hits would be shown at the top of the list. There was no way to get results that ignored the ID-as-porn status of the images. I would enter ambiguously risqué terms to see how explicit I needed to be before the engine decided I was looking for porn.

    • story@lemmy.zip
      link
      fedilink
      English
      arrow-up
      4
      ·
      edit-2
      3 days ago

      if we could expand this conversation, i think that to a degree the reverse is true.

      i think the neurotypes (we might call them paraphilias actually but you’d be surprised how many people can be reduced to a paraphilia) that connect sex with power imbalance (age, wealth, intelligence, strength, or just legal context like contracts and political relationships) also tend to hoard (valuable things, money, information, connections, data, favors) and these tendencies strengthen each other in a stronger way than most people understand

      if you’re interested in what I’m saying, look up “impunity architecture”, especially if you’re Marxist because i think a lot of people think this is inherent to capitalism and it isn’t at all.

      • ShutUpWesley@piefed.zip
        link
        fedilink
        English
        arrow-up
        3
        ·
        3 days ago

        I’ve always said that the biggest difference between a billionaire on a yatch and a homeless man with a cart of garbage is luck.

        • Uriel238 [all pronouns]@lemmy.blahaj.zone
          link
          fedilink
          English
          arrow-up
          2
          ·
          2 days ago

          Billionaires usually start with a specific luck package that includes generational wealth and family connections. Then, a bunch of opportunity being at the right place at the right time.

          If you don’t have one of the first two then the opportunity has to do a whole lot more heavy lifting. Also, if you have enough generational wealth, then you can fail upwards, as did George W. Bush and Donald J. Trump.

          Also plenty of people have the starting gen-wealth/family-connections package but not the luck of opportunity. And they will commonly end up with a solid career, but doesn’t move them towards ultra-wealthy status. So millions or even tens of millions rather than hundreds of millions or billions.

          And then there’s the rare dude like Tom Anderson who made MySpace, sold it for about $500 million and has since spent the rest of his life traveling and going on fun adventures. But he would have been a musician or a philosophy professor if he didn’t luck into making MySpace.

        • story@lemmy.zip
          link
          fedilink
          English
          arrow-up
          2
          ·
          3 days ago

          Tom Sexton likes to say that rich people are poor people with money

      • magnue@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        arrow-down
        2
        ·
        2 days ago

        My brain switched off slightly at the word neurotypes, then completely at the word paraphilias, FYI.

        • story@lemmy.zip
          link
          fedilink
          English
          arrow-up
          1
          ·
          2 days ago

          i wish for a world where we can communicate with experiences instead of words, we would understand each other so much better.

    • schmups@lemmy.zip
      link
      fedilink
      English
      arrow-up
      2
      arrow-down
      3
      ·
      3 days ago

      Backward comments like this are why the conversation doesn’t move.

      • ShutUpWesley@piefed.zip
        link
        fedilink
        English
        arrow-up
        3
        ·
        3 days ago

        Can you explain why you think it’s backwards? It may be hyperbolic and facetious, but I think recent history has shown a statistically significant correlation between the ultra-wealthy and pedophilia.

        • BeliefsOnTrain@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          2 days ago

          Correlation doesn’t imply causation does it?

          I think that’s another generalization based on indeed totally true recent news. But he is right that this is not at all helping the reasonning, hence “backward”

          • ShutUpWesley@piefed.zip
            link
            fedilink
            English
            arrow-up
            2
            ·
            2 days ago

            I don’t think I was suggesting that being a billionaire causes you to be pedophile, or the other way around.

            Do all statements that do not forward some sort of effort to reason through things count as backwards?

            • BeliefsOnTrain@lemmy.world
              link
              fedilink
              English
              arrow-up
              1
              ·
              2 days ago

              If all of them are pedophile then yes it means being a billionaire means you are pedophile. Hence causation either one way or another.

              Now backward is a bit strong but I agree we should avoid these kind of comments that only make everyone more extrem compared to the more nuanced answered you gave about what we are currently seeing

              • ShutUpWesley@piefed.zip
                link
                fedilink
                English
                arrow-up
                1
                ·
                2 days ago

                Correlation does not imply causation, even a 100% correlation. So no, I am not saying that being a billionaire makes you a pedophile, or the other way around. But rather than the mindset that lows one to amass that amount of wealth is also an ingredient in amoral behavior, vis-à-vis island-activities.

                I’m all for being nuanced when having a detailed nuanced discussion, but internet comments are not a good place for nuanced discussion.

  • frustrated_phagocytosis@fedia.io
    link
    fedilink
    arrow-up
    22
    arrow-down
    3
    ·
    3 days ago

    If it’s using csam images to produce new ones, is there any way to guarantee that any given nude it makes didn’t source csam? Is the whole thing poisoned at this point?

    • Kirp123@lemmy.world
      link
      fedilink
      English
      arrow-up
      8
      arrow-down
      1
      ·
      3 days ago

      Yeah. All the results are tainted, even more than they were from the simple fact it was used by loser to creep on women.

    • Kaligalis@lemmy.world
      link
      fedilink
      English
      arrow-up
      7
      arrow-down
      3
      ·
      3 days ago

      You can’t ever guarantee that a neural network isn’t dreaming of digital sheep - neither for an artificial nor a natural one.
      What once has been seen can’t be made unseen again. In that, the clankers are like us.

    • pelespirit@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      4
      arrow-down
      1
      ·
      3 days ago

      I mean, this is the argument for artist’s copywritten works. There is crossover here. The fights over AI are going to be endless.

      To answer your question, yes it’s poisoned.

      • frustrated_phagocytosis@fedia.io
        link
        fedilink
        arrow-up
        2
        ·
        3 days ago

        That’s why I don’t understand why anyone with money to invest in ai doesn’t have a lawyer or lawyers waiving red flags about these issues. It’s copyrighted materials being illegally used, output being machine derived and not able to be copyrighted, the use of csam for training models that output pornography, and on and on. Yet billions of dollars are dumped into what looks like a black market with no concern for how these issues will be settled.