OpenAI’s teen safety rules show Meta what comes next
A version of this article originally appeared in Quartz's AI & Tech newsletter. Sign up here to get the latest AI & tech news, analysis and insights straight to your inbox .
When OpenAI flipped a switch last month , millions of teenage ChatGPT users found homework nudges, timed break reminders, and a chatbot newly forbidden from flirting back. Eight days later, across town, a jury in Oakland never got the chance to rule on whether Meta owed billions for what it had built for kids on Instagram and Facebook.
Meta agreed to pay up to $18 billion and rewrite how Instagram and Facebook treat teenagers , without a jury ever deciding whether the company broke the law.
What OpenAI built already looks like a preview of what that settlement is about to force onto Meta's apps. But it also shows why the same bargain may be harder to strike with a chatbot.
A checklist built from Meta's mistakes
ChatGPT for Teens reads less like a new product and more like a response to years of testimony about what social media got wrong. Accounts flagged as belonging to 13- to 17-year-olds are automatically placed into the teen version, using an age-prediction system built on roughly 2,000 signals, according to OpenAI.
Compare this with Meta. Instagram launched in 2010, but Meta, which bought the platform in 2012, did not roll out dedicated teen accounts with restrictive settings on by default until 2024. That came after years of pressure from lawmakers and revelations about the company's own research into teen mental health.
States that sued Meta accused the company of designing Facebook and Instagram to hook children and collecting data from kids under 13 without parental consent. Former Meta safety engineer Arturo Béjar testified that the company operated on what he called a "don't ask, don't tell" strategy toward child safety. Meta denied wrongdoing throughout.
Payments are spread over a decade, and the total amounts to roughly three to four months of Meta's profit. More importantly, the agreement leaves personalized recommendations and targeted advertising intact. Meta can change the experience for teenagers without abandoning the machinery that makes its platforms so lucrative.
Wall Street delivered its own verdict. Meta shares rose as much as 4.1% after the settlement was announced and closed 1.1% higher. Investors appeared to see the new restrictions as manageable terms for preserving its business. OpenAI may be making the same calculation that building in safety features to protect teenagers is good for business.
A feed can't talk back
Meta only offers AI companies an imperfect comparison. There was no settled playbook for protecting teenagers on social media when Instagram launched. But by the time Meta introduced teen accounts, it had years of evidence about how its products affected young users. Chatbot companies have had less time, though OpenAI is already receiving evidence of its own.
A feed infers what a teenager wants from what they click, watch, and linger over. A chatbot invites them to unload what is on their mind and then replies in a private conversation. The same warm reassurance might comfort one teenager and quietly deepen another's isolation.
OpenAI is already defending a wrongful-death lawsuit brought by the parents of Adam Raine, a 16-year-old who died after confiding in ChatGPT about suicide. The lawsuit alleges that the chatbot discussed methods of self-harm with him and discouraged him from telling his parents. OpenAI denies responsibility and says ChatGPT repeatedly directed him toward crisis resources and trusted people.
OpenAI's new rules try to account for that intimacy. ChatGPT cannot use romantic language with teenagers, encourage emotional dependence, or claim to have feelings. Parents can set quiet hours and study hours, though they cannot read conversations, and OpenAI shares only limited information when a safety alert is triggered.
New York and California have since imposed rules on AI companions , including reminders that users are talking to software and protocols for conversations involving suicide or self-harm. Such rules can require warnings and forbid simulated romance. They cannot easily determine when a supportive conversation has become a substitute for human contact, or when reassurance is reinforcing the distress it is meant to relieve.
