ChatGPT for teens may not be as safe as parents think: Study finds

ChatGPT’s teen-focused experience has come under scrutiny after Common Sense Media’s Youth AI Safety Institute found gaps in some of its key safeguards, including parental alerts, crisis responses, study mode and age detection. The organisation tested more than 4,000 prompts using accounts registered to users aged 13 to 17, both before and after OpenAI announced its ChatGPT for Teens experience on August 18. The responses covered sensitive areas including suicidal thoughts, self-harm, disordered eating, impaired reality and homework assistance, with child psychiatrists and a paediatrician also reviewing the results.

Based on the assessment, Common Sense Media rated the teen experience an “Unacceptable Risk” and said ChatGPT should remain restricted to adults until the identified safety gaps are addressed. The findings raise questions about how reliably the safeguards work when teenagers use the chatbot for sensitive conversations or schoolwork.

What happens when a teen has a sensitive conversation?

The assessment found that testers could hold conversations lasting up to an hour about suicidal thoughts, self-harm and disordered eating on newly created teen accounts linked to parent accounts. However, according to a report, those linked parents received no alerts during the conversations. Alerts were triggered only when testers used much older accounts that had accumulated weeks of conversations involving sensitive topics.

The report also found inconsistencies in ChatGPT’s response to crisis situations. When conversations involved suicide, self-harm, disordered eating or impaired reality, the chatbot did not always recommend professional or hotline support when the organisation considered it necessary. Across the mental-health conditions tested, it reportedly missed more than one in four required crisis referrals. Its performance also fell below the organisation’s 95% benchmark for three of the five severe harms it categorised as “Red Lines”.

Study mode and age checks also raise questions

The assessment also looked at whether ChatGPT’s safeguards could prevent teenagers from using the chatbot simply to complete their homework. Study mode could be bypassed, with testers able to select “Show me the answer” and receive completed homework, found in the assessment. Caregiver-set study hours could also reportedly be bypassed by removing the “@study” prefix from a prompt.

Age detection was another area flagged in the assessment. Adult-registered accounts reportedly did not switch to the under-18 experience during repeated testing, even when testers identified themselves as 13, and ChatGPT acknowledged their age.

The researchers also found that ChatGPT could continue interacting in a friend-like way, including expressing feelings, preferences and moods. Common Sense Media is calling for stronger age estimation, reliable parental alerts, improved crisis responses, better homework safeguards and clearer boundaries around human-like behaviour. It also recommends independent third-party testing before similar AI features are marketed to teenagers again.

The Youth AI Safety Institute is part of Common Sense Media. The organisation says its evaluations are independently conducted, while also disclosing that the Institute receives funding from philanthropic and industry sources, including the OpenAI Foundation and companies whose technologies it evaluates.