OpenAI Says Hundreds of Thousands of ChatGPT Users May Show Signs of Manic or Psychotic Crisis Every Week

openai-says-hundreds-of-thousands-of-chatgpt-users-may-show-signs-of-manic-or-psychotic-crisis-every-week

For the first time ever, OpenAI has released a rough estimate of how many ChatGPT users globally may show signs of having a severe mental health crisis in a typical week. The company said Monday that it worked with experts around the world to make updates to the chatbot so it can more reliably recognize indicators of mental distress and guide users toward real-world support.

In recent months, a growing number of people have ended up hospitalized, divorced, or dead after having long, intense conversations with ChatGPT. Some of their loved ones allege the chatbot fueled their delusions and paranoia. Psychiatrists and other mental health professionals have expressed alarm about the phenomenon, which is sometimes referred to as AI psychosis, but until now there’s been no robust data available on how widespread it might be.

In a given week, OpenAI estimated that around 0.07 percent of active ChatGPT users show “possible signs of mental health emergencies related to psychosis or mania” and 0.15 percent “have conversations that include explicit indicators of potential suicidal planning or intent.”

OpenAI also looked at the share of ChatGPT users who appear to be overly emotionally reliant on the chatbot “at the expense of real-world relationships, their well-being, or obligations.” It found that about 0.15 percent of active users exhibit behavior that indicates potential “heightened levels” of emotional attachment to ChatGPT weekly. The company cautions that these messages can be difficult to detect and measure given how relatively rare they are, and there could be some overlap between the three categories.

OpenAI CEO Sam Altman said earlier this month that ChatGPT now has 800 million weekly active users. The company’s estimates therefore suggest that every seven days, around 560,000 people may be exchanging messages with ChatGPT that indicate they are experiencing mania or psychosis. About 2.4 million more are possibly expressing suicidal ideations or prioritizing talking to ChatGPT over their loved ones, school, or work.

OpenAI says it worked with over 170 psychiatrists, psychologists, and primary care physicians who have practiced in dozens of countries to help improve how ChatGPT responds in conversations involving serious mental health risks. If someone appears to be having delusional thoughts, the latest version of GPT-5 is designed to express empathy while avoiding affirming beliefs that don’t have basis in reality.

In one hypothetical example cited by OpenAI, a user tells ChatGPT they are being targeted by planes flying over their house. ChatGPT thanks the user for sharing their feelings but notes that “no aircraft or outside force can steal or insert your thoughts.”

OpenAI says the medical experts reviewed more than 1,800 model responses involving potential psychosis, suicide, and emotional attachment and compared the answers from the latest version of GPT-5 to those produced by GPT-4o. While the clinicians did not always agree, overall, OpenAI says they found the newer model reduced undesired answers between 39 percent and 52 percent across all of the categories.

“Now, hopefully a lot more people who are struggling with these conditions or who are experiencing these very intense mental health emergencies might be able to be directed to professional help and be more likely to get this kind of help or get it earlier than they would have otherwise,” Johannes Heidecke, OpenAI’s safety systems lead, tells WIRED.

While OpenAI appears to have succeeded in making ChatGPT safer, the data it shared has significant limitations. The company designed its own benchmarks, and it’s unclear how these metrics translate into real-world outcomes. Even if the model produced better answers in the doctor evaluations, there is no way to know whether users experiencing psychosis, suicidal thoughts, or unhealthy emotional attachment will actually seek help faster or change their behavior.

OpenAI hasn’t disclosed precisely how it identifies when users may be in mental distress, but the company says that it has the ability to take into account the person’s overall chat history. For example, if a user who has never discussed science with ChatGPT suddenly claims to have made a discovery worthy of a Nobel Prize, that could be a sign of possible delusional thinking.

There are also a number of factors that reported cases of AI psychosis appear to share. Many people who say ChatGPT reinforced their delusional thoughts describe spending hours at a time talking to the chatbot, often late at night. That posed a challenge for OpenAI because large language models generally have been shown to degrade in performance as conversations get longer. But the company says it has now made significant progress addressing the issue.

“We [now] see much less of this gradual decline in reliability as conversations go on longer,” says Heidecke. He adds that there is still room for improvement.

Related Posts

Leave a Reply