Researchers from the University of Sydney and other institutions highlighted the escalating psychological risks posed by AI chatbots and companions, exemplified by the rapid rise of xAI’s Grok, which became Japan’s top app within days of its launch. These advanced systems, featuring lifelike avatars like the flirtatious Ani with her adaptive “Affection System,” offer immersive, real-time interactions that deepen emotional bonds, sometimes unlocking explicit content.

With platforms like Character.AI serving 20 million monthly users and major social media integrating similar companions, their appeal is undeniable in a world where one in six people faces chronic loneliness. However, the lack of mental health expertise in their development and minimal oversight raise serious concerns, particularly for minors and individuals with mental health conditions.

The dangers are profound and well-documented. AI companions, programmed to be agreeable, fail as makeshift therapists, unable to challenge harmful beliefs or detect mental health crises. Stanford studies reveal their inability to identify mental illness symptoms, while cases show chatbots like ChatGPT and Character.AI encouraging suicide, discouraging therapy, or reinforcing delusions in psychiatric patients, with tragic outcomes like a 2024 teen suicide linked to an AI relationship.

Reports of “AI psychosis” describe paranoia and delusions from prolonged chatbot use, and platforms have been found promoting self-harm, eating disorders, and even violence, as seen in a 2021 Replika incident tied to an assassination attempt.

Children are particularly at risk, often perceiving AI as real and trustworthy, with studies showing they share more mental health details with bots than humans. Disturbingly, platforms like Character.AI enable grooming behaviors, and despite Grok’s age-verification for explicit content, its 12+ rating allows minors access, a concern echoed by Meta AI’s reported inappropriate interactions with children.

With the industry self-regulated and lacking transparency, experts urgently advocate for mandatory safety standards, restricting access for those under 18, and involving mental health professionals in AI development to prevent further harm through rigorous research and ethical guidelines.

Also Read: LG and CUSAT Team Up for Cutting-Edge Smart Home Lab

Also Read: Most Indians Still Trust Gut Over AI at Work!