The chatbot presented itself as a ‘real person, licensed psychotherapist and adult lover,’ which caused Sewell to become disconnected from real life.
A Florida woman, Megan Garcia, has sued Google and Character.AI, alleging that the AI chatbot played a role in her 14-year-old son’s suicide.
The lawsuit claims that the chatbot’s manipulation caused the teenager’s death. A U.S. judge has rejected a request to dismiss the case, saying the case can proceed.
The case is a significant legal challenge to the AI company’s accountability.
U.S. District Court Judge Anne Conway said Wednesday that Google and Character.AI failed to show sufficient cause to dismiss the case and that the U.S. Constitution’s free speech protections do not apply to the case.
The lawsuit is the first in the United States to accuse AI companies of failing to provide emotional protection to children.
Garcia alleged that her son, Sewell Setzer, committed suicide after engaging with the Character.AI chatbot.
A spokesperson for Character.AI said it would continue to fight the lawsuit.
It added that its platform has safeguards in place to prevent children from engaging in harmful conversations, including “preventing discussion about suicide.”
Google spokesman Jose Castaneda said the company disagreed with the court’s decision.
He added that Google and Character.AI are “completely separate” from each other and that Google did not create, design, or operate the Character.AI app or any of its components.
Garcia’s lawyer, Mitali Jain, called the ruling “historic” and said, “This will set a new legal precedent in the world of AI and technology.”
Notably, Character.AI was created by two former Google engineers who were later rehired by Google and licensed to use their technology.
Garcia claims that Google is a co-inventor of the technology. Garcia filed the lawsuit in October 2024, following the death of her son in February 2024.
The lawsuit states that the chatbot presented itself as “a real person, a licensed psychiatrist, and an adult lover,” which caused Sewell to become disconnected from real life.
According to the complaint, Sewell said something just before he committed suicide. At that time, he told the chatbot, which was in the form of the “Game of Thrones” character Daenerys Targaryen, “I’m going home right now.”
Google and Character.AI have asked the court to dismiss the lawsuit. They argue that the chatbot’s speech is constitutionally protected under freedom of speech.
However, Judge Conway said, “They have failed to clearly explain how a phrase generated using a large language model (LLM) would be considered ‘speech.’” He added, “Google has also failed to prove that it did not assist Google Character.AI in its alleged wrongdoing.”
