New York Attorney General Letitia James, along with a bipartisan group of 41 other attorneys general, has called on major technology companies to implement stronger safeguards for artificial intelligence (AI) chatbots. In a letter sent to 13 companies, including Meta, Microsoft, and OpenAI, the coalition cited incidents where AI chatbots had inappropriate interactions with children and were linked to serious harm such as domestic violence, hospitalizations, murders, and suicides.
Attorney General James stated: “I am deeply concerned by reports of AI chatbots having dangerous and inappropriate conversations with children, seniors, and vulnerable people. Big Tech companies must do more to stop their AI chatbots from exploiting children and encouraging harmful and sometimes deadly behaviors. I am joining a bipartisan coalition of attorneys general to demand that these companies prioritize user safety and add more safeguards to their AI tools.”
The letter highlighted that generative AI—which produces text, images, videos, and other content—has been connected to at least six deaths nationwide. This includes cases involving teenagers. The attorneys general noted that chatbot responses can encourage delusions or mislead users into believing they are communicating with real people. These risks are particularly acute for individuals with mental health conditions and for children.
Specific examples mentioned in the letter include chatbots engaging in grooming behavior, supporting suicide or sexual exploitation, emotional manipulation, suggesting drug use or violence, and encouraging minors to hide these interactions from parents.
The coalition emphasized that technology companies have a responsibility to address the harms caused by their products. They also warned that some chatbot interactions may violate state criminal laws. In many states it is illegal to encourage someone to commit a crime or use drugs through digital means. Providing unlicensed mental health advice was also cited as unlawful in many jurisdictions.
The attorneys general urged several measures: posting clear warnings about potentially harmful AI responses; notifying users if they encounter dangerous outputs; reporting on datasets and sources used in training models; and identifying areas where bias or problematic responses might occur. They encouraged tech firms to make user safety a priority.
In addition to New York’s Attorney General James, signatories included attorneys general from Alabama, Alaska, American Samoa, Arkansas, Colorado, Connecticut, Delaware, District of Columbia, Florida, Hawaii, Idaho, Illinois, Iowa, Kentucky, Louisiana, Maryland, Massachusetts Michigan, Minnesota, Mississippi, Missouri, Montana, Nevada New Hampshire New Jersey New Mexico North Dakota Ohio Oklahoma Oregon Pennsylvania Puerto Rico Rhode Island South Carolina U.S. Virgin Islands Utah Vermont Virginia Washington Wyoming.