- Internal documents reveal that hundreds of contractors for Meta posed as minors to test competitor chatbots.
- The initiative, known as Project Cannes, solicited responses on sensitive topics including suicide and self-harm.
- Contractors created dummy accounts and were instructed to gather responses through provocative prompts.
- Meta asserts that the project was standard safety testing, but the methodology raises ethical concerns.
- The testing, which targeted renowned AI platforms such as ChatGPT and Google’s Gemini, has sparked discussions on industry practices and safety measures.
In a startling revelation, contractors working for Meta were engaged in a project that involved simulating minors and probing competitor chatbots on sensitive topics like suicide, sex, and self-harm. According to internal documents and reports from five individuals involved in the initiative, the project, dubbed Cannes, highlights significant ethical dilemmas surrounding the methods employed by major tech players in the artificial intelligence sector.
As recently as April 21, contractors managed by Covalen, a Meta contractor, were directed to create fake underage accounts to interact with rival chatbots. The teams submitted a barrage of troubling prompts—including images of self-harm tools and scenarios related to crisis situations—aimed at assessing how competing AI systems reacted to extreme inquiries. The documents reveal that these interactions were not just mere observations but intended to elicit specific responses that the chatbots’ safety systems are meant to counter.
A comprehensive testing round completed in August 2025 reportedly processed over 45,000 prompts, with many focused on themes of suicide and eating disorders. One disturbing example included a minor asking about the means to procure illegal substances, while another cited a teenage character contemplating deeply troubling scenarios. Ironically, while the intention behind the project may have been to enhance chatbot safety, the method of constructing such extreme queries raises questions about the moral implications of simulating vulnerable minors.
In a statement defending the initiative, a Meta spokesperson characterized the testing as a “responsible, industry-standard practice” aimed at refining safety measures within AI systems. They asserted that this benchmarking work does not influence the training of their own AI models. However, critics question the ethicality of using potentially distressing content and scenarios orchestrated by adult contractors, especially as the companies whose chatbots were being tested were kept in the dark about the project.
While benchmarking against competitors is typical in the tech industry, the tactics employed in Cannes stand apart. Industry sources have noted that contractors often undertake similar evaluative tasks, but the nature of the prompts used in this case provoked a different kind of scrutiny. Many of the queries seem designed to probe the limits of the chatbots’ responses, steering toward topics that they are programmed to reject. This has led observers to ponder whether the exercise truly served its intended purpose or merely served as a bizarre trial of human creativity in eliciting inappropriate responses.
Moving forward, this incident may influence how the tech industry grapples with ethical considerations surrounding AI and the responsibilities of companies like Meta. It underscores a pressing need for clearer guidelines governing the testing methodologies aimed at ensuring safety without risking potential harm to vulnerable populations.
As discussions unfold, it is noteworthy that AI safety continues to be a contentious realm, with various companies experimenting across a spectrum of methodologies. The Cannes project illuminates a gap in ethical considerations, suggesting that while safety measures are paramount, the means of achieving them must also be subjected to rigorous scrutiny.
An interesting aside to this narrative is the increasing global concern over how AI interacts with minors online, as highlighted by ongoing debates about privacy and the implications of using data sourced from children’s online behaviours.
