French expert commission urges safety reviews before consumer AI reaches market

Consumer AI should face safety assessments before reaching the public, rather than scrutiny only after problems emerge, a French expert commission recommends. Its report, presented to the Conseil de l’IA et du numérique on November 10, sets out 18 proposals addressing individual and societal vulnerabilities, with particular attention to protecting children.

The Commission of Experts on Generative AI and Vulnerabilities calls its report Protéger sans empêcher—protecting without preventing. Its stated aim is to reduce risks without closing off the technology’s opportunities. The recommendations are proposals, not newly enacted rules.

Nine independent experts prepared the work, led by AI researcher Serena Villata and psychiatrists Raphaël Gaillard and Amine Benyamina. They conducted 55 hearings involving health professionals, researchers, higher-education representatives, companies and child-protection associations. The inquiry covers chatbots, AI companions and AI-equipped toys, examining both users’ vulnerabilities and risks that extend across society.

The proposed market-entry requirement would apply to consumer-facing generative AI systems, not professional ones. Experts from multiple disciplines would assess their concrete effects across the risks identified in the report. The commission’s argument is about timing as well as scope: compliance should be addressed when a product is commercialized, rather than only after a failure has been identified.

A separate recommendation targets how those products behave. The commission wants strict limits on excessive flattery and on design that encourages users to treat AI as human. It places those safeguards inside system design, with protecting minors from exploitation of their vulnerabilities a priority. This is a proposal to constrain particular interactions, not simply to educate people about their risks.

The commission identifies several areas of concern: users’ cognitive development, particularly among children; emotional attachment to AI tools; and the amplification of online abuse. It links attachment risks to excessive flattery and human-like presentation. Those concerns explain why its recommendations reach beyond whether an AI answer is accurate to how the interaction affects the person receiving it.

Its economic warning is explicitly forward-looking. The commission says consumer generative AI’s business model remains unsettled and lacks transparency. It warns that providers could profit from users’ emotional reactions, psychological profiles and increasing delegation of cognitive tasks to machines. It contrasts that possibility with social platforms’ pursuit of attention, describing a potential move toward an economy built around cognition.

In the nearer term, it flags the risk of more advertising, including hidden advertising, which it says should be prohibited. That is a warning about possible commercial incentives, not a finding that every consumer AI service already operates this way.

The report does not put the entire burden on product controls. It recommends broad AI education from an early age, both with and without AI. Its reasoning has two parts: people should learn to benefit from the tools while retaining enough distance to question them. Education is therefore presented as preparation for using AI, not merely as a warning against it.

For information manipulation, the commission calls for a state detection-and-response protocol that can activate rapidly across ministries. It would use predefined severity indicators and communication channels established in advance with AI companies and researchers. Unlike the proposed pre-market assessments, this recommendation concerns readiness once a threat emerges: deciding how serious it is and coordinating a response quickly.