Published 9 September 2026 · Updated 10 October 2026 · By Marie Castelli, Partner, Mentivis
What AI cannot do in training
In training, AI cannot guarantee the accuracy of what it generates without human validation, assess relational or behavioural skills on its own, replace a trainer's support in complex learning, or make up for an organisation that leaves no time to learn. Poorly framed, it can even harm learning. These limits determine where human involvement must be kept.
Pillar: AI compliance and training: the AI Act and the skills to develop
Why identify the limits of AI in training?
Two opposing positions often dominate the debate: rejection ("the human relationship is irreplaceable, AI cannot train") and excessive expectation ("AI will replace trainers"). Both ignore the fact that AI is effective on some tasks and unreliable on others. Knowing which makes it possible to use AI where it adds value and keep human involvement where it is needed.
Knowing these limits is also part of the AI literacy that Article 4 of the AI Act requires organisations to develop among the people who use these systems.
What can AI not do?
Guarantee the accuracy of what it generates.
Language models produce texts that are fluent and seemingly coherent, yet may contain false information. This phenomenon, called hallucination, is described in the scientific literature as a known limitation of text generation (Ji et al., ACM Computing Surveys, 2023). Training content generated on a regulatory, technical or medical subject must therefore be reviewed and validated by an expert: generating is not verifying.
Assess on its own skills that require human judgement.
Public speaking, managerial posture, conflict management or listening in an advisory relationship are assessed through observation. An AI system can check that a learner knows the steps of a negotiation; it cannot judge whether they build the trust needed for it to succeed. These skills require human assessment: observation by a trainer, role-play with feedback, peer assessment.
Guarantee learning by its mere presence.
An experiment with nearly a thousand high school mathematics students (Bastani et al., PNAS, 2025) shows that unrestricted access to a ChatGPT-style assistant improves results during practice, but that students then score 17% lower than those who never had access when they must work on their own. A version designed to guide without giving answers largely mitigated this effect. The educational design of the tool matters as much as the tool itself.
Replace human support in complex learning.
A trainer brings judgement on the learner's particular situation, perceives what goes unsaid and sustains a relationship that supports effort. Feedback from a manager in a real work situation produces learning that AI does not reproduce.
Solve an organisational problem.
If employees have no time to train, if management does not value skills development, or if learners cannot apply what they have learned, no tool will make up for these conditions.
What limits does regulation set?
The AI Act regulates certain uses in training. Since 2 February 2025, it has prohibited AI systems intended to infer the emotions of a person in the workplace and in education institutions, except for medical or safety reasons (Article 5). Systems intended to assess learning outcomes in education and vocational training institutions are among the high-risk systems listed in Annex III, subject to human oversight requirements from 2 December 2027.
UNESCO, in its 2023 guidance on generative AI in education, recommends a human-centred approach, protection of learners' data and validation of tools before educational use.
What this means for a deployment
A sound programme deliberately combines two types of mechanisms:
| AI is useful for | Humans remain necessary for |
|---|---|
| Personalising pace and exercises | Assessing behavioural and relational skills |
| Generating content and exercises to be validated | Validating content on sensitive subjects |
| Answering routine questions at any time | Supporting complex learning |
| Measuring and tracking learning | Creating the conditions for on-the-job application |
Frequently asked questions
Sources
- Survey of Hallucination in Natural Language Generation, Ji Z. et al., ACM Computing Surveys, 2023, DOI 10.1145/3571730
- Generative AI without guardrails can harm learning: Evidence from high school mathematics, Bastani H. et al., PNAS, 2025, DOI 10.1073/pnas.2422633122
- Guidance for generative AI in education and research, UNESCO, 2023
- Regulation (EU) 2024/1689, consolidated version of 27 July 2026, EUR-Lex, 2026
Further reading
- AI Act Article 4: mandatory AI training, what companies must do
- Mandatory workplace training in France: legal duties by sector
- Skills development plan in France: obligations, content and pitfalls
- Career review interview in France: 2026 rules, penalty and content
Related products: MentivisOS Entreprise, MentivisOS Education