Introduction
The use of Artificial Intelligence (AI) in personnel certification/certification of persons has the potential to revolutionize the way certification bodies (CB) assess competence. AI can not only streamline administrative tasks but also help in the job task analysis as well as exam development and administration. In addition, AI simulations for practical examinations can help assess competence in a manner that is not possible using traditional multiple-choice examinations. However, the increasing use of AI calls for research regarding its use.
Methodology
This study aims to understand the extent of use of AI among certification bodies accredited by the ANSI National Accreditation Board (ANAB) under ISO/IEC 17024. It is a preliminary first effort to gather data on adoption, implementation, and challenges. ANAB accredited clients attending the 2024 Credentialing Conference were asked to respond to four research questions.
- Does your certification body use AI in conducting certification activities?
- If yes, what percentage of certification activities are carried out using AI?
- What types of certification activities are carried out using AI?
- What are the main challenges facing your certification body in using AI:
Findings
A total of 31 certification body representatives who attend ANAB client day responded to the questions. The key insights from the data are as follows:
- Roughly 50% of CBs reported using AI in some capacity.
- Of those using AI, 75% indicated that they use it for less than 5% of their activities, showing that AI adoption is still minimal for many.
- 70% of CBs utilize AI for administrative tasks, which is the most common application.
- Other areas where AI is applied include item writing (40%), Job Task Analysis (JTA) (20%), and remote proctoring (20%).
Challenges in the Use of AI
The main challenges reported by the surveyed CBs preventing the use of AI could be grouped under the following.
- Test security, copyright protection, and concerns about information being fed into AI as training data
- Accuracy
- Lack of knowledge about how to use AI
- Acceptance by ANAB
Conclusion
The findings of this survey indicate that while AI is being introduced in certification of persons, its use is still relatively narrow, focused primarily on administrative activities with some additional specialized applications. It also suggests that the CB are cautious and aware of potential risks associated with the use of AI. Again, these findings are limited by the relatively small number of respondents and may not reflect the overarching trend in the certification world, particularly with other non-ANAB accredited CBs. However, these findings provide a need for clear guidance and requirements on the use of AI. It must be noted that ISO/IEC 17024 is currently undergoing revision. The draft standard includes a new section on the use of AI in certification of persons.