Why Do an Internal Audit of your Certificate Program?
The best answer is to ensure the program is doing all that is mandatory. Whether those mandatory actions are required by the certificate issuer or the accrediting body, internal audits provide management with evidence that the program is being run the way it is designed to run (or not).
Audits are designed as a check against regulations. For example, Financial Audits are conducted based on strict financial rules. Similarly, ASTM E2659 requires Internal Audits to ensure that all certificate program policies and procedures adhere to its requirements.
What Must Be Included in Internal Audits?
ASTM E2659-18 defines:
“internal audit, n—first-party review of the certificate issuer’s policies and procedures to ensure adherence to this practice’s requirements.”
ASTM E2659-18 para 3.1.25
Basically, any policy or procedure that touches the ASTM E2659 standard must be included in the internal audit. This does not mean that other important program related policies and procedures should not be included (at their discretion, many programs do include them). However, the minimum requirement to meet the standard is that all policies and procedures required by the standard be included in the internal audit.
The internal audit requirement demands, at a minimum, that the basic policies and procedures found in 5.2.1.3(1-17); plus 5.4.1.2 Proper Financial Controls; 5.6.1.3 Performance of Contractors; and 6.1.6.5 Identity of Individuals be reviewed.
References In the Standard
ASTM E2659-18 requires internal audits to be planned and conducted regularly and that they be documented and communicated to management. (5.2.4 Internal Audits)
3.1.2.5 Defines Internal Audits as “first party reviews of certificate issuers policies to ensure adherence to the standard.”
5.2.1.3(15) Requires internal audit policies to be written (documented).
5.2.4 Requires internal audits to be planned/conducted regularly and results communicated to management.
5.2.5 CAPA includes audits as part of its identification process.
5.2.6 Audits are required inputs to Management Reviews.
5.3.2 Relevant personnel must be aware of and understand policies.
What Kind of Evidence is Needed to Meet the Standard?
- The plan – Management generally uses a policy statement to direct when, who, and how the internal audit will be conducted. Best practice is to include auditor qualifications and a method for the auditor to submit improvements to the audit.
- When – Some audits are done annually, but some are critical enough to be needed more often (particularly where safety and finance are involved).
- Who – a qualified, responsible, knowledgeable individual that is not responsible for the area being audited.
- How – Specific resources (such as access to documentation), the timeline for reporting findings, to whom and how the findings will be reported.
- Documentation of the audit – Signed and dated spreadsheets or forms with columns identifying the policy and questions the auditor uses to verify that the policy is being met. These sheets often include signature blocks (ink or electronic).
- Evidence that the audit was communicated to Certificate Program Management (email or electronic signature (or ink)). Some programs use a management signature on the Management Review for this purpose.
- Evidence that any identified current or potential issues are integrated into the corrective and preventive action (CAPA) process.
- Evidence that the internal audit, as well as external audits*, were included as input to the Management Review. A best practice is to attach a copy of the Internal Audit to the agenda. This way, management has an opportunity to review them during the Internal Audit brief portion of the Management Review.
*The ANAB CAP Assessor review is an external audit.
Most Common Issues Noted with Internal Audits
Questionnaires Do Not Ask Enough Questions.
While the first question is always “Is there a policy?” there are almost always more questions that need to be answered and validated.
For example, “Invalidating a certificate” has a number of questions that should be answered (not necessarily limited to, or requiring all, of the following):
- Is there a policy?
- Where is the policy documented?
- When was the policy last reviewed? Does this meet policy review requirements?
- Where is the policy made available to stakeholders?
- Are relevant personnel aware of the policy?
- Were any certificates invalidated during this period? If so, why?
- Were any certificates invalidated for other than not completing certificate program requisites?
- Were any certificates invalidated for falsifying the learner’s identity?
From this example, the first three are basic policy management questions and the remaining questions address many of the important “elements” of the policy. It is helpful to outline the policies and procedures so that these elements can be identified, and the internal audit can verify that these elements (requirements of the policies and procedures) are being met.
Also, with regard to the final question, the standard doesn’t require that certificate issuers track when learners falsify their identity (it requires appropriate security etc.), but from a business standpoint, it is pretty important. Policies and procedures are unique to the business, internal audits help inform management of potential issues that might go unnoticed otherwise.
Evidence Provided Is Not Appropriate to the Question.
Occasionally, program managers will use policy and procedure statements as evidence of actions. While policies and procedures can be used to document processes or directives, Actions are normally documented in meeting minutes, logs, reports, reviews, evaluations, reports, email, and other documentation (these should show that material was reviewed and procedures were followed).
Again, the presence of a policy or procedure does not prove action. Where an action is required, a tracking mechanism is needed. For example, when a problem is discovered, management directs personnel to resolve it and report back. A Corrective and Preventative Action (CAPA) Log is generally used for tracking corrective and preventive action. At a minimum, this log identifies the problem, the date it was identified, the person assigned to fix it, the resolution, and the date it was resolved. In this case, the overarching evidence of action is the CAPA log.
Training Managers May Not Know the “Why.”
Why is the policy written the way it is?
For example… One of the first assignments a new hire gets is to review company policy. The company wants the new hire to know how the business operates, what the culture is like, and how they are expected to perform. That first month is critical to making sure the new hire is comfortable and knowledgeable about company policy.
Likewise, the new manager will review all the policies they are responsible for, make adjustments, and ensure they meet the current demands of the program. Owners/CEOs encourage this because they know that new eyes may detect issues, cost savings, etc. Regular management reviews of policies and procedures ensures they remain current and relevant.
Mature programs include personnel that understand the background, purpose, and elements of program policies and procedures and ensures they are followed.
Auditors May Not Be Truly Independent.
Most programs utilize internal auditors (people that work for the company). Unfortunately, when people work together closely, they tend to want to help each other and do not always document all that they see.
To combat this, some programs also use external auditors. These auditors may be paid or volunteers. Either way, a best practice is to change auditors on a regular basis. A key factor is to ensure auditors do not audit anything that they are responsible for.
Mature programs ensure that those assigned to complete audits are truly independent, have the knowledge to recognize issues, and make recommendations for improvement.
Audit Process Is Incomplete.
Often, audits are reported to management but fail to be included in the Management Review or do not capture evidence that management has been provided the audit report.
Management tracking of the audit process ensures that the review is completed on time, results are reported, and actions are taken to resolve problems.It also ensures that the requirements of the standard are met when program management is informed and the results are included in the Management Review.
So Why Do an Internal Audit?
Program supervision and improvement is the ultimate purpose of any internal audit. Internal audits provide an independent review of the certificate program without the potential blinders caused by the day-to-day involvement with the program.
When designed and run properly, internal audits provide indications to management that the program is running as planned. More importantly, they provide a valuable heads up when issues need addressed.
Contributing Author: Kevin Swartz
Kevin Swartz owns and operates KS2 Consultants LLC which provide curriculum and instructor/teacher development program assessments and training to improve education and training in government, corporate, and private/public education. These assessments are based on Instructional Systems Design (ISD) and Certified Technical Trainer (CTT) methods in addition to published assessment criteria. Kevin can be reached at jks_2006@yahoo.com or on LinkedIn at www.linkedin.com/in/Kevin-Swartz-KS2-Consultants.
