What exactly is meant by the term “Instructional design model” as referenced in ASTM E2659-18 section 6.1.1 and what would be considered evidence of compliance under this section?
For decades, professional training industry members have researched and developed comprehensive educational solutions to myriad problems, ranging from those of individual organizations to the general human population. The formal systematic ways of developing and implementing these educational solutions are known collectively as instructional design (ID) models.
What is an Instructional Design Model?
A formal instructional design model is a comprehensive approach to a given problem in the sense that it begins with the identification of an organizational, societal, or general problem or need, requires an analysis of the need to identify root causes, and then determines whether the need can be met with an educational or training program. If it is determined that an educational solution will meet the need, the model continues with the design, development, implementation, and evaluation of an educational intervention intended to solve the original problem.
Even after the program is implemented, final examinations are passed and certificates are issued, the ID model requires follow-up evaluations to demonstrate whether the course achieved (and continues to achieve) its purpose. This entire process (from need assessment through ongoing evaluation) is collectively known as the instructional design model, and such models are specifically what ASTM E2659-18 requires for program accreditation.
Since new techniques and methodologies are constantly being developed, ASTM E2659-18 does not specify that any particular ID model must be used, but it does require that an industry accepted model has been identified and followed. Generally, for an ID model to be considered acceptable under ASTM E2659-18, it must have been previously published in at least one credible educational and training industry publication.
What is the ADDIE Model?
To date, the ID model most commonly used by certificate programs is known by its acronym “ADDIE,” which stands for:
- Analyze
- Design
- Develop
- Implement
- Evaluate
The ADDIE model was originally developed at Florida State University for the U.S Military in the 1970s to guide its training programs. While certainly not the exclusive ID model for certificate programs, it remains the most widely used and recognized by accredited certificate programs. You may have noticed that the initial application for ANAB Certificate Accreditation Program (CAP) accreditation actually incorporates ADDIE language. Whichever ID model you chose as the blueprint for your organization’s certificate program, ASTM E2659-18 requires that its steps and processes be properly followed and documented as evidence that the model did in fact guide program development.
The Five Components of the ADDIE Model
Because ADDIE’s five components are typical (in some form) of all ID models, it may be helpful to review the following brief description of each stage:
ANALYZE (assess) the need: The use of the term “needs assessment” is quite specific in ASTM E2659-18 and often is a source of confusion for certificate program accreditation applicants. When a need (or problem) is identified as described above, usually the next best step is to look for its root causes to fully understand and define the problem in an effort to determine a solution. This process of analyzing the need in order to determine its causes and potential solutions constitutes the needs assessment (analysis) phase of the instructional design plan. Once the need has been thoroughly analyzed and well defined, the instructional design process moves ahead to solve the defined problem. It is also often during needs assessment that the people who will benefit from the educational program are identified, and these groups become the program’s primary stakeholders, which is another useful term to know as you complete the CAP application.
Notice that ASTM E2659-18 Section 6.1.2 (1) includes the phrase “instructional design model and processes” that the certificate issuer (CI) uses to identify learner needs. The term “process” here refers to a variety of techniques commonly used by instructional designers to formally identify needs. Such industry accepted techniques include a relevant professional literature review, interviews, focus groups, market surveys, a job-task analysis, observation, and experimentation, or some combination of these techniques. Accreditation requires that whatever analytical technique is used, the process must be described and documented. Thorough documentation of the methodology and processes used to identify the need and its root causes will serve as the evidence that the certificate issuer has met this portion of ASTM E2659-18’s requirements.
DESIGN: The design part of the ID process centers around the translation of the fully analyzed need into clearly stated desired outcomes for learners after their successful completion of the instructional program. Along with the stated desired outcomes, some specific pieces of the program are clarified in the design phase, some of which are expressly referenced in ASTM E2659-18. This includes as the identification of targeted audience for instruction, the certificate term of validity, and the instructional program’s purpose, goals, scope, requisites, prerequisites. During the design phase, clarity is gained regarding what the course is intended to achieve (and not achieve), what (if any) previous knowledge, skills, ability, experience, etc., will be required before taking the course, what will constitute successful course completion, and how long the learning will be considered valid (3 years, 5 years, for life, etc.)
The key task in the design phase is the determination and formal statement of the program’s “learning outcomes” or “learning objectives” (LOs). In fact, clearly defined and well stated LOs are among the most important and essential building blocks of accredited certificate programs. Simply put, LOs are the intended specific and demonstrable competencies expected of those who have successfully completed the educational program. LOs are formal statements of the competencies, skills, and knowledge deemed essential to meeting the originally identified market training need. They describe what the successfully trained learner will be able to do as a result of training. Well written LOs are complete statements that describe the successful program graduate in a tangible way; thus, LOs are written with verbs that can be verified through some form of testing, often including words like “demonstrate,” “distinguish,” or “solve,” as opposed to more general and internal processes like “understand” or “be aware of.”
DEVELOP: Course development is really about the “build out” of the educational program and the final assessment (exam). There is a very strong connection here to the design phase in that after the learning objectives are determined as described above, both the course material and the summative (final) exam are centered around the LOs. The LOs drive the curriculum, and the final exam is intended to measure and verify learner achievement of the LOs. As with the other elements of the ID model, it is expected that industry accepted (and previously published) learning theories and principles will guide the specific features of program development.
It is in this phase that the design plans come to life in the form of videos, curriculum content, webpages, case studies, etc. These tools are developed specifically to aid learners in achieving the LOs. The final assessment (a criterion-referenced test) is created in this phase as well, and it is developed solely to ensure that the LOs have been achieved. The type of exam, the number of exam items (questions), test delivery system, and scoring are all established during the development phase of the ID process with the guiding LOs in mind.
IMPLEMENT: As the term suggests, implementation includes the actual delivery of the course, the end of course exam, the scoring of the exam, and the granting of earned certificates. Delivery of course materials can occur in a variety of ways, including in-person instruction or online presentation. A key consideration in ASTM E2659-18 is the emphasis on the consistency in delivering educational content, which refers both to multiple presentations (often by different presenters) and consistency between different delivery forms such as online and in-person learning. How does the certificate issuer ensure that each educational presentation is consistent even when presented in different ways, at different times, and by different presenters?
One of the key ways certificate issuers ensure consistency is with established course policies and procedures for instructor training and calibration (if appropriate), as well as specific guidelines for in-person learning events. The same is true for exam administration, so policies and procedures regarding exam security, test-taker identification, and related issues should be considered, documented, and carefully followed. As previously stated, all learning must be focused and ultimately evaluated based on whether or not LOs have been achieved. While the subject of comprehensive program evaluation is discussed in the next section, it will be expected that even prior to the formal evaluation, regular monitoring or “micro-evaluations” are occurring during implementation. This is especially important during the pilot phase of the course, where feedback should be solicited to correct and discover problems in the instruction or examination process.
EVALUATE: In ID models and ASTM E2659-18, program evaluation is considered more comprehensive than a simple post-course survey of learner feedback. It is a complete look at the educational program’s performance that considers each phase of the ID model and ultimately whether the course is effectively meeting the originally identified need. Examples can be found in specific requirements of the program evaluation stated in ASTM E2659-18 Section 6.1.10:
- measurement of the program’s performance versus stated performance objectives (which are often program benchmarks such as number of learners, minimum customer satisfaction ratings, complaint reduction targets, etc.)
- evaluation of the effectiveness of learner assessment methods, including an analysis of how final exam items (questions) are performing and how reliable (consistent) the final exam measures learner performance
- a method for monitoring and identifying the need for change in the program’s purpose, scope, and learning outcomes
- documentation of the program’s evaluation methodology, findings, and proposed changes (if any)
All this adds up to a more formal evaluation based upon an industry accepted program evaluation methodology and supported by evidence that often includes a written report, meeting minutes, statistical summaries, and the production of any other significant evidence produced, maintained, and considered as part of the evaluation process. As mentioned above, “industry accepted standard practice” means education and training industry methodologies that have been previously published in recognized industry publications. Since ASTM E2659-18 does not require a specific evaluation methodology, it is best to research and identify such in an industry publication and reference it by name in your program’s Instructional Design Plan.
The Importance of Following a Recognized ID Model
To sum up, it is essential that your certificate program identify and follow a recognized ID model during program conception and development and that the processes followed during these steps are well documented with a written ID plan. When this is done properly, it will provide sufficient evidence that the standard has been met under section ASTM E2659-18 section 6.1.1 and many of the remaining parts of section 6. Make certain that the chosen ID model has been previously published in a recognized industry publication.
Contributing Author: Dave Bertolet
Dave Bertolet is an ANAB assessor for assessing compliance with the ASTM E2659 Standard Practice for Certificate Programs standard and the accreditation of certificate programs. He is also Senior Consultant for Mickie Rops Consulting with a focus on audits to assess organizational and programmatic compliance with industry standards and accreditation requirements; creating systems and policies for certificate issuers seeking accreditation; facilitating practice analyses to form the basis of valid credentialing test content outlines; and conducting classical test analysis for credentialing assessments.
Dave is president and owner of Renovations for Life, LLC, a general contractor specializing in home modifications for the physically challenged. He has taught college-level coursework in accounting, finance and business management consulting. He holds an MBA and a masters’ in instructional design. Dave can be reached by email at dave@msrops.com.
