It can be difficult to decide what questions to include on the final exam of a certificate course. A common approach is to comb through the course and write questions based on content that is relatively easy to formulate into a multiple-choice question. However, this could easily result in a set of items that do not provide a fair representation of the course, with too many items that are either redundant or trivial.
There is a better way: is the table of specifications. Its purpose is to provide a blueprint to ensure that you have a balanced measure of your course’s learning objectives. This post shares an example table of specifications for a fictional certificate-based course so you can see what it looks like and think about how you might create one for your program.
What Is a Table of Specifications?
A table of specifications is a two-way chart that identifies the format and content of the assessment, using the learning objectives and the instructional content as a guide. There are many ways to build a table of specifications and many different versions of these tables. The process used in this post is adapted from chapter 6 of the textbook Measurement and Assessment in Teaching by Miller, Linn, & Gronlund (2012, 11th ed.).
Table of Specifications Example
For example, a nationwide grocery chain plans to pilot a certificate-based course with their employees that teaches them to develop healthy meal plans for the purposes of designing packaged food kits. The team in charge of this effort conducted a needs assessment that considered the learner level, experience and skill to identify the main topics for instruction and the course learning objectives. The team identified five learning objectives, listed below.
General course objectives
- Recalls important nutrients
- Recognizes the top foods for healthy eating
- Interprets nutritional labels
- Describes a healthy recipe
- Describes a balanced meal plan
They also developed the following high-level outline of the course topics.
Course content outline
- Macronutrients
- Micronutrients
- Processed foods
- Whole foods
- Nutritional labeling
- Meal templates
(Note: the team also created specific learning outcomes for each of the general course objectives, and a more detailed course content outline, but that level of detail is not required for a table of specifications, so these are omitted from this example.)
With the learning objectives and the content of the course established, a table of specifications was then created. According to Miller et al. (2012), creating the table of specifications is a three-step process:
- List the general learning objectives across the top of the table.
- List the high-level content areas down the left side of the table.
- Determine what proportion of the test items should be devoted to each objective and each content area.
An example of the table of specifications for this fictional grocery chain is presented below. This table shows the number of test items dedicated to each area of the test based after considering the emphasis given to each content area and learning objective in the course.
First, the team decided how many items they wanted to have on the test. They considered the nature of the subject, the depth of content, the level of thinking skills, and the length of the course. With those factors in mind, they decided that the test should have 85 questions. They put this number into the bottom row, in the column titled TOTAL.
Second, the team discussed how much emphasis to give to each topic area on the test. Time spent for each topic in the course is a useful metric (second column), so the team decided to use it for determining the number of test items for each topic area. To determine how many items on the test to dedicate to each content area, they divided the total number of items by the total minutes, then multiplied it with the number of minutes for each topic.
For the Macronutrients topic, the calculation was 85/235 X 45. This resulted in 16 items, after rounding to the nearest whole number. They proceeded to do this for every content area, rounding these values to the nearest whole number and at the end, checking to make sure the sum was still 85.
Third, they filled in the rest of the table working row by row. In the first row, they considered that they had 16 items to cover macronutrients under the five objectives. They filled in the values in that row by considering how macronutrients were presented during the course and what skills (i.e., knowing, interpreting or describing) were targeted by that instructional content. They repeated this process for the rest of the rows. Not all entries in the table have items, which is fine. For example, for the objective Nutritional Labels, there are no items for Whole foods. This is because Whole foods do not have nutritional labels, so it was not covered in the course and therefore it did not make sense to include it in the test. The team reviewed the final distribution of items in the table of specifications until they were satisfied that they that the table reflected the emphasis given during the instruction and the importance of the objectives.
| Objectives | ||||||||
| Content | Minutes on Topic | Knows | Interprets | Describes | Total # Items | Total % Items | ||
| Important Nutrients | Top Foods for Healthy Eating | Nutritional Labels | A Healthy Recipe | A Balanced Meal Plan | ||||
| Macronutrients | 45 | 5 | 2 | 3 | 2 | 4 | 16 | 19% |
| Micronutrients | 50 | 5 | 2 | 3 | 3 | 5 | 18 | 21% |
| Processed foods | 35 | 2 | 2 | 3 | 3 | 3 | 13 | 15% |
| Whole foods | 25 | 1 | 4 | 0 | 3 | 1 | 9 | 11% |
| Nutritional labeling | 15 | 0 | 0 | 5 | 0 | 0 | 5 | 6% |
| Meal templates | 65 | 4 | 6 | 2 | 5 | 7 | 24 | 28% |
| TOTAL | 235 | 17 | 16 | 16 | 16 | 20 | 85 | 100% |
Table 1. Healthy Eating Course Table of Specifications
This example was rather simple because the learning objectives were amenable to using a single assessment type (in this case, selected response) for evaluating student learning. An appropriate assessment for this course might be a combination of basic and enhanced multiple-choice items, matching items, and fill-in-the blank items, based on the learning objectives. These items all have in common a feature where the right answer is provided within the alternatives given. If some of the objectives had verbs such as “constructs,” “delivers,” or “measures,” then performance-based tasks would have to also be assessed and accounted for in some way, not necessarily within the table of specifications but rather as an addendum to the table.
Developing test specifications is one of the most effective ways to ensure that the test adequately covers the content and learning objectives of a course. It may seem time consuming at first, but it will make developing the test much easier once the plan has been created. It also provides key content-related validity evidence of your test, which is pertinent to ANSI/ASTM E2659-18 Standard Practice for Certificate Issuers as well as the Standards for Educational and Psychological Testing (AERA, APA, NCME, 2014)
Finally, “table of specifications” and “test blueprint” are not mentioned in ANSI/ASTM E2659-18 Standard Practice for Certificate Issuers. However, I should point out that it is inferred in two clauses in section 6.1.6:
6.1.6.2: The summative assessment(s) shall “be aligned with and assess the extent to which learners achieve the intended learning outcomes.”
The requirement doesn’t state how such alignment is to be done, but an alignment matrix of some kind, such as the table of specifications, is the most effective method for ensuring alignment between the assessment and the course objectives.
6.1.6.3: The summative assessment(s) shall “use assessment methods and delivery appropriate for and consistent with the certificate program’s purpose, scope, and intended learning outcomes.”
The table of specifications is a first step in informing which assessment methods are appropriate given the learning objectives of the course and ensuring proportional representation of them in the different formats that an assessment may take.
References
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (Eds.). (2014). Standards for educational and psychological testing. American Educational Research Association.
Miller, M. D., Linn, R. L., & Gronlund, N. E. (2012). Measurement and assessment in teaching (11th ed.). Upper Saddle River, NJ: Pearson Education, Inc.
Contributing Author: Kathy Tuzinski
Kathy Tuzinski is a Contract Assessor for ANAB’s Certificate Accreditation Program. She has a science-practioner’s background in the testing industry as a psychometrician, consultant, and industrial/organizational psychologist. She started Human Measures to provide the testing industry support in developing testing programs that are based on science – advising companies on job analysis and competency modeling, test development and test validation, legal requirements for employee selection, and relevant testing standards, including the AERA, APA, and NCME Standards for Educational and Psychological Testing and the SIOP Principles for the Validation and Use of Personnel Selection Procedures. She is a member of the American Psychological Association and the Society for Industrial/Organizational Psychology, has published several articles in peer-reviewed journals, and is co-editor of the book Simulations for Personnel Selection. She holds a Master of Arts degree in Industrial/Organizational Psychology from the University of Minnesota. You can reach her at https://www.human-measures.com/.
