What are the required elements, or the ingredients, of “successful” conformity assessment? In this three-part blog series, ANAB explores the three-ingredient recipe needed for successful conformity assessment. The concepts apply to any conformity assessment regardless of the activities therein or who performs them.
In Part 1 of this series, user needs were explored to identify effective specified requirements (often in the form of consensus standards) as the first ingredient in successful conformity assessment. In Part 2 the word “demonstration” was considered in detail to understand optimized schemes as the second ingredient in successful conformity assessment. Both these ingredients are necessary before successful conformity assessment can take place. The third ingredient arises when conformity assessment is implemented in actual practice—that is, when the methodology in a scheme is followed to perform a demonstration that particular object fulfills specified requirements.
Defining Conformity Assessment in ISO/IEC 17000
Given the variety of activities and persons or organizations that perform a conformity assessment, it seems reasonable that conformity assessment should be successful and widely practiced as it gives users needed confidence/assurance with justifiable costs/burdens.
ISO/IEC 17000:2020 defines “Conformity Assessment” as “demonstration that specified requirements are fulfilled.” It also states that a “. . . . demonstration can add substance or credibility . . . that specified requirements are fulfilled, giving users greater confidence . . .” or assurance. This statement seems obvious given how broadly conformity assessment is implemented. The special word “can,” however, makes the statement more complex. “Can” is special in the sense of its specific meaning in ISO/IEC Standards. ISO/IEC Directives Part 2 7.5 indicates the word “can” in ISO documents means “there is a possibility of.” So, per ISO/IEC 17000 there “is a possibility” conformity assessment can add substance or credibility and thus give users confidence or assurance. But apparently, there is also the possibility conformity assessment might not add substance or credibility and not give users confidence or assurance.
The Third Ingredient in Conformity Assessment
Conformity Assessment in Practice
The discussion of schemes in Part 2 showed that the methodology in the scheme is followed when conformity assessment is put into actual practice. Specifically, a series of activities are performed in accordance with the scheme to demonstrate a specific object described in the scheme fulfills specified requirements identified in the scheme. An event must start or trigger that process; demonstrations do not spontaneously initiate for no particular reason. That event is nearly always a request from the provider of the object for such a demonstration. But who does the provider of the object engage with when making such a request? Unless the methodology in the scheme requires all the activities to be performed by the providers themselves, the provider must direct their request to some other person or organization. Who is that?
Part 2 used the term “bodies” for the persons or organization who follow the scheme to carry out activities in accordance with the scheme methodology to complete a demonstration. This reflects the ISO/IEC 17000 series of standards generally use “conformity assessment bodies” or “bodies” to designate the persons or organizations who carry out conformity assessment activities. Conformity assessment is triggered by a request from the provider of the object to one or more conformity assessment bodies. Even when the scheme requires the providers themselves to be the conformity assessment body, a request to the part of the provider organization acting as a conformity assessment body would still start the process.
So, is the third ingredient the existence of a conformity assessment body(ies) to carry out activities in accordance with the scheme?
That question needs to be considered in the context of why conformity assessment is put into practice in the first place. Part 1 emphasized that the purpose of conformity assessment activities is to provide confidence or assurance to users (interested parties) regarding benefits or avoidance of potential problems that result from fulfillment of specified requirements. In this sense, conformity assessment does not produce a tangible outcome such as a product, a processed material, or raw material. Nor are the activities undertaken solely to provide a service desired by the provider of the object such as maintenance of equipment, cleaning a facility, or training of staff. Instead, the measure of success of conformity assessment is whether needed confidence and assurance is provided to the interested parties.
Since confidence and assurance for interested parties is the intended end result, then it seems logical that successful conformity assessment depends on more than just the existence of a conformity assessment body, and that it would need certain characteristics for the demonstration to provide needed confidence and assurance. These characteristics, which are needed when conformity assessment is put in to practice, are the third ingredient.
Capability for Conformity Assessment Activities
Perhaps the most obvious characteristic stems from the fact that activities in the demonstration must be correctly performed in accordance with the scheme. As a result, the conformity assessment body must have the capability to perform the activities in accordance with the methodology of the scheme. Otherwise, the activities have little meaning and the demonstration does not directly or indirectly provide any significant confidence or assurance. It is common for people with specific knowledge and skills to be needed for activities in a scheme, and these people must also have any number of appropriate resources to apply that knowledge and skill to capably perform conformity assessment activities as a basis for confidence and assurance.
The requisite knowledge and skill and the ability to apply it are referred to as “competence” in the field of conformity assessment. The ISO/IEC 17000 series standards commonly define “competence” as ability to apply knowledge and skill to achieve an intended result, which in the case of conformity assessment is a correctly performed demonstration. Competence, and the necessary associated resources, is a necessary characteristic of a conformity assessment body so that conformity assessment delivers confidence and assurance; in other words, for conformity assessment to be successful.
Pressures in Conformity Assessment Activities
But competence is not the only necessary characteristic. Parts 1 and 2 showed that the costs from fulfilling specified requirements and costs of completing activities in a scheme needed to be the minimum necessary for delivering confidence and assurance. Even when so minimized, however, these costs can be significant for the provider of the object and might only be recovered if the demonstration has a positive outcome. In addition, Part 2 showed that a scheme can be on an upward spiral of delivering value related to the interested parties it satisfies and the markets they can open. At some point, a positive outcome from a demonstration in accordance with a very high value scheme can be an existential necessity for the provider of the object. Both the costs and the potential value can create very high pressures for a positive outcome from a demonstration in accordance with the scheme. As noted in Part 2, these pressures incentivize providers of objects to ensure fulfillment of specified requirements using actions outside the demonstration. However, these pressures can also drive other behaviors within the demonstration, which can impede confidence and assurance for interested parties.
Specifically, as Aesop’s goose that laid the golden egg shows, it can be a human trait to focus too intently on an outcome and destroy the process that provides it. In conformity assessment, this human trait can take the form of allowing pressures for a positive outcome to play a role within the demonstration. However, pressures on the provider of the object for a positive outcome are irrelevant to the confidence and assurance needs of interested parties; those confidence and assurance needs are the same regardless of the pressure the provider faces.
If such pressure plays a role within the demonstration, then the interested parties will have no way of knowing the extent to which the outcome was based on relevant activities and results versus irrelevant pressures for a positive outcome. Based on that uncertainty, the demonstration’s ability to engender confidence and assurance will be destroyed. As a result, conformity assessment bodies must be isolated from those pressures, and outcomes of demonstrations must be based solely on the activities performed (in accordance with the scheme) and the results of those activities. This isolation from such pressures in conformity assessment is referred to as “impartiality.” ISO/IEC 17000 defines impartiality as “objectivity with regard to the outcome of a conformity assessment activity,” which means irrelevant pressures are disregarded so that objectivity regarding the outcome of a conformity assessment activity is achieved.
In less frequent cases, pressure can also exist within interested parties for a negative outcome from a demonstration. For example, a very significant negative consequence may arise for interested parties from a positive outcome from a demonstration. This is especially true when conformity assessment pertains to settling or deciding disputes (legal or other) among various parties. A positive outcome can eliminate accountabilities or responsibilities that are helpful to some parties but harmful to others; for some parties, a negative outcome is preferable. Conformity assessment bodies must also be impartial in the sense that they are isolated from pressures for negative outcomes from the demonstration. True impartiality, or isolation from all irrelevant pressures, along with competence are necessary characteristics of a conformity assessment body so that conformity assessment delivers confidence and assurance; in other words, for conformity assessment to be successful.
Variations in Conformity Assessment Activities
The final necessary characteristic of a conformity assessment body emanates from the observation in Part 2 that schemes almost never provide the specific details for every activity to be performed in a demonstration. Conformity assessment bodies must make the choices and decisions not addressed by the scheme each time a demonstration in accordance with the scheme is performed. Both providers of objects and interested parties generally understand and tolerate reasonably small variations from one demonstration to the next. Both recognize that performing absolutely identical demonstrations by eliminating all variations results in unjustified costs and burdens, since the added rigor makes no meaningful impact on the confidence and assurance the demonstration provides.
However, conformity assessment bodies must keep variations to a reasonable minimum as a basis for delivering the expected level of confidence and assurance intended by the scheme. When variations become too large, interested parties will have uncertainty about the extent of confidence and assurance delivered from one demonstration to the next. If interested parties do not perceive the same general level of confidence and assurance as justified by all demonstrations according to a scheme, they can reject the scheme in total and seek other means to meet their confidence and assurance needs. As a result, conformity assessment bodies must be consistent in their demonstrations using the same scheme. Consistency is especially important when the scheme provides only a general methodology, and a great deal of latitude is thus given to conformity assessment bodies when performing actual demonstrations for a specific object. Consistency is a necessary characteristic of a conformity assessment body so that conformity assessment delivers confidence and assurance; in other words, for conformity assessment to be successful.
Competent, Impartial, and Consistent Conformity Assessment Bodies
The third ingredient in successful conformity assessment is a set of three characteristics of the persons or organizations carrying out conformity assessment activities: competence, impartiality, and consistency. A lack of any of these three characteristics results in unsuccessful conformity assessment; specifically, confidence and assurance are not provided. An incompetent body makes a demonstration meaningless for interested parties and no provider of objects will incur the costs of completing it. A biased or inconsistent body introduce such uncertainty about the demonstration that interested parties will not accept it as the basis for confidence or assurance. The impact of the absence of any of these the three characteristics shows that the competence, impartiality, and consistency of a conformity assessment body are collectively the necessary third ingredient for successful conformity assessment.
Involvement of Interested Parties
This third ingredient aligns with the first two ingredients, which were also specific items with necessary characteristics. The three ingredients can be stated as 1) specified requirements which are effective, 2) schemes which are optimized, and 3) conformity assessment bodies which are competent, impartial, and consistent. In Parts 1 and 2, the means of achieving the effectiveness of specified requirements and optimization of schemes was the same: their development and maintenance needed to use processes that allowed the involvement of all interested parties, or at least representation of the perspectives of interested parties. In this way, the effectiveness of standards and the optimization of schemes could be reliably attained. Given that specified requirements and schemes are both types of information, their development and maintenance can take advantage of digital environments and technology to literally include representation of interests worldwide. As a result, it is common to see standards that are seen as effective and schemes that are seen as optimized by interested parties globally.
Involvement of interested parties was also the original means of achieving and assuring competence, impartiality, and consistency of conformity assessment bodies. For many years, conformity assessment bodies achieved and assured their competence, impartiality and consistency by at least interacting with if not actually being controlled or governed by interested parties needing confidence and assurance from work of the conformity assessment body. However, a conformity assessment body is not information like a specified requirement or a scheme. Technological advances like digital environments can not provide a means for all worldwide interested parties to engage with a conformity assessment body as that body grows in size and diversity of activities. So, as conformity assessment bodies grow, a problem arises: how can confidence and assurance about the competence, impartiality, and consistency of conformity assessment bodies be provided if there are too many interested parties to engage with?
Standards for Conformity Assessment Bodies
To address that need, specified requirements in the form of international standards have been developed for the competence, impartiality, and consistency of various kinds of bodies performing types of conformity assessment activities. These are the ISO/IEC 17000 series standards, which have been in use worldwide for decades, and their extensive application and ongoing maintenance have established that their fulfillment delivers the benefits and avoids the potential problems related to competence, impartiality, and consistency of conformity assessment bodies.
These standards, whose scopes covers different types of conformity assessment bodies, have been coupled with defined methodologies through which conformity assessment bodies’ fulfillment of the applicable ISO/IEC 17000 series standard is demonstrated. In a few cases, the conformity assessment body itself performs the demonstration. In other cases, peer conformity assessment bodies perform the demonstration for each other. In still other cases, a separate body follows a methodology and performs a demonstration that the conformity assessment body fulfills the applicable ISO/IEC 17000 series standard.
This is just another example of successful conformity assessment: the ISO/IEC 17000 series standards are the effective specified requirements, the methodology is provided by an optimized scheme, and a body of some type performs activities in accordance with the scheme to demonstrate the conformity assessment body fulfills the specified requirements. In this situation, the conformity assessment body is itself the object, and there are interested parties with needs for confidence and assurance that the conformity assessment body fulfills the applicable ISO/IEC 17000 series standard.
Defining “Conformity Assessment Body” in ISO/IEC 17000
When the conformity assessment body itself performs the demonstration, it is both the provider of the object (the conformity assessment body itself is the object) as well as the body that performs the demonstration. In both its roles, it can be unambiguously referenced as the “conformity assessment body.” When conformity assessment body peers perform the demonstration, each participating conformity assessment body can be either the of the object or a peer (when it is participating in the demonstration of fulfillment of another conformity assessment body). The terms “conformity assessment body” to designate its role as the object and “peer” as its role when performing a demonstration can be used to unambiguously indicate the role it plays.
What term should be used for a separate body that is not a peer and that performs the demonstration that a conformity assessment body fulfills the applicable ISO/IEC 17000 series standard? If that separate body is also designated a “conformity assessment body,” then it would be a hardship to tell the difference between provider of the object (the “conformity assessment body”) and the separate body that performs the demonstration (also the “conformity assessment body”).
This problem is overcome in the definition of “conformity assessment body” in ISO/IEC 17000. ISO/IEC 17000 defines “accreditation body” as the body in this scenario that performs a demonstration in accordance with a scheme to demonstrate a conformity assessment body fulfills specified requirements for competence, impartiality, and consistency. As a result, the definition of “conformity assessment body” in ISO/IEC 17000 is “bodies performing conformity assessment activities other than accreditation.” The different names for the bodies are provided so that ambiguities are avoided when conformity assessment is a demonstration that a conformity assessment body fulfills specified requirements for competence, impartiality and consistency.
To End We Arrive At the Beginning
One obvious question now comes to mind: “Is there an ISO/IEC 17000 series standard for the competence, impartiality and consistency of accreditation bodies?” The answer is YES, and this standard, ISO/IEC 17011, has also been proven effective by its application for many years and the involvement of interested parties in its development and maintenance. Are there optimized schemes that provide a methodology for demonstrating accreditation bodies (the object) fulfill ISO/IEC 17011? YES, and these schemes define a peer assessment methodology. Many accreditation bodies participate in those schemes and thus are both the object (accreditation body) and the peer.
Wherever conformity assessment in its many forms is successful the three ingredients of 1) effective specified requirements; 2) optimized schemes; and 3) competent, impartial, and consistent bodies will be found. If the needed characteristics of any of these ingredients diminish, then the related conformity assessment is threatened, and both providers of objects and interested parties can begin to look for alternatives to meet confidence and assurance needs. For the conformity assessment “goose” to continue to provide the “golden eggs” of confidence and assurance, then caring for its health and wellbeing by maintaining the effectiveness of specified requirements, optimization schemes, and competence, impartiality, and consistency of bodies must be the concern of all involved. That is the recipe that uses the three ingredients for successful conformity assessment.