What are the required elements, or the ingredients, of “successful” conformity assessment? In this three-part blog series, ANAB explores the three-ingredient recipe needed for successful conformity assessment. The concepts apply to any conformity assessment regardless of the activities therein or who performs them.
In Part 1 of this series, user needs were explored to identify effective specified requirements (often in the form of consensus standards) as the first ingredient in successful conformity assessment. The definition of conformity assessment explicitly refers to this ingredient since it includes the term “specified requirements.” Another term in that definition points us to the second ingredient. That term is “demonstration.”
Defining Conformity Assessment in ISO/IEC 17000
Given the variety of activities and persons or organizations that perform a conformity assessment, it seems reasonable that conformity assessment should be successful and widely practiced as it gives users needed confidence/assurance with justifiable costs/burdens.
ISO/IEC 17000:2020 defines “Conformity Assessment” as “demonstration that specified requirements are fulfilled.” It also states that a “. . . . demonstration can add substance or credibility . . . that specified requirements are fulfilled, giving users greater confidence . . .” or assurance. This statement seems obvious given how broadly conformity assessment is implemented. The special word “can,” however, makes the statement more complex. “Can” is special in the sense of its specific meaning in ISO/IEC Standards. ISO/IEC Directives Part 2 7.5 indicates the word “can” in ISO documents means “there is a possibility of.” So, per ISO/IEC 17000 there “is a possibility” conformity assessment can add substance or credibility and thus give users confidence or assurance. But apparently, there is also the possibility conformity assessment might not add substance or credibility and not give users confidence or assurance.
The Second Ingredient in Conformity Assessment
“Demonstration” in Conformity Assessment
Understanding why the word “demonstration” was chosen for use in the ISO definition of “conformity assessment” is the first step to finding the second ingredient. The extremely broad concept “conformity assessment,” per its definition, covers an enormous range of activities. Perhaps the word “demonstration” was chosen because it was very general and all the many types of activities that might be part of conformity assessment can be generally represented by it.
While that is logical and is part of the reason for choosing the word “demonstration,” the primary reason was because “demonstration” best represents the type of activities in conformity assessment. When any specific instance, or occurrence, of conformity assessment is examined in detail, the fact that the activities therein comprise a demonstration is clear. Thus, it was an obvious choice for use in the definition of “conformity assessment.”
Conformity assessment as a “demonstration” becomes especially obvious when the organization (or body) performing the demonstration is different than the organization that provides or is the object that is purported to fulfill requirements. In these cases, there is commonly an agreement or contract between the two organizations describing the roles and responsibilities for the activities that will be performed in conformity assessment. These agreements clearly indicate that a demonstration of fulfillment of requirements will be carried out.
In virtually all cases, such agreements also clarify that the demonstration does not transfer responsibility for fulfillment of requirements. This arrangement can also be the case when the same organization carries out the demonstration and is or provides the object. Such a demonstration does not necessarily transfer the responsibility for fulfillment of requirements from one part of the organization to another.
But if a set of activities (generally characterized as a “demonstration”) does not transfer responsibility, how can it provide confidence or assurance that the specified requirements are fulfilled and that the subsequent benefits from that fulfillment can be expected? To answer that question, and identify the second ingredient of successful conformity assessment, a more detailed understanding of the activities that comprise a “demonstration” without transfer of responsibility is needed.
Conformity Assessment Scheme
The word “demonstration” does not by definition specify, describe nor in any way limit the activities that could comprise conformity assessment. The choice of activities, when and how frequently they are carried out, and the persons or organization (bodies) to perform them (collectively hereafter, “the choices”) can and do vary from one demonstration to another. In fact, the choices are as varied as the standards that can serve as the specified requirements.
The choices for activities in the demonstration are provided to some extent in the “scheme” for conformity assessment. ISO/IEC 17000:2020 4.9 defines “conformity assessment scheme” (or “conformity assessment programme”) as a “set of rules and procedures that describes the object of conformity, identifies the specified requirements and provides the methodology for performing conformity assessment.” The last element, the “methodology for performing conformity assessment” provides some portion of (but not necessarily all)the choices for activities in the demonstration.
A scheme always exists any time conformity assessment is carried out. The methodology, or the choices for activities, are always decided and defined in some manner. In fact, the methodology, or choices for activities, are always decided prior to the activities being performed. Otherwise, conformity assessment would be a random set of spontaneous activities chosen for no particular reason, which is illogical and would clearly make the demonstration meaningless. It may be that a “scheme” (the choices) applies to only one demonstration that a single object fulfills specified requirements and will never be used again. Or, a “scheme” (the choices) may be decided once and used repeatedly for many demonstrations involving different objects or different specified requirements so that consistency between different demonstrations is achieved. However, in all cases, the scheme (the choices) are decided and defined before they are performed.
The methodology in a scheme rarely specifies every single activity or detail for performing a demonstration. Rather, schemes commonly provide general methodology so they can be used for a range of objects and specified requirements (standards). For example, some schemes cover thousands of types of products and hundreds of individual specified requirements (standards) with a single methodology. In all cases, any details for performing the demonstration that are not specified by the scheme will by necessity be decided by the persons or organizations (bodies) who carry out the activities to demonstrate an object fulfills specified requirements.
Impact of Schemes
The scheme (the choices) would seem to have a direct impact on the resulting assurance that the specified requirements are fulfilled and that the subsequent benefits will result. It would be logical to assume that the more extensive the activities within a scheme, the greater the likelihood that nonfulfillment of requirements will be detected and corrected and thus the higher the assurance. Based on this assumption there would be a direct relationship between the number and extent of activities in the scheme (the choices), and assurance. Such a direct relationship exists for schemes which provide assurance at a specific point in time or for a very short period of time. The number and extent of activities within a scheme in such a situation would be chosen based on the assurance needed.
When assurance is needed over any but the shortest periods of time then another contributor to assurance becomes important: the actions persons or organizations (usually the providers of the object) can take separate from the demonstration to ensure fulfillment of specified requirements. A demonstration as a set of activities (the choices) cannot ensure fulfillment of requirements since it takes place after the design, creation, production, provision, etc. of the object that fulfills the requirements. However, persons or organizations which provide the object can take actions during the design, creation, production, provision, etc. of the object and those actions can ensure fulfilment of requirements.
When actions to ensure fulfillment of requirements are effective, they too make a significant contribution to assurance for the interested parties. In cases where such actions are taken, assurance would be directly related to the combination of the scheme (the choices) and the separate provider activities; assurance would not be directly related only to the number and extent of activities in the scheme.
Regardless of the time frame over which assurance is provided, conformity assessment is implemented only when its costs and burdens for the provider are justified by the benefits of satisfying authorities and market operators who demand assurance (the interested parties). More specifically, conformity assessment schemes need to specify methodologies (the choices) that will provide the needed assurance for the lowest possible cost and burden. Assurance beyond what is needed provides no benefits and the related costs and burdens are not justified. Insufficient assurance for interested parties makes conformity assessment meaningless regardless of how little the cost and burdens for providers. Interested parties or providers will seek other alternatives if their respective needs are not met.
When assurance is needed beyond a single point or short period of time, schemes commonly utilize assurance from providers’ actions to ensure fulfillment of specified requirements to minimize costs and burdens for providers. Schemes are able to take this approach because incentives exist that schemes can leverage to deliver significant contributions to assurance with little incremental cost and burden for providers.
Incentives As Assurance
A successful demonstration of fulfillment of specified requirements according to a scheme becomes the primary mechanism providers use to satisfy the ongoing assurance demands of interested parties who control access to market(s). The demonstration is thus reasonably considered to be an asset for the provider. Both the number of the interested parties satisfied, as well as the size of the market(s) they control, factor into the value of the demonstration as an asset. In general, the higher the number of interested parties satisfied and the bigger the market(s) to which access is gained, the greater the value of the asset.
However, regardless of its value, the asset only provides a return if ongoing fulfillment of requirements is consistently achieved. A demonstration ceases to provide a return when it is subsequently contradicted by objects failing to fulfill requirements. In some cases, the contradicted demonstration may even become a liability, especially if it seriously damages the reputation of the provider. Thus, a provider is incentivized to ensure ongoing fulfillment of requirements to protect the asset and potentially avoid a liability.
Two means exist to increase the value of the asset and thus increase the incentive for providers to take actions to ensure fulfillment of requirements separate from the demonstration. First, the value of the asset can be increased by increasing the number of interested parties whose assurance needs are satisfied by the demonstration. When the same demonstration satisfies more assurance needs its value goes up. Second, the value of the asset can be increased by adding additional markets for which the demonstration will provide access. When the value of the demonstration goes up the incentive to protect that asset (by ensuring fulfillment of requirements) also goes up.
In addition, schemes can increase the incentive by requiring ongoing activities as indicators (generally not full demonstrations) that the provider is achieving consistent fulfillment of requirements and the demonstration remains valid. When such actions are sufficient to refocus the provider’s attention to both the value of the asset and the actions needed to protect it, a stronger incentive results. These types of activities in schemes are generally referred to as surveillance (see ISO/IEC 17000 A.5).
Because higher asset value is attractive to providers and higher assurance is attractive to interested parties, schemes can enter in to a “virtuous cycle” through these activities. The more the scheme expands to meet assurance demands of interested parties and markets, the more valuable the demonstration becomes. The more valuable the demonstration becomes, the greater the incentive providers have to ensure ongoing fulfillment of requirements. The greater the incentive providers have within a scheme for ongoing fulfillment of requirements, the more readily the scheme will be acceptable to additional interested parties and markets.
Identifying and expanding the assurance needs that must be fulfilled and specifying the most cost-effective mix of activities in the scheme to meet those assurance needs is the process of optimizing a scheme. Ongoing review and adjustments to schemes are common to keep a scheme optimized and are especially important when events create the need for more assurance or lower costs/burdens. Clearly, ongoing interactions with interested parties and providers of objects are needed to understand the changing needs and perceptions of each so a scheme can stay optimized.
Understanding all the aspects of “demonstration” in the definition of conformity assessment shows that optimized schemes are the second ingredient for successful conformity assessment. As with effective specified requirements, the converse of this statement provides confirmation. If highly effective specified requirements are in place, but the scheme provides insufficient assurance or unnecessary costs and burdens (the scheme is not optimized), conformity assessment will not occur let alone be successful.
The Third Ingredient in Successful Conformity Assessment
Effective specified requirements and optimized schemes seem to provide everything needed for successful conformity assessment, at least in theory. But conformity assessment is not a theoretical exercise; it must be put into practice. Actual demonstrations of fulfillment of effective specified requirements in accordance with an optimized scheme must be put into practice for conformity assessment to be successful. Considering the implications of conformity assessment as a practical activity leads to the third and final ingredient for successful conformity assessment.
