Types of Audits

Before we discuss the different “types” of audits, we must first define the “scope” of the audit.

The audit “scope” also includes defining criteria such as:

• The specific site(s) covered by the audit (e.g., specific addresses, Building Numbers)
• Any specific function(s) or process(es) to be covered (e.g., Sales, Purchasing, Engineering, Production)
• The Audit Criteria for:
  
  • a QMS Internal audit (covering specific internal documents; procedures, Work Instructions, etc.),
    
  • a QMS Standard Audit (specifying a QMS Standard such as ISO 9001, AS 9100, ISO 17020, ISO 17025, etc.),
    
  • a Product Audit (i.e., for a specific product),
    
  • a Configuration Audit (i.e., for a specific product configuration)

While there are many types of audits, and audit strategies, the most common types are:

• Quality Management System Audits
• Product Audits
• Configuration Audits

ALL of the above audits can be performed either internally (on the company itself) or externally (on suppliers/subcontractors).

While ISO 19011:2018 Annex A mentions several different “types” of audits (as “scopes”), neither ISO 19011:2018 nor ISO 9000:2015 defines nor describes the difference between these audit types.

ISO 19011:2018 A.12 Audit of supply chain
The audit of the supply chain to specific requirements can be required. The supplier audit programme should be developed with applicable audit criteria for the type of suppliers and external providers. The scope of the supply chain audit can differ, e.g. complete management system audit, single process audit, product audit, configuration audit.

Value-Added Audits (VAA)
While all of the above-listed audit types have an objective of verifying “compliance”. a “Value-Added Audit” (VAA) focuses on improvement through the elimination or reduction of non-value-added (NVA) activities/steps in a sequence. A common approach is to utilize a detailed flow chart of the process - identifying those “activities/steps” that either “add no value” or “add no value but are necessary” (e.g., an inspection activity to mitigate the possibility of nonconforming product(s) being delivered).

A VAA is often used to identify opportunities for improvement. more information on VAAs is at the bottom of this article.

The most common audits are Internal Audits of the Quality Management System (required by most QMS standards, e.g., ISO 9001, AS9100). However, some companies also perform Quality Management System Audits of their suppliers/subcontractors.

System Audit
An audit is conducted on a management system. It can be described as a documented activity performed to verify, by examination and evaluation of objective evidence, that applicable elements of the system are appropriate and effective and have been developed, documented, and implemented in accordance and in conjunction with specified requirements. (Source: https://asq.org/quality-resources/auditing)

Quality Management System (QMS) Audits can be performed at one time (e.g., as Certification Bodies do during their Re-Assessments) or of portions of the QMS (e.g., as Certification Bodies do during their Surveillance Audits).

Most common in QMS audits, audit sub-types can include:

• Process Audits
• Clause-based Audits
• Horizontal Audits
• Vertical Audits

Process Audits
ASQ has an article titled “What is Auditing”, which includes a section titled: “The Three Different Types of Audits”. While not “official” definitions of the terms, the article describes a “Process Audit” (actually, an audit sub-type) in a way that is very understandable.

Process audit
This type of audit verifies that processes are working within established limits. It evaluates an operation or method against predetermined instructions or standards to measure conformance to these standards and the effectiveness of the instructions. A process audit may:

• Check conformance to defined requirements such as time, accuracy, temperature, pressure, composition, responsiveness, amperage, and component mixture.
• Examine the resources (equipment, materials, people) applied to transform the inputs into outputs, the environment, the methods (procedures, instructions) followed, and the measures collected to determine process performance.
• Check the adequacy and effectiveness of the process controls established by procedures, work instructions, flowcharts, and training and process specifications.

(Source: https://asq.org/quality-resources/auditing)

Despite the fact that there are many different types of audits, ISO 9001 & AS9100 registrars and consultants have been promoting “Process-based” audits for years. And, largely due to “indoctrination” (from the registrars) in how to think about ISO 9001 & AS9100, many auditors consider a “process-based” audit to be the ONLY way possible to determine ”whether the quality management system is effectively implemented and maintained”. This is further supported by a “non-binding” opinion provided in the US TC 176 - TG22 - Interpretations (Read: Re-writing ISO 9001:2015... through Interpretation).

In fact, they've been promoted to the point that some ISO 9001 & AS9100 auditors have actually issued (unjustified) nonconformities to companies for having performed “clause/element-based“ internal audits rather than the preferred “process-based” audits! These nonconformities are based upon a highly subjective interpretation of ISO 9001 & AS9100.

Despite the subjective interpretations/opinions of some auditors, companies should decide the type of internal audit that best provides ”information on whether the quality management system is effectively implemented and maintained“.

If faced with an external (e.g., 3rd Party) auditor who insists that you must perform “process-based” internal audits in order to ”provide information on whether the quality management system is effectively implemented and maintained“, then there are several ways to argue this.

1. Neither ISO 9000:2015 nor AS 9100:2016 requires that a “process audit” or “process-based audit” be performed in order to ”provide information on whether the quality management system is effectively implemented and maintained“.
2. No ISO document (including ISO 19011:2018 nor ISO 9000:2015) defines the term “process audit” or “process-based audit”.
3. The ”ISO 9001 Auditing Practices Group Guidance on: Demonstrate conformity to the standard“ does not require “process-based” internal audits to demonstrate conformity with ISO 9001:2015.
4. While the ”ISO 9001 Auditing Practices Group Guidance on: Audit Planning“ states ”Develop the plan around the processes – not the clauses of the standard“, this is contained in a section titled “USEFUL TIPS TO AUDITORS”… because this is NOT a requirement.
5. The official ISO document ”The Process Approach in ISO 9001:2015“ describes the process approach toward the QMS WITHOUT requiring (or even mentioning) a “process-based” internal audit.

The above should be sufficient information to ”appeal“ this type of invalid nonconformity.

The point is that auditors should be there to verify compliance with “actual” requirements. Not imagine/invent requirements where none exist.

Clause-based Audits
A “clause-based” (aka “element-based”) is where an audit verifies compliance with individual clauses/elements contained in a QMS Standard. For example, only auditing ISO 9001, section 7.2 “Competence”.

Horizontal Audits
A horizontal audit is an audit of “one” process across multiple departments in the business. Common examples of this approach include:

• Competence (ISO 9001/AS9100, sec. 7.2)
• Document Control (ISO 9001/AS9100, sec. 7.5)
• Corrective Action (ISO 9001/AS9100, sec. 10.2)

Using “Competence” as an example, the auditor would examine a sampling of records used to provide evidence of the competence of the personnel performing quality-related work in every area (e.g., Sales, Procurement, Engineering, Planning, Production, Inspection, Shipping).

Using “Document Control” as another example, the auditor would verify control of documents in every area where documents are distributed (or otherwise required to be controlled - e.g., documents of external origin).

Vertical Audits
A vertical audit is an audit of “all” the processes used by a department. Common examples of this approach include:

• Sales (ISO 9001/AS9100, sec. 8.2)
• Engineering (ISO 9001/AS9100, sec. 8.3)
• Procurement (ISO 9001/AS9100, sec. 8.4)
• Production (ISO 9001/AS9100, sec. 8.5, 8.6 & 8.7)

Using “Sales” as an example, the auditor would:

1. interview personnel concerning the process inputs, activities, and outputs;
2. examine a sampling of records (required by sec. 8.2.3.2) used to provide evidence of a contract/order review having been performed (with its results) and any records created to describe any new requirements for the products and services (e.g., Change Orders, Amended/Revised Orders).

Using “Procurement” as another example, the auditor would interview personnel concerning the process inputs, activities, and outputs. And examine a sampling of the records required by ISO 9001/AS9100, sec. 8.4 to be retained.

During a full system audit, the auditors “should” verify that the linkages between processes provide effective communication of requirements.

A “Product Audit” is performed with the objective of verifying whether a particular product (e.g., parts or assemblies, processed material, software) conforms to requirements (i.e., drawings, BOMs, technical specifications, product/material standards).

In fact, the ASQ definition of a “Product Audit” includes “services”.

Product Audit
This type of audit is an examination of a particular product or service, such as hardware, processed material, or software, to evaluate whether it conforms to requirements (i.e., specifications, performance standards, and customer requirements). (Source: https://asq.org/quality-resources/auditing)

Product audits can be either performed internally or at supplier sites.

These audits should begin by identifying the particular product(s) that the customer is purchasing from the supplier. If the company has any specific product/service-related concerns, then these should be considered when planning the audit.

A product audit typically begins by verifying that all of the customer requirements have been communicated to production personnel e.g., through travelers, drawings, Work Instructions, Specifications). If the requirements were not adequately communicated to production, then there is typically a communication breakdown between the Sales function and Production.

The auditor should then follow an example of the product model (e.g., Part Number) being purchased by the customer through the entire production process… verifying that all requirements were met, including confirming that:

• the product was manufactured to the correct product configuration – through verifying that the revision levels of all component/detailed parts match the revision levels required by the customer (typically through confirming that the detailed parts & subassemblies used match the product BOM (Bill of Material)
• the correct raw material was used (e.g., supported by a Material Test Report, Certificate of Analysis, Chemical Analysis Report).
• the work was performed in the proper environment (e.g., a Cleanroom environment)
• proper equipment is provided (e.g., to prevent ESD damage),
• any special processing was performed by a qualified process (e.g., a Nadcap certified process),
• measuring instruments had the appropriate accuracy (“Accuracy Ratio” between the instrument and the tolerance measured), range & resolution.

The product audit would also examine completed records (from previous product runs) providing evidence that the practices observed are consistent (e.g., completed Job Travelers, nonconformity reports of this specific product model (part number), and, if available, a Pareto chart identifying key issues with this product. If there are issues identified (either externally, through customer complaints or internally, through employees identifying nonconformities), the auditor should evaluate the effectiveness of any analysis (e.g., Fishbone, charts, 5-Whys) and actions are being taken to mitigate (through risk management) or eliminate (through corrective action) these issues. This is an area where both ISO 9001 & AS 9100 are weak.

All of the objective evidence collected should provide confidence to the customer that the specific product model (e.g., Part Number) is being produced in accordance with requirements.

Is this how “Quality Auditing performs Supplier Audits?

If not, then this may be a new area of opportunity… perhaps through “branding” these as “Product Audits” through:

• A series of articles educating customers as to how product audits can benefit them… even of suppliers who have a registered QMS.
• Creation of a generic “Product Audit” checklist to be used by “Quality Auditing” to provide consistency in both performance and reporting.

Supplier/Product Audits are typically of suppliers identified by customers, as having significant quality issues. And a major part of supplier audits is the follow-up. Either verifying that the actions taken by the supplier are effective, or working with the supplier to determine and implement an effective plan to improve product quality. These two activities should be performed jointly in order to justify the cost of Supplier/Product Audits. Source Inspections I was talking with Oshan about these types of audits (on 12/9/21). A source inspection is where a Purchaser/Buyer (i.e., customer), OR their representative (e.g., “Quality Auditing”), verifies the conformity of a product (e.g., whether based on a sampling of a batch/lot, or 100%) AFTER production, and prior to the product(s) being shipped to the customer.

Downstream Audits
A “Downstream Audit” starts at the beginning of the value stream and follows an order from receipt (e.g., “Sales”) through each functional area and process, in sequence until the end (e.g., the Shipping area). The auditor should verify that all of the requirements are communicated through each stage of the value stream such that the final product/service satisfies all of its requirements.

Upstream Audits
An “Upstream” audit begins with a final product/service (e.g., awaiting delivery in the Shipping area) and follows the value stream in reverse order (i.e., “upstream”) to where the contract/order was received (e.g., the Sales area). The auditor should verify that all of the requirements were accurately communicated from each preceding stage of the value stream, ensuring that the final product/service satisfies all of its requirements.

While “configuration audit” is also not defined by ISO (even in ISO 10007:2003, “Quality management systems — Guidelines for configuration management”), the U.S. Federal Aviation Administration (FAA) has defined the term in Order1800.66 (Incl Chg 3), "Configuration Management Policy".

Configuration Audit
Product configuration verification accomplished by inspecting documents, products, and records; and reviewing procedures, processes, and systems of operation to verify that the product has achieved its required attributes (performance requirements and functional constraints), and the product’s design is accurately documented. Sometimes divided into separate functional and physical configuration audits.

Although it doesn't define the generic term “Configuration Audit”, the U.S. Defense Acquisition University - Glossary does define the specific terms “Functional Configuration Audit” & “Physical Configuration Audit” (PCA).

Functional Configuration Audit
The formal examination of functional characteristics of a configuration item, or system, to verify that the item has achieved the requirements specified in its functional and/or allocated configuration documentation. (Source: U.S. Defense Acquisition University - Glossary)

Physical Configuration Audit (PCA)
Physical examination of the actual configuration of the item being produced. It verifies that the related design documentation matches the item as specified in the contract. The system product baseline if finalized and validated at the PCA. (Source: U.S. Defense Acquisition University - Glossary)

While all of the above-listed audit types have an objective of verifying “compliance”. a “Value-Added Audit” (VAA) focuses on improvement through the elimination or reduction of non-value-added (NVA) activities/steps in a sequence.

“Value-Added Audits” are performed “internally” and tend to be used by companies who have adopted the use of Lean 6 Sigma tools. In a “Value-Added Audit”, the auditor examines each activity/step in a process asking whether the activity/step adds value to the product or service. If the customer isn't paying for the activity/step being performed, then it doesn't add value. However, there may be some activities/steps that a necessary to properly complete an activity/step that does add value.

As a part of the VAA, the auditor seeks to determine whether there is any justification for each NVA activity/step in the sequence/process. NVA results in some form of waste.

The following eight lean manufacturing wastes, mostly derived from the TPS (Toyota Production System), have a universal application to businesses today. The acronym for the eight wastes is DOWNTIME. Downtime stands for:

• Defects
• Overproduction
• Waiting
• Not utilizing talent
• Transportation
• Inventory excess
• Motion waste
• Excess processing

Performing a VAA is a great way to identify significant areas for improvement. However, at a minimum, whoever is assigned to perform a VAA should be a Lean Six Sigma certified Green Belt. Otherwise, the VAA will likely fail to prove beneficial.