How OCAP effects AS9100 Companies

The revised SAE AS9104/1A requires AS 9100 CBs (Certification Bodies… i.e., Registrars) to use the “Organization Certification Analysis Process (OCAP)” to determine an overall “risk rating” for each registered company. This “risk-rating” will then be used during audit planning — to provide reductions or increases (by ±10%, as per AS9104/1A, “Table 9 - Audit Duration Risk Adjustments”) in certification and surveillance audit durations (i.e., number of audit days required) from baseline audit durations found in SAE AS9104/1A, “Table 8 - Audit Duration Per Site”.

The OCAP is defined in SAE AS9104/1A as:

3.8 Organization Certification Analysis Process (OCAP)
An interactive process between the organization and CB to determine the organization’s AQMS scope and associated certification audit program, and conduct a risk assessment for certification within the ICOP scheme.

The OASIS database is scheduled to begin supporting the new SAE AS9104/1A criteria by 3/31/23. And the new OCAP process is required to be fully implemented by the Certifying Bodies (CBs) no later than 3/31/24.

The new OCAP process requires each Certified Organization (i.e., AS 9100 series registered company) to submit data to the Certifying Body (CB) — no more than 90 days prior to the start of the audit. This data will likely be gathered using a questionnaire (which some CBs have already begun to gather - so you should begin planning to address this information ASAP) — and the answers provided will be used by the CB for determining an overall “risk rating” (High, Medium, Low) for the certified company. The overall “risk rating” is calculated using AS9104/1A, Table 7 - “Organizational Risk Determination”, shown below.

While there is really nothing that a company can do to alter its complexity (without major changes in how it operates), the remaining “Risk Factors” are completely in control of the business. However, if there are changes in your organization, the graphic image below is essentially the same as the one shown in AS91041A, “Figure 2 - Organization complexity risk level”. The risk level for your organization should coincide with the criteria contained in the quadrant that best describes your organization.

The “Risk Factor” criteria relating to internal audit performance is:

Strategy

1. To ensure a “Properly resourced audit program”, have more than one qualified internal auditor. If your company has limited resources, then consider outsourcing either a portion or the entirety of the internal audit program to a company that specializes in quality auditing… and can support AS9100 internal auditing activities (e.g., Quality Auditing, LLC).
2. The criterion relating to a “Multi-event audit program, audit full QMS annually” means that, in order to be classified as a low risk, the company must “spread out” their internal audits over the course of a year (e.g., Quarterly, Monthly). For small companies, this could be difficult, but with only two or three core processes, these could be broken out to be 2 or 3 separate audits (with separate audit reports).
3. The criterion relating to an “Audit program driven by risk and data” is not defined AS9104/1A. For further guidance see Risk-Based Internal Audits. And use my free “Risk-Based Audit Planning Criteria” form (in MS Word).
4. The best way to ensure that you have an “effective corrective action program”, is to avoid engaging in Corrective Action "Whac-A-Mole".

AS9100sec. 5.1.2 “Customer Focus”, already requires (1) On-Time Delivery and (2) Conformity of Delivered Product or Service to have “Quality Objectives” associated with them (as KPIs - Key Performance Indicators).

AS9100, sec. 5.1.2 “Customer Focus”
Top management shall demonstrate leadership and commitment with respect to customer focus by ensuring that:
d. product and service conformity and on-time delivery performance are measured and appropriate action is taken if planned results are not, or will not be, achieved.

These metrics are also required to be monitored and reviewed in AS9100, sections: 9.1.2 & 9.3.2

A problem that I often see in many AS9100 companies is management establishing “Quality Objectives” based on the “average” of the KPI data. This approach guarantees that they will fail to achieve their “Quality Objectives” half of the time. Referring to AS9104/1A, Table 7 - “Organizational Risk Determination”, shown above, this approach would categorize them as a “High” risk company.

Other Aerospace companies take a conservative approach by establishing “Quality Objectives” that are easily achieved.

For example, a company with a Quality Objective of 97% OTD… but only achieving 95%, will score more poorly than a company that established a Quality Objective of 90% OTD and achieved 91% OTD. If this sounds like IAQG is promoting mediocrity… there is another piece to the puzzle.

If you have one or more customers issuing “Supplier Score Cards”, whatever objectives and minimum goals they establish are considered to be your required Quality Objectives by the CBs. This is intended to circumvent companies from establishing easily achievable Quality Objectives. However, since the majority of AS91xx companies are small… and do not receive “Supplier Score Cards”, this reveals that IAQG is actually focused on the “significant few” (suppliers) rather than the “trivial many”.

It could be argued that CB auditors will be more closely examining the quality improvements taking place (Ref. AS9100, sec. 10.3, “Continual Improvement”) to ensure that companies establishing easily achieved “Quality Objectives” (and be classified as “Low” risk companies). However, there are MANY quality improvements that can be made that do not directly impact the required “Quality Objectives” (e.g., scrap reduction, SMED / Set-up Time, reducing employee time-to-competence through more effective training).

Consequently, if your company does not receive “Supplier Score Cards”, and management insists upon establishing aggressive (delusional/unrealistic) “Quality Objectives”, receiving additional audit time due to being classified as a “High” risk company is entirely “self-inflicted”.

Strategy
In order to meet the requirements AS9100 and realize “value” from these KPIs, management “should” input this data to a “Process Capability Analysis” to determine what their actual process is capable of achieving. These charts are normally unilateral with an automatically calculated control limit and trend line. This allows management to make better-informed decisions and even identify problems earlier by tracking trends (which may be seasonal).

Use the control limit(s) established by the “Process Capability Charts” to determine what your “quality objectives” should be (for purposes of AS9100). Improvements should be driven by data-based management - rather than emotion-based management (e.g., “pie-in-the-sky” objectives that ignore the reality reflected in the process capability charts).

In addition to the above KPIs, the OCAP requirement includes “Customer Complaints / Feedback” as requiring a “Quality Objective”.

This data is typically highly subjective (e.g., customer “perceptions”) and is much more difficult to analyze due to numerous factors. For example, some customer complaints and feedback (e.g., from “Customer-Satisfaction Surveys”) are directly related to customer-imposed restrictions on the level of control the company has over its processes (e.g., customer-mandated suppliers and/or service providers, customer-specified material/parts, fixed/frozen or customer-approved processes). In other cases, after reasonable efforts have been made to understand the customer’s requirements, the customer continues to communicate incorrectly/poorly stated requirements – primarily due to the customer’s continued inability to adequately define and/or articulate their requirements.

Upon re-visiting AS9100, sec. 9.1.2, “Customer Satisfaction” we read:

Information to be monitored and used for the evaluation of customer satisfaction shall include, but is not limited to, product and service conformity, on-time delivery performance, customer complaints, and corrective action requests. The organization shall develop and implement plans for customer satisfaction improvement that address deficiencies identified by these evaluations, and assess the effectiveness of the results.

Since customer complaints and corrective action requests are already required to be monitored, it makes more sense to establish “objectives” associated with this data rather than to initiate subjective “Customer-Satisfaction Surveys”.

Examples of potential objectives include:

Based on the concerns described above, the company should have a process for analyzing “customer complaints” to determine their validity prior to being input to the KPI.

Also, since AS9100, clause 8.5.1,c,2 allows for sampling, it is recognized that there will be an acceptable number of defects delivered (through an assigned AQL or AOQL). Consequently, the company should have a process for analyzing all “corrective action requests” received from customers to determine whether the associated nonconformity was the result of an “Assignable Cause” variation (which can be addressed through a corrective action) or was the result of a “Common Cause” variation (with NO assignable cause) in the process. Where the nonconformity was due to a “Common Cause” variation in the process, the only way to address this is through changing the process (if possible) or introducing risk controls to mitigate the probability or impact of recurring nonconformities.

AS9100 auditors are required (by SAE AS9101, "Quality Management Systems - Audit Requirements for Aviation, Space, and Defense Organizations") to complete a “Process Effectiveness Assessment Report” (PEAR) for EACH “key” process identified by the customer. The PEARs (AS9101 (Rev. F) Form 3) include a matrix (shown below) where the auditor selects a number (1-5) as an indication of the effectiveness of the process based upon whether:

• No nonconformities AND the “Quality Objective(s)” were met (scoring a “5”)
• One or more nonconformities were identified BUT the “Quality Objective(s)” were met (scoring ≤4)
• No nonconformities were identified BUT one or more “Quality Objective(s)” were NOT met (scoring ≤4)
• One or more nonconformities were identified AND one or more “Quality Objective(s)” were NOT met (scoring ≤3)

If the terminology in the PEAR is confusing (i.e., “Planned Activities” vs. “Planned Results”), this is explained in AS9101 (Rev. F) sections:

3.7 Planned Activities
The means, methods, and internal requirements by which the organization intends to achieve planned results of a given process to meet customer requirements. Planned activities include conformity to process requirements and maintained documented information.

3.8 Planned Results
The intended performance of a process as determined and measured by the organization. Planned results include product and service conformity and On-time Delivery (OTD) to meet customer requirements, and may include other elements related to the process, as defined by the organization.

As per AS9104/1A, “Table 7 - Organizational Risk Determination”, the AS9100 auditor will use the PEAR with the lowest score to calculate the “Risk Factor”.

Strategy
The strategy to have a low “Risk Factor” in this category is to ensure that ALL “Quality Objectives” (associated with PEARs) are (1) realistic and (2) consistently met. In addition, you should focus your internal audits on those processes associated with PEARs to mitigate the possibility of any non-conformities arising from those areas.