Supplier Quality Surveys - You're probably doing them wrong

I've seen a large number of Supplier Quality Surveys (e.g., “Supplier Evaluation Survey”) completed by suppliers (i.e., a self-assessment) and provided to potential customers as a basis for “qualifying” (i.e., “approving”) them as a supplier.

First and foremost (I must state the obvious), since these surveys are typically completed by the supplier… they're simply not objective.

Second, the VAST majority of these surveys ask questions regarding the existence of certain quality controls - and are completely worthless because the questions fail to address (1) the effectiveness of those controls, and (2) rarely require any records to be submitted supporting their effectiveness.

For example:

CORRECTIVE ACTION Yes No
1 - Is a documented process(s) maintained for the implementation of corrective actions?
2 - Does the process provide for problem analysis and determination of root cause(s)?
3 - Are controls established to ensure that corrective actions taken are effective?

You may be thinking that the above appears perfectly fine… until you consider:

  1. The “documented process(s) maintained for the implementation of corrective actions” appear only slightly better than if it had been written by a first-grader… with only the most basic understanding of corrective action (at least it wasn't written in crayon).
  2. The process for “problem analysis and determination of root cause(s)” is limited to the basic “5 Whys”… which might be fine… provided (1) personnel actually understand what the “5 Whys” is and (2) never encounter any complex cause & effect chains.
  3. The “controls established to ensure that corrective actions are effective” do not involve people capable of “critical thinking”.

The point is that a “Supplier Quality Survey” can ask all sorts of questions that are, in the end, meaningless.

So how can you improve this simple form?

A great step is to ask “quality-related” questions relating to topics that are (1) “value-added”, and (2) include “performance-related” data that reveal more about the supplier. For example, the following 3 questions will provide you with much more pertinent, and value-added information about a supplier than most 20-page “Supplier Quality Surveys”:

For Manufacturers:
What is your FPY (First-Pass Yield)?
What is your product return rate?
What is your average OTD (On-Time Delivery)?

Please attach data supporting these metrics for the past 6 months.
Also, attach details of any activities taking place to improve these metrics.

Suppose that two manufacturing suppliers met all of the detailed “generic” requirements in a “Supplier Quality Survey”… and both are either ISO 9001 or AS9100 certified. However, upon reviewing the above set of questions, the first supplier reportedly achieves 65% FPY, 5% returns, and 55% OTD, with no meaningful improvement activities planned (only corrective actions - which are entirely “reactive”). The second supplier reportedly achieves 96% FPY, 1% Returns, and 90% OTD, and they have meaningful improvement activities in-process (e.g., Kaizens, DMAICs).

Without having asked the above questions, these two suppliers would have “appeared” equal, when they clearly are not.