Process Auditing: A Change in How Auditors Work

ISO/TS 16949 and ISO 9001 quality management systems

by Robert Kozak

Adding value by auditing using the process- and metric-driven approach requires new methods and an increased focus on supplier performance metrics (i.e., scorecards). A change in how auditors work the process approach is evolving, with more sophisticated techniques, more rigorous testing and increased objectivity. In today’s competitive economy, all parties involved in audit processes need to be knowledgeable in process auditing methods to get more value. Balanced feedback in audit reporting helps drive improvement in manufacturing and in its support processes. The key question remains: How can we prove the effectiveness of the system? Process- and metric-driven systems are validated using various indicators.

Indicators of implementation and effectiveness:

  • Delivered part quality performance
  • Customer disruptions, including field returns
  • Delivery performance, including incidents of premium freight
  • Customer notifications related to quality or delivery issues

The process principles described in ISO 9001:2000 and ISO/TS 16949 are powerful tools for organizations to focus on results that affect the bottom line as well as customer satisfaction metrics. W. Edwards Deming said many times, “There’s no substitute for knowledge.” Proper system design requires in-depth knowledge of the processes to optimize planned outcomes. The experts for this process design are the members of your own management team. These process owners are a critical link to meeting business objectives and are also the key in the audit process.

A quick review of process principle basics:

  • All activities are linked as sequential steps.
  • Change or transformation takes place in processes.
  • The law of conservation is applied to defined processes.
  • Optimization results in better utilization of resources.
  • Customer expectations affect the context of all processes.
  • Improvement in processes is a key factor.
  • Attention to the people involved in the process is essential.

The selection of people for interfacing with the external audit team shouldn’t be taken lightly. The process owners are key contacts, as are the other people in their teams who will be included as auditees. Examples of guides not suited to help in the audit process include new hires who don’t know the ropes, extremely defensive individuals and disgruntled employees. Considering those individuals who are internal auditors and understand the system can really help your audit team be effective and efficient.

While there is no requirement in ISO 9001 to conduct process audits, ISO/TS 16949

requires audits for each manufacturing process to determine effectiveness. The aim of ISO/TS 16949 places emphasis on defect prevention and the reduction of variation and waste in the supply chain. During process audits, we constantly test the “what if” scenarios to identify probable failure within the given context and situation. A good auditor always has a mindset to identify things that could go wrong. Process-oriented auditing requires some of the risk-based methods shown below:

  • Take an objective approach based on facts (not nit-picking).
  • Verify current customer scorecards and employ the “know how” for interpretation.
  • Verify that control plans are being followed and are effective.
  • Focus on the expected outcomes and objectives (KPIs).
  • Follow-up on corrective and effective preventive countermeasures.
  • Interview of all levels and functions including top management.
  • Prioritize the most customer-disruptive complaints, greatest number of recalls and highest cost complaints.
  • Think of the weakest link and the use of the process metrics to guide the audit.
  • Specify commodity experience in auditing key processes such as manufacturing, test and design.
  • Plan the audit based on risks to the customer and the organization.
  • Interview process owners and people that carry out the processes.
  • Use the 3D (do, document, demonstrate) method and the PDCA analysis method.
  • Pose reasonable worst-case and failure scenarios to test the process.
  • Use sophisticated sampling techniques such as block and selective sampling as well as scientific hypothesis testing.
  • Undertake a thorough review of PPAP files with correlation to design records and linkage to FMEAs, control plans, process flow, MSAs, capability studies and master samples.

The International Automotive Oversight Bureau’s latest sanctioned interpretation for process-based auditing lays out the requirements for registrar auditors. For starters, any auditor must be able to comprehend the process approach and process maps of the company being audited using the process maps/landscapes as guidance, and using them to identify and verify inputs and outputs of the process including testing of the controls.

Audit planning must include certain activities:

The analysis of processes: This includes looking for any problem areas with a starting point for the audit using supplier scorecards, customer complaint data, warranty information, spills, disruptions and field recalls. In particular, this means checking if there is any special status such as Q1 revocation, needs improvement, new business hold and CSII, key process indicators (KPIs), high scrap, rework and other indicators of poor trends in quality, waste and variation. Focused audit trails should be developed based on weak areas or the identification of poor trends. For example, if a poor trend in welding is identified, more time should be allocated toward the welding process. If the company has any special status conditions, this may indicate a lack of readiness for the audit. The process sequence and identification map is also reviewed to confirm the inclusion of interfaces as well as methods to control, monitor and measure the processes. The six Ps of process auditing are: proper previous planning prevents poor performance. Scope is another factor that should be considered, for example, employee count, shift patterns, remote support locations and functions, and whether or not product design is applicable.

Products and/or services provided to the customer: This includes analysis of the commodity supplied, process knowledge regarding that product, and if there are any special characteristics, safety items and services provided. The audit team will ask to see scorecards for all commodities supplied. Product knowledge in the commodity supplied aids the auditor in allocating sufficient time in the audit to key processes, which typically include design, test and manufacture, depending on the scope of the organization. Multiple product lines may require selection based on new applications, technology and process capability studies. Auditing the lines with lower Cpks may provide a better test of the system to ensure defective parts are contained and that corrective actions are effective.

Interfaces (inputs/outputs): Upstream and downstream suppliers are critical to any process. Consider a manufacturing site that’s part of a large design-responsible organization. The design interface may be with a remote design location. Typical inputs and outputs, in this case, are engineering changes, request for engineering changes, drawings, specifications, FMEAs, control plans and regular technical communications. Coordination with other processes such as sales, customer service, labs, maintenance and shipping are some other interfaces that are considered when auditing various processes. Auditors aren’t just interested in the middle of the road; they also verify the supply and delivery sides of the process. These interfaces should be defined and audited for their inputs and outputs, and should be checked against the KPIs that apply.

Risks to the customer: Spills, high parts per million, poor delivery trends, recalls, disruptions and high warranty costs are all aspects to look for in process auditing. A thorough review of the readiness review information should be performed. The customer complaint status and history are very important to verify trends, and to look for repeated incidents and prompt closure. Another indicator may be Cpk data, especially any with less than 1.33. Risks are then used to plan the audit and develop specific audit trails based on those processes, considering both upstream and downstream interfaces. The FMEAs also identify the highest risks to the customer, including the end-users who are the occupants of the vehicle. Consider priority sampling for those products that have safety-related characteristics (FMVSS, EC, ECE, JIS) such as brakes, tires, lighting, glass, airbags, seating, hood latches, steering, fuel systems and other components that affect safety and occupant protection.

For example, trends in complaints were noted with regard to contamination issues. This prompted audit trails into manufacturing and maintenance as new antistatic devices had been introduced in the manufacturing lines. Based on my previous experience with contamination controls, I know that this type of de-ionizing device has a limited life expectancy, but it’s impossible to tell just by looking at them if they are de-ionizing. The manuals on these devices require periodic testing to verify that the antistatic devices are functioning properly. In this case, the supplier hadn’t taken the time to read the manuals and hadn’t done the periodic tests to ensure the contamination controls devices were functioning properly. The preventive maintenance database didn’t specify these checks, either.

The same logic applies to error-proofing devices. Without frequent verification of

“golden samples” (i.e., known good and known bad), it’s possible to be “running blind” with a process and inadvertently pass nonconforming parts. Another common test auditors may select is to have a known bad part run through the process to ensure the error-proofing device is properly functioning.

Possibilities to combine processes for economical and effective audit trails: Logistics for planning the audit can help, so a site map or facility plan may be a tool used to help identify specific processes. The best practice is to identify the location of the labs, incoming and outgoing flow, and in particular, where all rework/nonconforming material areas are located. “Hotspots” and key controls in the system can be visually identified and keyed to make the audit logistics more efficient and add confidence that these areas are sampled in the audit. A site plan can be a useful tool for communicating these “hotspots” to the audit team, which may not be familiar with the facility layout.

Interfaces such as sales and design may be audited by the same auditor to follow specific projects/products sampled throughout the system. Purchasing process and receiving inspection processes typically have an interface regarding nonconforming material and corrective action. Auditing the various shifts allows auditors to cross check and validate customer corrective actions, mistake-proofing devices and process control plan adherence, as done with previous shifts. Typically, new people are on the third shift. This may be a good time to test the effectiveness of training and inquire with new operators about their training with respect to making and checking for good parts.

Prioritization of audit processes to consider:

Follow up issues of prior audits (internal and external): Root cause identification and lack of systemic corrective action has been identified by the IAOB as another weakness in ISO/TS 16949 implementation. Corrective actions are scrutinized for root cause(s) and systemic corrective action. On-site audit closure and verification is a valid and effective option, especially when the corrective action to the audit appears weak. If major nonconformities and/or customer special status conditions exist, this typically requires on-site verification. In fact, for all periodic surveillance audits additional time will be added to the audit plan to verify corrective actions from the last audit.

The selection of any major nonconformities and the verification of product- and system-related corrective actions is a method to prioritize which findings to verify first. It’s typical to verify the corrective action not only on paper, but also against a new project to interview people and to test the effectiveness in the workplace.

Customer audits may have also occurred and are subject to verification for any corrective actions and results. For ISO/TS 16949, in particular, corrective actions for product, process, and internal systems audits are viewed to determine trends and possible deficiencies in the system. Audit schedules showing an increase after customer complaints is a typical verification. The training of internal auditors and linkage to any customer-specified training needs are reviewed.

Customer complaints and countermeasures: Some of my best insights into an organization’s weakness, and effective development of audit plans and methods, come from a close examination of these sort of data. This is where selective sampling is in order based on risk priority. This is also where many systems fail to be effective based on repeated complaints. A large sample of selected corrective action records allows the audit team to have the details to confirm any and all countermeasures on significant complaints and systemic failures.

Allocation of time to verify on the shop floor and across all shifts is warranted to ensure the effectiveness of corrections made to the process. Special processes such as heat treat, welding, painting and plating are of great interest to confirm corrective actions due to their nature and many critical applications.

Value-added activities are more than just conformance to the specification. If the audit finds possibilities for improved defect reduction, excessive variation, waste and inconsistency in processes, the possibility for value is realized. Many times cost of quality isn’t calculated for internal costs. Value depends on the organization’s expectations, cost-saving suggestion or return on investment. Common sense and practical findings may be of value to some, while an objective approach may be of value to others. The way that auditors interact with people to put them at ease and use positive reinforcement to counteract the negative tendencies may also be seen as a value. A good process auditor will try to inquire about value, expectations and what the audited organization expects.

Finalization of the audit plan including the processes, sequence, timing and interviewees should be prepared well in advance of the audit to ensure all processes are covered. This plan should then be distributed to all process owners and include interfaces with any remote site support locations.

Reporting for process-based auditing requires a balanced approach that covers not only nonconformities, but also significant positive findings and opportunities for improvement. Examples of positive findings include customer metrics, internal metrics, awards, investments in mistake proofing, improvement trends and positive reinforcement. Opportunities consider waste reduction, defect prevention and variation reduction strategies to bring value to the equation. Nonconformities based on solid objective evidence allow the requirement, source and specific evidence to be demonstrated in concrete terms for solid understanding by all parties.

Comparison of performance to objectives is accomplished by examination and interviewing. If the process is falling short, what is being done to correct, add resources or shore it up? Tracing the steps in the process by using selected sample products and orders allows conclusions to be drawn as to whether planned results are achieved and whether the controls are effective. Larger sample sizes are typically requested when in doubt. Auditors no longer rely on the glossy red, yellow and green trend charts alone. Actual verification of monitoring and accuracy of measurement is scrutinized and verified. Current customer scorecards are discussed to ensure that any disputes are known so that proper interpretation is achieved.

The weakest-link method applied to process auditing helps identify system deficiencies and possibly exposes the root causes if enough details are identified in the finding. It’s a best practice to obtain copies of objective evidence to help explain nonconformities.

Weakest link examples include:

  • Top three worst suppliers
  • Top ten worst process capability features
  • Gages that failed calibration and highest R&R percentage on MSAs
  • New programs that are late and have many late engineering changes
  • Most complex build
  • New and/or transferred employees
  • Oldest machines or those that most frequently break down
  • Toughest material to process
  • Worst environmental conditions for processing
  • Poorest tooling
  • Overdue reactive and preventive maintenance
  • Overdue engineering changes
  • Bottlenecks
  • Repeated customer complaints and ineffective corrective action

Selective sampling utilizing the weakest link scenario will most likely yield some findings. This method of testing is focused on proving that the system works to control the weak links.

Control testing by tracing the process steps can reveal fundamental flaws in a process.

Verification of proper methods, settings, operator craftsmanship, error-proofing devices, and other inputs are tested to ensure a defect-free product. In addition, work instructions and problem-solving methods are reviewed to determine whether process issues are identified and quickly corrected. A deep dive into SPC charts can usually yield out-of-control process events. A block sample might involve one or two months worth of data.

PFMEAs, control plans and work instructions create the technical memory, analogous to a recipe to bake a cake. Process audits verify the recipe, methods and controls to address the proper build of a product and that the documentation is clearly in-sync with it. A common finding is that control plans don’t match actual practice. With layered process audits, these are conducted by all levels of supervision, including plant management, on a regular basis. Layered process audits are a DaimlerChrysler customer specific requirement as of July 2004.

Benefits of process audits include:

  • Reducing build variation based on work standardization
  • Minimizing the amount of part defects
  • Providing management interface with operators
  • Reducing end-of-line inspection
  • Improving morale
  • Reinforcing safety regulations

With all these process audits, poka yokes and advanced product quality planning techniques, spills and recalls should be optimized. Expect your TS audit team to have more sophisticated methods of auditing. As management systems mature so do the techniques required to test the systems. The challenge remains to up the bar on defect prevention and reduce waste and variation in the supply chain. Four steps that can help you meet this challenge follow:

  • Send your management team and auditors to an ISO/TS16949 training.
  • Have your management team perform regular process audits and layered process audits.
  • Team up your internal auditors as guides for your external audits.
  • Stay on top of supplier scorecards as a focus to guide your own process audits.

About the author
Bob Kozak is a technical specialist/lead auditor with DNV Certification and has more than 20 years of experience in the first-tier automotive industry including lighting, interior/exterior trim, electro-mechanical products and mirrors. He has worked on project in Japan, Taiwan and Europe and has held various positions including quality manager, process engineer, facilities/EHS manager, lab manager and technical manager. Kozak is currently pursuing his M.B.A. and frequently provides lectures, workshops and training seminars for the industry. Kozak has authored award winning training courses, numerous articles and publications and is a co-author of Safety Management and ISO/QS 9000 (Quality Resources, 1995).

Click here to return to InsideQuality home page.

Copyright © 2004 QCI International. All rights reserved.
Quality Digest can be reached by phone at (530) 893-4095.
Contact us via Contact Form