The Internal Quality Audit Process for Orthopaedic Medical Device Companies: Risk Indicators and Uncertainties

FDA analysis of factory inspections has shown that manufacturers who do not have an adequate quality audit system usually do not have an adequate quality system. An evaluation of approximately 2,400 manufacturers that received cGMP-QSR inspections by FDA showed that manufacturers with an adequate quality audit system were in compliance with approximately 96 percent of the GMP requirements, while those that did not have an adequate audit system were in compliance with approximately 70 percent of the requirements.

Definitions

Audit (from cGMP-QSR):A systematic and independent examination of a manufacturer’s quality system that is performed at defined intervals and at sufficient frequency to determine whether both quality system activities and the results of such activities comply with quality system procedures, that these procedures are implemented effectively, and that these procedures are suitable to achieve quality system objectives.

Audit (from ISO):A systematic and independent examination to determine whether quality activities and related results comply with planned arrangements, and whether these arrangements are implemented effectively and are suitable to achieve objectives

Risk: A combination of the probability of occurrence of harm and the severity of that harm.

Notes on Affective Uncertainty: The effectiveness and compliance of a quality management system can be jeopardized by not enabling compliance through the use of the process approach, not understanding the operation and control of these internal steps, having incompetent individuals design and implement these procedures, by not measuring, monitoring and analyzing these processes in an honest and viable manner and by not listening to your internal and external customers.

A Useful Tool in a Risk-based Industry

The analysis and use of feedback data from product acceptance, audits, complaints, repairs and other sources are necessary parts of a self-correcting quality system. The audit of a quality system is one of the most important cGMP-QSR requirements. A quality system should change with the risks presented to an orthopaedic device company. Therefore, internal audits are the primary tool for assuring that quality system changes are not only correct, but also disinclined to present more risk when implemented and linked with other processes.

A quality audit is a documented, independent inspection and review of a quality system. The audit is performed on a periodic basis in accordance with written procedures. The objective is to verify, by examination and evaluation of objective evidence, the actual degree of compliance with those elements of the quality system under review. These audits are an essential part of every medical device manufacturer's effort to assure safe and effective devices. Regardless of how well a quality system is planned, monitoring of the system is required if the quality system program is to be effective in assuring that finished devices meet specifications.

If conducted properly, a quality audit can detect quality system defects. Isolation of high risk trends and correction of factors that cause defective products can help prevent the production of unsafe or nonconforming devices.


The Basic Audit Process Points for Consistency (and To Minimize Risk)

A. AUDIT PREPARATION - The Quality Auditor reviews applicable change control records subsequent to a design transfer, any FDA clearance delay information, recall records, standard manufacturing procedures, device histories, complaint history, device labels and inserts, previous audits with results, follow-up audits, plus any other document relative to the audit.

B. AUDIT INITIATION - The Quality Auditor prepares/updates an audit checklist for systematic examination of the area to be audited, informs the Manager of the department being audited at the start of the audit, and reviews any observations.

C. AUDIT ANALYSIS -The Quality Auditor reviews the data gathered, verifies important details and writes an audit report according to the format delineated in the internal audit work instruction.

D. ISSUANCE OF AUDIT REPORT - The Quality Auditor issues the written audit report to Management with Executive Responsibility. Audit reports shall be stamped "Confidential.”

E. CORRECTIVE ACTION - The appropriate process owner shall be responsible for developing a schedule for correcting deficiencies cited in the audit report and submitting same within five working days to the Quality Auditor. Included in the correction schedule shall be the responsible individual, and the date by which corrective action will be completed.

F. AUDIT FOLLOW-UP -The Quality Auditor maintains a log listing deficiencies, responsible individual, target date for corrective action and actual date of correction. If the same deficiency occurs on a second follow-up audit, Management with Executive Responsibility shall be notified in writing by the Quality Auditor.

Setting the Risk-based Stage

The popularity and usefulness of ISO 14971:2007 (a guideline for risk management) is geometrically increasing in the medical device industry. Historically, risk was primarily relegated to the design process when analyzing whether the medical device conformed to defined user needs and intended uses during design validation. Risk management can be (and is) used during at least the product realization phases, and is linked to other aspects of a total quality system, e.g., training, auditing, complaint handling, corrective actions, servicing, customer feedback, etc.

With the advent of the cGMP-QS Regulation and ISO 13485:2003, all processes within this Regulation and Standard, respectively, “should be considered as to how they provide input to, or benefit from the results of, risk management activities.” That being said, there is a natural and direct linkage with measuring, monitoring and reviewing the suitability and effectiveness of the quality management system (at defined intervals). Because there are always related uncertainties in every quality system of every medical device company, the use of the “auditing tool” is not only required. but an important way to:

  • measure the effectiveness of the quality system
  • provide objective evidence that adequate controls are in place
  • assure that products and processes conform with specifications

Reducing Uncertainties in the Internal Audit Process:

(Lack of) Auditor Independence and Competence

A basic annoyance of every audit process is having auditors who are not qualified to conduct and implement the requirements for auditing…and just because they have passed a test to become certified doesn’t mean that they can effectively audit. Competent auditors are a rare breed of experienced individuals with excellent people skills, knowledge of investigative techniques, an empirically-driven approach using consistency and objectivity without compromising logic and years of hands-on experience. Using incompetent auditors increases the uncertainty of your company’s use of this measurement tool. An auditor could understand the process steps of conducting an audit, but not have an understanding of process orientation, systems-based architecture and the many nuances of the Regulation and Standard. In my audit classes, I commonly spend at least the first day teaching future auditors how the architecture of the GMPs and ISO works. Without this knowledge of process linkages, discovering objective evidence and the “in-between-the lines” compliance requirements, the auditor can only profess to be an auditor. The uncertainty of accurate results and a thorough output directly increases the risk of having a “false sense” of meeting the requirements.

Auditors not only have to be competent and experienced, but must be free from bias and the influences which could affect objectivity and basic compliance. If the auditor is officially part of the process or system being audited, this could also lead to skewed results. Auditors should also be quietly observant, inherently honest, have an investigative mind, possess people skills related to “care-frontational” interviews and audit communication, be adaptive and cooperative without jeopardizing thoroughness and require much needed organizational skills.

The requirement for an independent audit should generally be met; however, if a very small manufacturer, particularly one at which everyone is directly involved in daily design and production activities, concludes that independent audits would be unduly burdensome or impractical, then the requirement for independence may be waived. However, if FDA finds, as a result of inspection or other means, that this waiver has compromised the quality system, FDA may require an independent audit, increase the frequency of FDA cGMP-QSR inspections or take other appropriate regulatory action.

The Audit Schedule and Planning

Companies must realize that conducting effective and timely quality audits is critical to success. Without the feedback from the quality audit, device companies will be forced to operate in an open loop system with no assurance that the process used to design and produce devices is operating in a state of control. The industry standard for auditing at defined and planned intervals should be related to the type and classification of the medical devices being designed and produced, the size and complexity of the company and the company’s “track-record” for product non-conformances, types and quantity of complaints, the success of corrective actions and, generally, the maturity of the quality management system. Commonly, companies with compliance issues, immature systems and highly scrutinized outputs should audit more frequently to measure effectiveness and improve their compliance posture.

Generally, the audit schedule will direct the medical device company to perform an audit of each process in the quality management system at least once per year, so that compliance trends can be realized and brought to management’s attention. If there are compliance and process uncertainties, failures and questionable anomalies within the framework of the quality system, then the frequency of performing audits should be increasingly redefined to accommodate timely and effective corrective actions and improvement. Audits may “trigger” additional audits to further define and check the “health” of all quality processes. If audit findings indicate the potential for similar findings in other linked (or not) activities, common audit practices require a revision to the present schedule to assess possible non-conformances and non-compliances.


A stagnant and unchanging audit schedule could mean that your quality management system is perfect, or that you’re looking at the same things over and over again without a true return on investment. Uncertainty comes with complacency and the lack of true investigative initiative. Complacency will lull your company into a false sense of security and mutually-admired compliance. The risks, of course, lie quietly awaiting the next FDA inspection or ISO audit, i.e. Top Management might say during a typical FDA 483 discussion, “We’ve never seen that before.”

Some Causes for an Inept Quality Management System That Lead to Findings: Some Uncertainties That Auditors Look For

When performing audits (which is part of what I do) at a variety of companies with divergent core competencies and final outcomes, the risks involved with uncertain process compliance can present in myriad ways.

Usually the foremost and most revealing cause of compliance issues is not having “true” management support that must start at the top and affectively “spill” into the rest of the organization. A jury-rigged infrastructure that has human resource issues related to a general lack of competency is one part of the outcome when commitment is just “a rubber stamp.” The other is the use of quality metrics that are not linked to business strategy and a “real” quality policy and quality objectives.

Auditors should look for symptoms belying an overall lack of discipline, over-worked personnel without leadership and a general lack of teamwork and true process orientation. Non-robust training initiatives tied to this lack of authority commonly generate objective evidence that is inconsistent, barely indicative of compliance and unplanned. These are just a few of the indicators that can increase the uncertain nature of the quality management system and have a direct and negative effect on both product and process defectives. Auditors who are not cognizant of how these types of risks portray uncertain behavior will simply audit departments and not the process architecture.

Below are just a few “uncertainty” indicators that I use to initialize an audit trail.

  • Top management doesn’t consistently attend management reviews
  • Internal audits are either late or completely missed
  • Corrective actions take an inordinate length of time to close
  • Corrective actions are not closed out based upon the effectiveness of actions taken
  • The Approved Supplier List is primarily populated with suppliers that were “grandfathered in” and not truly evaluated and monitored for performance
  • The quality policy hasn’t changed in years
  • High incidences of product non-conformances that are also found by customers (in the field)
  • Supplier performance is not linked to incoming acceptance activities
  • The ISO auditor never finds any of these issues
  • Change control is out of control
  • The company is an inspection-based company rather than a validation-based company
  • Sampling plans are not based upon statistical rationale or history
  • Service reports don’t include a check box that asks if this issue could be a complaint

Audit Certification during High Risk Occurrences

Under 704(e) of the FD and C Act, FDA has authority to review and copy all records required by the QS regulation; however, FDA has elected not to review audit reports. The exception [820.180(c)] to FDA's policy of not seeking access to reports of audits of quality systems is that FDA may seek production of these reports in litigation under applicable procedural rules, along with other confidential documents. Thus, a copy of the current audit report should be maintained by the manufacturer. FDA policy was established because the agency does not wish to prejudice audits by having auditors concerned that their comments will be reviewed by FDA investigators. Although FDA investigators do not have routine access to audit reports, they can request, if the risk presents itself, manufacturers to certify that audits have been conducted and the results documented; however, investigators do not routinely request certification. If requested, an employee in management with executive responsibility should certify, in writing, that the manufacturer has complied with the audit requirements of the QS regulation.


Usually, investigators will ask questions regarding the audit report such as:

  • Who prepared the report?
  • What does the quality system audit include?
  • When was the report written?
  • Using the checklist, how should the audit be conducted?
  • Who reviewed the information and wrote the report?
  • Were corrective action and re-audit(s) taken based on the audit result?

If investigators suspect that audits are not being conducted, questions to determine consistency in answers may be addressed to those individuals who should have reviewed these reports. FDA investigators will routinely review audit procedures and audit checklists.

The Audit Reporting Mechanisms (and the General Lack of Responsiveness)

The audit report should be written to provide the auditee with a permanent record of the audit conducted and information upon which to base corrective action and improve the quality management system. The audit report should accurately reflect the intent and content of the audit. The following categories should be addressed in a basic report with clear and accurate language.

  • The scope and objectives of the audit
  • Details of the audit plan
  • Identification of the audit criteria against which the audit was conducted
  • Identification of non-conformities

1.  Details of each non-conformity written in concise and understandable language; know that the auditee will have to understand the defective so that corrective action is approached in a “straight-up” manner

2. The audit criterion or regulatory requirements, so that the auditee can make reference to same while constructing corrective action plans

3. Based upon the risk to the quality management system or the device(s) involved, give each finding a defined severity rating, e.g. critical, major, minor, recommendation, etc.

4.  The dates of submission for any corrective actions, as necessary

  • The effectiveness of the auditee’s quality system in meeting the quality objectives
  • Details of documents and records that were reviewed
  • Recommendations for follow-up actions, as necessary

The auditor is responsible for following up on the corrective actions until closure is proposed by the auditee. At that point, the auditor will review the data and pointed objective evidence to determine if the corrective actions that were taken are effective. This I find is the highest risk, post audit: planning for corrective actions, implementing these actions and then following up to realize full effectiveness. The following post-audit items present the highest risks to companies when they are not appropriately addressed.

  • Corrective actions last far too long without documented interim points of completion
  • The root cause is not disseminated, because the defective process point is not well understood
  • Too many defective processes are addressed solely on training anomalies
  • The audit report is not issued within a week or two, thereby making it hard for the auditee and auditor to understand what the issues were. My rule of thumb is that audit reports are sent to the auditee on or before 48 hours of conducting the audit.
  • The audit report represents another agenda (besides reporting the results of process non-conformances)
  • Audit findings are closed out prematurely
  • Top-level management is not apprised of te findings from audits conducted

Auditing is a tool that has been around long before a regulated industry was conceived. The audit has a purpose that is aligned with compliance to Standards and Regulations, improvement and assuring that procedures are valid and viable for companies manufacturing safe and effective medical devices.

Full quality system audits are required to make certain that:

  • The established quality system is adequate for producing devices that consistently meet the device master record requirements
  • All system requirements are being met
  • The system will continue to function when new products are introduced, changes are made and the workforce is understaffed, i.e. the risks are escalating

Mitigating uncertainty is achieved by using the process approach to corrective actions and improving the quality management system based upon the needs of customers (the end-user and regulatory bodies) and the demands of a dynamic business strategy.

John Gagliardi has had success over the past 40 years in the Medical Device and Pharmaceutical industries because of his practical approach to process-orientation and business. He has been actively involved in research and development, quality assurance, training, operations, process architecture, FDA inspections and regulatory affairs. John specializes in building systems in a compliant and business-ready manner. Email John at JGAGL777@One.Net.

MidWest Process Innovation, LLC
513-573-0085 (phone)
www.midwestprocessinnovation.com