Quality Metrics Initiative: An Update

As Quality Maturity evolves based on the FDA’s metrics strategy, it was documented the agency wanted “a maximally efficient, agile, flexible, pharmaceutical manufacturing sector that reliably produces high quality drug products without extensive regulatory oversight.” Now we discuss how far has the FDA come to establishing these metrics. Susan Schniepp, Distinguished Fellow of Regulatory Compliance Associates® Inc., offers her insight and expertise on this subject.


Industry and regulatory agencies continue to make progress in establishing quality metrics for the pharmaceutical industry.


The FDA’s Janet Woodcock stated her agency wanted “a maximally efficient, agile, flexible, pharmaceutical manufacturing sector that reliably produces high quality drug products without extensive regulatory oversight”.


The groundwork to achieve this realization started with the adoption of Title VII, Sections 705 and 706 of the Food and Drug Administration Safety and Innovation Act (FDASIA). It has been almost 10 years since Dr. Woodcock revealed her vision and almost three years since the industry began to hear rumblings that FDA was going to establish quality metrics that would be used to determine the suitability of a manufacturer’s ability to provide quality products to the patient.


QA Metrics


So how far have FDA and the industry come in establishing a set of quality metrics applicable to the pharmaceutical industry? The aforementioned sections of FDASIA gave FDA the ability to establish criteria to perform risk-based inspections of bio/pharmaceutical manufacturers and to request certain documents be provided in advance or, in some cases, in lieu of inspections.


Quality Control Metrics


The FDA next published a notice in the Federal Register asking for assistance from industry in developing a strategic plan and quality metrics to prevent drug shortages. FDA’s objective seems to be to reduce companies’ regulatory burden allowing for innovation while maintaining responsibility of protecting the public health.


Quality Metrics in Manufacturing


In response to the Federal Register notice, several manufacturing organizations have made recommendations to the agency on what they perceived as fair, reasonable, and equitable quality metrics. Organizations that independently participated in the development of quality metrics include:


  • Parenteral Drug Association (PDA)
  • International Society for Pharmaceutical Engineering (ISPE)
  • Generic Pharmaceutical Association (GPhA)
  • Pharmaceutical Researchers and Manufacturers of America (PhRMA)
  • Consumer Healthcare Products Association (CHPA).


These organizations held conferences, wrote white papers, and polled their members to try and define what specific quality metrics would be appropriate for the intended purpose. Representatives from the above organizations, covering all aspects of the pharmaceutical industry, met to try and come to a consensus on exactly what quality metrics could be and how they could be used as an indicator of a company’s ability to continuously provide high quality medicine to patients.


First Pass Quality


The outcome of the meeting resulted in a set of proposed consensus metrics that included lot acceptance rate, product complaint rate, confirmed out-of-specification rate, and recall rate. The proceedings from the meeting clarified that the “consensus set of metrics are somewhat rudimentary, and provide limited information about the culture of quality at a given organization”.


According to the published proceedings, “many [participants] remarked that a strong quality culture is a critical component in driving the system and processes that underpin the quality control and assurance infrastructure at an organization. However, quality culture is also difficult to capture through metrics.”


Supplier Quality Metrics


It was at this point where industry took off in different directions; PDA and ISPE have been the most engaged organizations in subsequent efforts. ISPE designed and asked members to participate in a pilot program for quality metrics, while PDA held a conference on quality metrics that focused on how to define a mature quality culture.


The metrics used in the ISPE Quality Metrics Pilot Program include the consensus metrics from the Brookings meetings as well as others. Some of the unique metrics used by ISPE in their pilot include timeliness of annual product quality reviews, recurring deviations rate, corrective action and preventive action (CAPA) effectiveness rate, process capability, and quality culture.


Quality Assurance Measurements


A PDA quality culture metrics conference that was held focused on metrics in assessing quality assurance. The hypothesis for the meeting was that mature quality attributes have a strong relationship to positive quality culture behaviors. The consensus from the meeting was that measuring a quality culture is subjected and defined by a set of behaviors, beliefs, values, attitudes, and governance and that quality assurance goes beyond traditional quality systems in creating a framework for a strong quality culture.


Training Metrics Scorecard


PDA divided quality systems into three types:


  • Traditional
  • Enhanced
  • Other


PDA defined traditional quality systems by traditional metrics including deviations, complaints, CAPA, etc. Enhanced quality systems were defined as those that had advanced programs such as risk management, knowledge management, quality by design, quality manual, etc. The other quality systems were much more evolved and have programs like shared quality goals, rewards and recognition programs, and cost of quality awareness programs, etc.


Types of Quality Metrics


Conference attendees defined the top five mature quality attributes as the following:


  • Program to show how employee’s specific goals contribute to overall quality goals
  • Program to measure, share, and discuss product quality performance and improvement from shop floor to executive management
  • Continuous improvement program/plans with active support of CEO and corporate management of quality management systems (QMS)
  • Program that establishes quality system maturity model and action plan and tracking to measure progress
  • Internal survey measuring a company/site quality culture.


FDA Quality Metrics


Meanwhile, FDA presented the following metrics for measuring a quality system:


  • Lot acceptance rate (the number of lots rejected by the establishment in a year divided by the number of lots attempted by the same establishment in the same year)
  • Right-first-time rate (the number of lots with at least one deviation by the establishment in a year divided by the number of lots attempted by the same establishment in the same year)
  • Product quality complaint rate (the number of complaints received by the manufacturer of the product concerning any actual or potential failure of a unit of drug product to meet any of its specifications, divided by the total number of lots released by the manufacturer of the product in the same year)
  • Invalidated out-of-specification (OOS) rate (the number of OOS test results invalidated by the establishment, or contracted establishment in a year divided by the total number of tests performed by the establishment in the same year).


Future of Quality Metrics


So what is the future of quality metrics? The most important part of the puzzle is important to keep in mind that there are no perfect answers and collaboration is imperative. The concept of unintended consequences needs to be addressed so everyone is on a level playing field. Data trending can be more valuable in determining the robustness of a quality system than direct comparisons, and an open, honest quality culture will drive the integrity of the quality metrics.


Article Details



BioPharm International
Vol. 28, No. 5
Pages: 41-41


About RCA

Regulatory Compliance Associates® Inc. (RCA) provides worldwide services to the following industries for resolution of compliance and regulatory challenges:


We understand the complexities of running a life science business and possess areas of expertise that include every facet of R&D, operations, regulatory affairs, quality, and manufacturing. We are used to working on the front lines and thriving in the scrutiny of FDA-and globally-regulated companies.


As your partners, we can negotiate the potential minefield of regulatory compliance and private equity due diligence with insight, hindsight, and the clear advantage of our unique expertise and experience.

  • Founded in 2000
  • Headquartered in Southeastern Wisconsin, with offices in West Central Florida; North Central Colorado; and Central Eastern Europe
  • Expertise backed by over 500 industry subject matter experts

Our website uses cookies to give you the best possible experience.

By continuing to use this site, you agree to the use of cookies.