Quality Metrics

Susan Schniepp, Distinguished Fellow at Regulatory Compliance Associates®, discusses the quality metrics journey in determining the suitability of pharmaceutical products.


quality metrics


One of the most discussed and debated topics on today’s pharmaceutical landscape is the issue of quality metrics. Establishing, maintaining, and interpreting quality metrics to determine the suitability of pharmaceutical products has become a high priority for the FDA. To understand the issue of quality metrics it is important to start at the beginning.




The first stop on this journey took place in 2012 when Congress passed the Food Drug Administration Safety and Innovation Act (FDASIA) which enhanced FDA’s capability to proactively react to, prevent, and alleviate drug shortages. This direction is embedded in the language contained in Title VII—Drug Supply Chain and Title X—Drug Shortages.


Specifically, Title VII Section 705 of the Act states FDA:


“…shall inspect establishments described in paragraph that are engaged in the manufacture, preparation, propagation, compounding, or processing of a drug or drugs (referred to in this subsection as ‘drug establishments’) in accordance with a risk-based schedule established by the Secretary”.


QA Risk Factors


This section also describes the risk factors considered in establishing the inspection schedule. One of the risk factors listed in this section is:


“Any other criteria deemed necessary and appropriate by the Secretary for purposes of allocating inspection resources.”


Section 706 of the same act authorizes FDA to request certain company information in advance of or in lieu of inspections by stating


“Any records or other information that the Secretary may inspect under this section from a person that owns or operates an establishment that is engaged in the manufacture, preparation, propagation, compounding, or processing of a drug shall, upon the request of the Secretary, be provided to the Secretary by such person, in advance of or in lieu of an inspection…”


In addition, Title X section 506C-1 (Annual Reporting on Drug Shortages). Title X section 506C-1 requires FDA to annually provide Congress “a report on drug shortages…”


Quality Metrics Examples


The second part of the journey occurred in the Federal Register Notice. In that notice, FDA asked the industry to:


“assist the Food and Drug Administration in drafting a strategic plan on drug shortages as required by the Food and Drug Administration Safety and Innovation Act…”


This notice asked a series of thought-provoking questions about more specific quality metrics including:


“What metrics do manufacturers currently use to monitor production quality?” and “How frequently would such metrics need to be updated to be meaningful?”.


The industry reaction to this information was varied. Many trade organizations responded to the questions in the Federal Register. Some prepared white papers while others held meetings to discuss the issue with their members. The general consensus was that industry needed to seriously engage with FDA to define which quality metrics would provide information to the agency supporting efforts to eliminate drug shortages and to be used in establishing a risk-based approach to inspections.


FDA Quality Metrics


The third leg of the journey was the issuance of a draft guidance titled Request for Quality Metrics. In between the issuance of the Federal Register and the release of this original draft guidance there were many industry comments submitted to the agency suggesting various metrics that might be employed by the agency.


In addition, industry trade organizations held conferences and seminars to discuss the issue with the stakeholders and agency representatives. A number of these trade organizations published white papers based on the proceedings from these conferences and put forth the position that the need to submit metrics to the agency was redundant because of the requirement to submit annual product reviews. The agency listened to industry and some of the feedback was incorporated into the guidance.


Quality Performance Indicators


The quality performance indicators that were chosen as metrics for the updated guidance include:


  • Lot Acceptance Rate
  • Product Quality Complaint Rate
  • Invalidated OOS Rate
  • Annual Product Review
  • Product Quality Review
  • On Time Rate


The guidance also contained three optional metrics intended to measure quality culture:


  • Measuring Senior Management Engagement
  • CAPA Effectiveness
  • Process Capability/Performance


This version of the quality metrics guidance generated 83 comments from industry submitted to the docket. A number of these comments focused on the practicality of submitting the requested metrics. In response to this concern the agency issued the Quality Metrics Technical Conformance Guide: Technical Specifications Document that explained how companies would be expected to submit the data to the agency.


Quality Management


The fourth part of our journey came when the second version of the guidance, titled Submission of Quality Metrics Data Guidance for Industry was released for comment. The docket for commenting on this most recent version of the guidance closed. There were 25 comments submitted. 12 from trade associations, 10 from individual firms, one from a hospital group, one from an academic institution and one from a private citizen.


The guidance clearly stated the


“…metrics can be also useful to FDA: to help develop compliance and inspection policies and practices…. improve the agency’s ability to predict, and therefore, possibly mitigate future drug shortages; and to encourage the pharmaceutical industry to implement state-of-the-art, innovative quality management systems for pharmaceutical manufacturing.”


Quality Metrics


The FDA also clarified the goal of the metrics program by stating


“As described in this guidance, FDA is initiating a voluntary reporting phase of the FDA quality metrics reporting program.”


This current version of the document requires companies to report on only three metrics:


  • Lot Acceptance Rate
  • Product Quality Complaint Rate
  • Invalidated OOS Rate


The optional metrics intended to measure quality culture were removed.


Total Quality Management


These proposed metrics are not new to the pharmaceutical industry. Many of them are currently being used by companies to measure their total quality management performance. In order to be successful, companies need to review and analyze the information the FDA is asking for, as well as other metrics they are collecting, and identify potential problem signals. Quality leadership teams should look to solve issues and self-correct before regulatory inspections occur.


The problem with the proposed metrics, however, is that they are lagging indicators of performance. There is no set requirement on which metrics a company should track to measure their overall performance. Each company should determine which metrics to track based on their operations, number of facilities they operate and where they are located, what types of products they manufacture, and what type of culture exists in their places of business. Any metrics chosen must be meaningful and written to provide a clear analysis of ongoing activities.


Quality Assurance Management


It is important for operations and quality to agree on any type of quality assurance metrics. Additionally, how to report the results to management is as important as the results themselves. The interpretation of the data is a crucial element because it may include a root-cause analysis of its own and may help to promote continuous process improvements.


When choosing a metric, it is important that the architects of the QA metric are aware of consequences that may inadvertently drive negative behavior. Management attempting to incentivize achievement of the goal such as offering a financial award if the goal is achieved may lead to inappropriate behaviors that do not address the real issue. In these cases, it is generally not the metric that will drive the behavior but rather use of behavioral rewards. Reward for achievement rather than analysis of the real underlying causes will not lead to sustainable positive change.


Quality Control Management 


When managed properly, metrics are an important tool to drive positive design controls and process improvements. Regardless of what metrics a company chooses to measure their performance, achieving a quality culture is important in assuring reported quality control metrics are accurate and reliable. A quality culture requires management and employees to establish an environment where responsibility, accountability, and reliability are paramount. This helps each person deliver a high-quality product to the customer and sustained performance over time.


Management must educate employees and provide the tools and environment where they can perform their functions in an atmosphere that encourages excellence and continuous improvement. Continuous improvement programs are, in fact, reliable indicators of the presence of a quality culture. Where the industry’s and the agency’s destination on this journey ends is yet to be seen. What is evident is that companies must have a robust quality metrics program so they may continue to supply quality products without interruptions to patients.


Planning Quality Management


Companies need to keep in mind that when planning a quality management program, they should evaluate numerous data input points. This would include, but not limited to:


  • Product-quality attributes
  • Manufacturing site performance
  • People metrics
  • Quality-system metrics


For product-quality metrics, companies should consider reporting on batch-specific data such as trending drug product, drug substance, and stability test results against customer complaint rates.


Internal Audit


Indirect product-quality metrics could include environmental monitoring, water trend results, and yield rates. When establishing site metrics, the company could look at inspection history including internal audit findings and maintenance history such as equipment age versus defect-failure rates.


People metrics should consider ongoing job-specific training and education, skills and experience assessments, and employee turnover rate by job function and site.


Root Cause


Quality systems metrics might look at change control, investigation root-cause trends, and release-testing cycle times. Companies that can successfully establish a robust metrics program that helps them continuously improve should have no trouble meeting the FDA metrics requirements.


About RCA’s Pharmaceutical Consulting Services 


Regulatory Compliance Associates (RCA) has helped thousands of pharmaceutical companies meet regulatory, compliance, quality assurance, and remediation challenges. With more than 20 years of experience with FDA, Health Canada, EU and global regulatory agencies worldwide, Regulatory Compliance Associates® offers leading pharmaceutical consultants. We’re one of the few pharma consulting companies that can help you navigate the challenges associated with industry regulations.


Our pharmaceutical consulting firm includes over 500 seasoned FDA, Health Canada & EU compliance consultants and regulatory affairs experts who understand industry complexities. It’s a pharma consultancy founded by regulatory compliance executives from the pharmaceutical industry. Every pharmaceutical industry consultant on the Regulatory Compliance Associates team knows the unique inner workings of the regulatory process. 


Client Solutions


Whether you’re in the product planning, development or pharmaceutical lifecycle management stage or need a remediation strategy for a compliance crisis, Regulatory Compliance Associates will guide you through every pharmaceutical consulting step of the regulatory process and create a customized approach depending on your product and your pharma company’s individual needs. Our regulatory compliance clients include:


  • Companies new to FDA, Health Canada or EU regulations and regulatory compliance
  • Start-up organizations with novel submissions to 510(k) submissions from multi-national corporations
  • Investment firms seeking private equity due diligence for pre-acquisition and post-deal research
  • Law firms seeking pharmaceutical consulting firm expertise in the remediation of warning letters, consent decrees, 483’s or import bans


Regulatory Affairs


Regulatory affairs is Regulatory Compliance Associates backbone. We exceed other pharma consulting companies with industry experts experienced in complexities of the pharmaceutical and biopharmaceutical industries. Our pharma consulting expertise spans all facets and levels of Regulatory Affairs, from Regulatory Support for New Products to Life Cycle Management, to other services like Outsourced Regulatory Affairs, Submissions, Training, and more.


As your partner, we can negotiate the potential assessment minefield of regulatory compliance services with insight, hindsight, and the clear advantage of our breadth and depth of knowledge and regulatory compliance consulting. We offer the following pharma consulting regulatory affairs services for pharmaceutical companies.


  • New Product Support
  • Product Lifecycle
  • Other Regulatory Services
  • Combination Products


Compliance Assurance


The regulations process surrounding pharmaceutical companies can be tricky for even the most experienced industry veteran to understand. Just one misstep could mean significant and lasting consequences for your business. At Regulatory Compliance Associates, we offer the pharma consulting experience and pharma consultants necessary to guide you through the quality compliance process.


  • Assessments
  • Audits
  • Regulatory Agency Response
  • Preparation and Training
  • Inspection Readiness
  • Data Integrity


Quality Assurance


Regulatory Compliance Associates Quality consulting includes assessments, strategy, implementations, staff augmentations, and identification of quality metrics to ensure continuous improvement. Our pharma consultants understand the strategic thinking needed to align your business needs and goals. Regulatory Compliance Associates quality assurance services include quality experts with experience spanning major corporations and start-ups. Our pharmaceutical consulting firm knows firsthand how to achieve, maintain, and improve quality, and we excel in transferring pharma consulting knowledge to your organization.


  • 21 CFR Part 11
  • Data Integrity
  • Manufacturing Support
  • Facility Support
  • Quality Metrics


Remediation Services 


Regulatory Compliance Associates has a proven remediation services approach to managing FDA Warning Letters, Consent Decrees, Remediation and other serious regulatory situations. Our pharma consultants know how to partner with executive, legal, and communication teams. Each RCA pharma consulting Expert will develop a response that will be accepted by the regulatory agency and be realistic to execute.


Regulatory Compliance Associates pharma regulatory consultants will develop a comprehensive proof book of documented evidence demonstrating the corrective action taken to remediate non-compliant issues. In addition, each Regulatory Compliance Associates pharma consulting Expert understands compliance enforcement. We’ll prepare a comprehensive pharma consulting strategy to assist in your remediation efforts, drive continuous improvement, and maintain regulatory compliance with the regulations.


  • Regulatory Action
  • Regulatory Compliance
  • Regulatory Enforcement
  • Warning Letter
  • 483 Observation
  • Oversight Services
  • Risk Management Plan


About Regulatory Compliance Associates


pharmaceutical consultantsRegulatory Compliance Associates® (RCA) provides pharmaceutical consulting to the following industries for resolution of life science challenges:



We understand the complexities of running a life science business and possess areas of expertise that include every facet of R&D, operations, regulatory affairs, quality, and manufacturing. We are used to working on the front lines and thriving in the scrutiny of FDA, Health Canada, MHRA and globally-regulated companies.


As your partners, we can negotiate the potential minefield of regulatory compliance and regulatory due diligence with insight, hindsight, and the clear advantage of our unique expertise and experience.


  • Founded in 2000
  • Headquartered in Wisconsin (USA)
  • Expertise backed by over 500 industry subject matter experts
  • Acquired by Sotera Health in 2021


About Sotera Health


The name Sotera Health was inspired by Soteria, the Greek goddess of safety, and reflects the Company’s unwavering commitment to its mission, Safeguarding Global Health®.


Sotera Health Company, along with its three best-in-class businesses – Sterigenics®Nordion® and Nelson Labs®, is a leading global provider of mission-critical end-to-end sterilization solutions and lab testing and advisory services for the healthcare industry. With a combined tenure across our businesses of nearly 200 years and our industry-recognized scientific and technological expertise, we help to ensure the safety of over 190 million patients and healthcare practitioners around the world every year.


We are a trusted partner to 5,800+ customers in over 50 countries, including 40 of the top 50 medical device companies and 9 of the top 10 pharmaceutical companies.


Commitment to Quality


Our Certificate of Registration demonstrates that our Quality Management System meets the requirements of ISO 9001:2015, an internationally recognized standard of quality.


To begin the Regulatory Compliance Associates scoping process today, please enter your information in the blue form below and click the submit button at the bottom of the webpage. 


Our website uses cookies to give you the best possible experience.

By continuing to use this site, you agree to the use of cookies.