Quality Metrics

Susan Schniepp, Distinguished Fellow at Regulatory Compliance Associates® Inc., discusses the quality metrics journey in determining the suitability of pharmaceutical products.

 

quality metrics

 

One of the most discussed and debated topics on today’s pharmaceutical landscape is the issue of quality metrics. Establishing, maintaining, and interpreting quality metrics to determine the suitability of pharmaceutical products has become a high priority for the FDA. To understand the issue of quality metrics it is important to start at the beginning.

 

FDASIA

 

The first stop on this journey took place in 2012 when Congress passed the Food Drug Administration Safety and Innovation Act (FDASIA) which enhanced FDA’s capability to proactively react to, prevent, and alleviate drug shortages. This direction is embedded in the language contained in Title VII—Drug Supply Chain and Title X—Drug Shortages.

 

Specifically, Title VII Section 705 of the Act states FDA:

 

“…shall inspect establishments described in paragraph that are engaged in the manufacture, preparation, propagation, compounding, or processing of a drug or drugs (referred to in this subsection as ‘drug establishments’) in accordance with a risk-based schedule established by the Secretary”.

 

QA Risk Factors

 

This section also describes the risk factors considered in establishing the inspection schedule. One of the risk factors listed in this section is:

 

“Any other criteria deemed necessary and appropriate by the Secretary for purposes of allocating inspection resources.”

 

Section 706 of the same act authorizes FDA to request certain company information in advance of or in lieu of inspections by stating

 

“Any records or other information that the Secretary may inspect under this section from a person that owns or operates an establishment that is engaged in the manufacture, preparation, propagation, compounding, or processing of a drug shall, upon the request of the Secretary, be provided to the Secretary by such person, in advance of or in lieu of an inspection…”

 

In addition, Title X section 506C-1 (Annual Reporting on Drug Shortages). Title X section 506C-1 requires FDA to annually provide Congress “a report on drug shortages…”

 

Quality Metrics Examples

 

The second part of the journey occurred in the Federal Register Notice. In that notice, FDA asked the industry to:

 

“assist the Food and Drug Administration in drafting a strategic plan on drug shortages as required by the Food and Drug Administration Safety and Innovation Act…”

 

This notice asked a series of thought-provoking questions about more specific quality metrics including:

 

“What metrics do manufacturers currently use to monitor production quality?” and “How frequently would such metrics need to be updated to be meaningful?”.

 

The industry reaction to this information was varied. Many trade organizations responded to the questions in the Federal Register. Some prepared white papers while others held meetings to discuss the issue with their members. The general consensus was that industry needed to seriously engage with FDA to define which quality metrics would provide information to the agency supporting efforts to eliminate drug shortages and to be used in establishing a risk-based approach to inspections.

 

FDA Quality Metrics

 

The third leg of the journey was the issuance of a draft guidance titled Request for Quality Metrics. In between the issuance of the Federal Register and the release of this original draft guidance there were many industry comments submitted to the agency suggesting various metrics that might be employed by the agency.

 

In addition, industry trade organizations held conferences and seminars to discuss the issue with the stakeholders and agency representatives. A number of these trade organizations published white papers based on the proceedings from these conferences and put forth the position that the need to submit metrics to the agency was redundant because of the requirement to submit annual product reviews. The agency listened to industry and some of the feedback was incorporated into the guidance.

 

Quality Performance Indicators

 

The quality performance indicators that were chosen as metrics for the updated guidance include:

 

  • Lot Acceptance Rate
  • Product Quality Complaint Rate
  • Invalidated OOS Rate
  • Annual Product Review
  • Product Quality Review
  • On Time Rate

 

The guidance also contained three optional metrics intended to measure quality culture:

 

  • Measuring Senior Management Engagement
  • CAPA Effectiveness
  • Process Capability/Performance

 

This version of the quality metrics guidance generated 83 comments from industry submitted to the docket. A number of these comments focused on the practicality of submitting the requested metrics. In response to this concern the agency issued the Quality Metrics Technical Conformance Guide: Technical Specifications Document that explained how companies would be expected to submit the data to the agency.

 

Quality Management

 

The fourth part of our journey came when the second version of the guidance, titled Submission of Quality Metrics Data Guidance for Industry was released for comment. The docket for commenting on this most recent version of the guidance closed. There were 25 comments submitted. 12 from trade associations, 10 from individual firms, one from a hospital group, one from an academic institution and one from a private citizen.

 

The guidance clearly stated the

 

“…metrics can be also useful to FDA: to help develop compliance and inspection policies and practices…. improve the agency’s ability to predict, and therefore, possibly mitigate future drug shortages; and to encourage the pharmaceutical industry to implement state-of-the-art, innovative quality management systems for pharmaceutical manufacturing.”

 

Quality Metrics

 

The FDA also clarified the goal of the metrics program by stating

 

“As described in this guidance, FDA is initiating a voluntary reporting phase of the FDA quality metrics reporting program.”

 

This current version of the document requires companies to report on only three metrics:

 

  • Lot Acceptance Rate
  • Product Quality Complaint Rate
  • Invalidated OOS Rate

 

The optional metrics intended to measure quality culture were removed.

 

Total Quality Management

 

These proposed metrics are not new to the pharmaceutical industry. Many of them are currently being used by companies to measure their total quality management performance. In order to be successful, companies need to review and analyze the information the FDA is asking for, as well as other metrics they are collecting, and identify potential problem signals. Quality leadership teams should look to solve issues and self-correct before regulatory inspections occur.

 

The problem with the proposed metrics, however, is that they are lagging indicators of performance. There is no set requirement on which metrics a company should track to measure their overall performance. Each company should determine which metrics to track based on their operations, number of facilities they operate and where they are located, what types of products they manufacture, and what type of culture exists in their places of business. Any metrics chosen must be meaningful and written to provide a clear analysis of ongoing activities.

 

Quality Assurance Management

 

It is important for operations and quality to agree on any type of quality assurance metrics. Additionally, how to report the results to management is as important as the results themselves. The interpretation of the data is a crucial element because it may include a root-cause analysis of its own and may help to promote continuous process improvements.

 

When choosing a metric, it is important that the architects of the QA metric are aware of consequences that may inadvertently drive negative behavior. Management attempting to incentivize achievement of the goal such as offering a financial award if the goal is achieved may lead to inappropriate behaviors that do not address the real issue. In these cases, it is generally not the metric that will drive the behavior but rather use of behavioral rewards. Reward for achievement rather than analysis of the real underlying causes will not lead to sustainable positive change.

 

Quality Control Management 

 

When managed properly, metrics are an important tool to drive positive design controls and process improvements. Regardless of what metrics a company chooses to measure their performance, achieving a quality culture is important in assuring reported quality control metrics are accurate and reliable. A quality culture requires management and employees to establish an environment where responsibility, accountability, and reliability are paramount. This helps each person deliver a high-quality product to the customer and sustained performance over time.

 

Management must educate employees and provide the tools and environment where they can perform their functions in an atmosphere that encourages excellence and continuous improvement. Continuous improvement programs are, in fact, reliable indicators of the presence of a quality culture. Where the industry’s and the agency’s destination on this journey ends is yet to be seen. What is evident is that companies must have a robust quality metrics program so they may continue to supply quality products without interruptions to patients.

 

Planning Quality Management

 

Companies need to keep in mind that when planning a quality management program, they should evaluate numerous data input points. This would include, but not limited to:

 

  • Product-quality attributes
  • Manufacturing site performance
  • People metrics
  • Quality-system metrics

 

For product-quality metrics, companies should consider reporting on batch-specific data such as trending drug product, drug substance, and stability test results against customer complaint rates.

 

Internal Audit

 

Indirect product-quality metrics could include environmental monitoring, water trend results, and yield rates. When establishing site metrics, the company could look at inspection history including internal audit findings and maintenance history such as equipment age versus defect-failure rates.

 

People metrics should consider ongoing job-specific training and education, skills and experience assessments, and employee turnover rate by job function and site.

 

Root Cause

 

Quality systems metrics might look at change control, investigation root-cause trends, and release-testing cycle times. Companies that can successfully establish a robust metrics program that helps them continuously improve should have no trouble meeting the FDA metrics requirements.

 

About RCA’s Pharmaceutical Services

 

Regulatory Compliance Associates® Inc.’s background and experience in the Pharmaceutical industry has helped thousands of companies meet regulatory, compliance, quality assurance, and remediation challenges, in addition to their strategic consulting needs. 

With more than 20 years of experience with FDA, Health Canada, EU and global regulatory agencies worldwide, RCA offers leading pharmaceutical consultants that can help you navigate through the challenges associated with evolving industry regulations.

Our team of over 500 seasoned FDA, Health Canada and EU compliance consultants and regulatory affairs experts can understand the complexities surrounding the pharmaceutical industry and the unique inner workings of the regulatory process. 

 

Client Solutions

 

Whether you’re in the product planning, development or pharmaceutical lifecycle management stage or need a remediation strategy for a compliance crisis, RCA® Inc. will guide you through every step of the regulatory process and create a customized approach depending on your product and your pharma company’s individual needs. Our clients include:

 

  • Companies new to FDA, Health Canada or EU regulations and the pharmaceutical industry
  • Start-up organizations with novel submissions to 510(k) submissions from multi-national corporations
  • Investment firms seeking private equity due diligence for pre-acquisition and post-deal research
  • Law firms seeking expertise in the remediation of warning letters, consent decrees, 483’s or import bans

 

Regulatory Affairs

 

Regulatory affairs is Regulatory Compliance Associates® Inc.’s backbone and we fully understand the complexities of the pharmaceutical and biopharmaceutical industries. Our expertise spans all facets and levels of Regulatory Affairs, from Regulatory Support for New Products to Life Cycle Management, to other services like Outsourced Regulatory Affairs, Submissions, Training, and more.

As your partner, we can negotiate the potential assessment minefield of pharmaceuticals with insight, hindsight, and the clear advantage of our breadth and depth of knowledge and experience. We offer the following four regulatory affairs services for pharmaceutical companies.

 

  • New Product Support
  • Product Lifecycle
  • Other Regulatory Services

 

Compliance Assurance

 

The regulations process surrounding pharmaceutical companies can be tricky for even the most experienced industry veteran to understand, and just one misstep could mean significant and lasting consequences for your business. At RCA® Inc., we offer the experience and resources necessary to guide you in quality compliance.

 

  • Assessments
  • Audits
  • Regulatory Agency Response
  • Preparation and Training
  • Inspection Readiness

 

Quality Assurance

 

Regulatory Compliance Associates® Inc.’s Quality Assurance services include assessments, strategy, implementations, staff augmentations, and identification of quality metrics to ensure continuous improvement, aligning with your business needs and goals. Our consultants are quality experts with experience spanning major corporations and start-ups. We know firsthand how to achieve, maintain, and improve quality, and we excel in transferring this knowledge to your organization.

 

  • 21 CFR Part 11
  • Data Integrity
  • Manufacturing Support
  • Facility Support

 

Remediation 

 

Regulatory Compliance Associates® Inc. has significant experience and a proven approach to managing FDA Warning Letters, Consent Decrees, Remediation and other serious regulatory situations. We know how to partner with executive, legal, and communication teams, and will assist management with a response that will be accepted by the regulatory agency and be realistic to execute. We can develop a comprehensive proof book of documented objective evidence demonstrating the corrective actions taken to remediate non-compliant issues. In addition, RCA can help prepare a comprehensive strategy to assist in your remediation efforts, drive continuous improvement, and maintain compliance with the regulations.

 

  • Regulatory Action
  • Warning Letter
  • 483 Observation
  • Oversight Services

 

About RCA

 

pharmaceutical consultantsRegulatory Compliance Associates® (RCA) provides healthcare consulting services to the following industries for resolution of compliance and regulatory challenges:

 

 

We understand the complexities of running a life science business and possess areas of expertise that include every facet of R&D, operations, regulatory affairs, quality, and manufacturing. We are used to working on the front lines and thriving in the scrutiny of FDA and globally-regulated companies.

As your partners, we can negotiate the potential minefield of regulatory compliance and regulatory due diligence with insight, hindsight, and the clear advantage of our unique expertise and experience.

  • Founded in 2000
  • Headquartered in Wisconsin (USA)
  • Expertise backed by over 500 industry subject matter experts
  • Acquired by Sotera Health in 2021

 

To begin the RCA® scoping process today, please enter your information in the blue form below and click the submit button at the bottom of the webpage. 

 

 

Our website uses cookies to give you the best possible experience.

By continuing to use this site, you agree to the use of cookies.
Continue