Susan Schniepp, Distinguished Fellow of Regulatory Compliance Associates® Inc., discusses how the FDA’s proposed guidance for the metrics of quality raises questions about quantifying the tangibles and intangibles of quality culture.
FDA’s proposed guidance for quality metrics raises questions about quantifying the tangibles and intangibles of quality culture.
The long awaited, anxiously anticipated FDA guidance on quality metrics was finally distributed for comment on July 28, 2015. The official title of this guidance for industry is Request for Quality Metrics, Guidance for Industry (1) and its potential release has been looming on the horizon since 2012.
The intent of FDA to establish quality metrics first emerged in 2012 when Congress passed the Food and Drug Administration Safety and Innovation Act (FDASIA) enhancing FDA’s capability to proactively react to, prevent, and alleviate drug shortages. Specifically, Title VII Section 705 of the Act states FDA
“shall inspect establishments described in paragraph  that are engaged in the manufacture, preparation, propagation, compounding, or processing of a drug or drugs (referred to in this subsection as ‘drug establishments’) in accordance with a risk-based schedule established by the Secretary.”
Section 706 of the same act allows FDA to request certain information from companies in advance of or in lieu of inspections by stating,
“Any records or other information that the Secretary may inspect under this section from a person that owns or operates an establishment that is engaged in the manufacture, preparation, propagation, compounding, or processing of a drug shall, upon the request of the Secretary, be provided to the Secretary by such person, in advance of or in lieu of an inspection…”.
In the Feb. 12, 2013 Federal Register Notice (3), FDA asked the industry to
“assist the Food and Drug Administration in drafting a strategic plan on drug shortages as required by the Food and Drug Administration Safety and Innovation Act…”
This notice asked a series of thought-provoking questions including “What metrics do manufacturers currently use to monitor production quality?” and “How frequently would such metrics need to be updated to be meaningful?”. After a few years of actively engaging and listening to industry in a variety of venues, this new guideline has finally been released.
The metrics proposed in the guideline are not new to the pharma industry. Many of them are currently being used by companies to internally measure performance. In some cases, the specified metrics are also reported to the agency via the annual report or are contained in the annual product review. The bio/pharma industry needs to review these metrics and ensure they will provide meaningful data while avoiding unintended consequences.
The underlying and understated tenet used to determine a company’s well-being is a measure of their quality culture. The culture of a company dictates the veracity of their metrics. The best way to ensure the data reported has merit is to assess the quality culture of the submitting organization.
It is in this area that the new guidance lacks clarity. The guidance leaves the opportunity open to establish quality-culture metrics by stating,
“these metrics are not intended to be an all-inclusive set of the quality metrics that FDA could consider useful to assess a product and manufacturer’s state of quality. For example, senior management commitment to quality is an important factor in evaluating the overall health of the PQS [pharmaceutical quality system] and quality culture”
“…the Agency is committed to a dialog with industry to consider benchmarks and standards that could provide acceptable metrics that specifically demonstrate senior management’s commitment to a culture of quality …”
This commitment to establishing quality culture metrics is further evidenced by the section in the guideline titled “Optional Metrics Related to Quality Culture and Process Capability/Performance”. In this section, FDA states,
“acknowledges the importance of quality culture to the overall state of quality of the product, process, and commitment to quality”
FDA proposes three voluntary metrics to try to get at the elusive quality culture. The first optional metric proposed is intended to measure senior management engagement by assessing whether the head of the quality unit and the head of the operations unit have signed the annual product review (APR) or product quality review (PQR).
The second optional metric proposed is corrective action and preventive action (CAPA) effectiveness. The measurement for this metric is to indicate the percentage of corrective actions that required retraining of personnel, the assumption being that the root cause of the original deviation (real or due to insufficient analysis) was determined to be insufficient or ineffectual training.
The third proposed metric is intended to measure a firm’s process capabilities through a series of three questions. The real question should be if these three optional metrics, taken together, shed any light on the quality culture.
Achieving a quality culture requires management and employees to establish an environment where responsibility, accountability, and reliability are paramount, and to understand the role each person performs in delivering a high-quality product to the customer and sustaining that performance on a continual basis.
Management must educate employees and provide the tools and environment where they can perform their functions in an atmosphere that encourages excellence and continuous improvement.
Product Quality Review (PDR)
Assigning the head of quality and the head of operations the task of signing the APR or the PQR does not ensure management engagement nor does it mean that the quality culture is lacking. It is up to an organization to establish the appropriate level of responsibility and signing authority for APR and PQRs.
It is up to senior management to provide the people charged with these activities the necessary resources to complete the task in a timely manner with the expectation that they will be held accountable for the contents.
The second optional quality culture metric is specific to CAPA. The proposed metric is to report the percentages of corrective actions involving the retraining of personnel. Without context supporting the retraining of personnel, this metric does not offer insight into the true culture of an organization.
It could be argued that any CAPA that results in a reduction or elimination of a recurring deviation would require an element of training personnel. In fact, retraining of personnel on the CAPA issue, how it was solved, and how to implement the necessary change is evidence of management engagement. It should be expected that a majority of CAPAs involve some retraining of personnel.
Critical Quality Attributes (CQA)
The third quality-culture optional metrics involves trying to use critical quality attributes (CQA) as a key indicator of a quality culture. Of the three optional metrics proposed, this one does provide some measurement of the existence of a quality culture.
On the surface, the questions just seem to be a regurgitation of information contained in the APR or PQR. Upon closer evaluation, however, it is clear that FDA is trying to measure whether a company drives for continuous improvement through their review and assessment of threshold levels established with CQAs.
Companies that establish corrective actions to CQAs and link them to a requirement to issue a CAPA when they exceed the established threshold levels demonstrate a commitment to continuous improvement. Continuous improvement programs are, in fact, reliable indicators of the presence of a quality culture.
The establishment of simple quality metrics that not only measure the quality of the product but also reflect the quality culture of an organization is required to assist FDA in establishing a risk-based audit program. The problem is that it is difficult to measure something as intangible as culture with cold, hard data. The remaining question is: If taken together, are the three proposed optional metrics indicative of a quality culture? The answer is, maybe.
Careful thought and consideration should be exercised when determining what to measure, how often to measure, how to interpret and communicate the data, and what the expectation is for using the data to drive positive change. Management needs to be cognizant of the fact that whatever metrics are reported, they must be developed, evolved, and adjusted over time to maximize their impact on driving positive change.
When choosing a metric, it is important that leadership is aware of unintended consequences that may inadvertently drive negative behavior instead of the root problem. Management attempting to incentivize achievement of the goal such as offering a financial award if the goal is achieved, may lead to inappropriate behaviors that do not address the real issue.
In these cases, it is generally not the metric that will drive the behavior but rather use of behavioral rewards. Reward for achievement rather than analysis of the real underlying causes will not lead to sustainable positive change. When managed properly, metrics are an important tool to help drive positive change and quality process improvements.
Upon observation, an unhealthy quality culture is easy to identify. People in a poor culture do not understand their job and its importance to the business. They often appear stressed, and they hide their mistakes or blame others for their errors. There is no evidence of teamwork.
People work in silos and rarely, if ever, seek input or advice from others. Metrics that could potentially be used to measure a poor culture include a large employee turnover, an overabundance of deviations attributed to human error, and lack of pride in the performance of employees’ jobs.
In contrast, a robust, healthy quality culture can be evidenced by alignment of goals between quality and operations. Product validation is a collaboration between self-sustained work teams that focus on continual improvement, and employees who incorporate quality into their daily jobs.
They are not afraid to speak up and offer suggestions for improvement to their colleagues. People understand the importance of their jobs and respect each other and their management. This culture welcomes inspections and views these inspections as another tool to use in their continual improvement initiatives. Metrics that could potentially be used to measure a healthy quality culture include a small employee turnover, deviations that identify a root cause other than human error, and pride in the performance of their jobs.
When establishing a metrics program, companies should evaluate numerous data input points including, but not limited to, product-quality attributes, manufacturing site performance, people metrics, and quality-system metrics. For product-quality metrics, companies should consider reporting on batch-specific data such as trending drug product, drug substance, and stability test results against customer complaint rates.
Indirect product-quality metrics could include environmental monitoring, water trend results, and yield rates. When establishing site metrics, the company could look at inspection history including internal audit findings and maintenance history such as equipment age versus defect-failure rates. People metrics should consider ongoing job-specific training and education, skills and experience assessments, and employee turnover rate by job function and site. Quality systems metrics might look at change control, investigation root-cause trends, and release-testing cycle times.
There is no set requirement on which metrics a company should track to measure their overall performance. Each company should determine which metrics to track based on their operations, number of facilities they operate and where they are located, what types of products they manufacture, and what type of culture exists in their places of business.
The metrics chosen must be meaningful and written to provide a clear analysis of ongoing activities. It is important for operations and quality to agree on the metrics and how to report them to management to avoid overreaction to the data. It is not sufficient to simply report the data. The interpretation of the data is of crucial importance because it may include a root-cause analysis of its own.
Vol. 39, No. 9
About RCA’s Pharmaceutical Services
Regulatory Compliance Associates (RCA)® has helped thousands of pharmaceutical companies meet regulatory, compliance, quality assurance, and remediation challenges. With more than 20 years of experience with FDA, Health Canada, EU and global regulatory agencies worldwide, RCA offers leading pharmaceutical consultants that can help you navigate through the challenges associated with evolving industry regulations.
Our team of over 500 seasoned FDA, Health Canada and EU compliance consultants and regulatory affairs experts can understand the complexities surrounding the pharmaceutical industry and the unique inner workings of the regulatory process.
Whether you’re in the product planning, development or pharmaceutical lifecycle management stage or need a remediation strategy for a compliance crisis, RCA® Inc. will guide you through every step of the regulatory process and create a customized approach depending on your product and your pharma company’s individual needs. Our clients include:
- Companies new to FDA, Health Canada or EU regulations and the pharmaceutical industry
- Start-up organizations with novel submissions to 510(k) submissions from multi-national corporations
- Investment firms seeking private equity due diligence for pre-acquisition and post-deal research
- Law firms seeking expertise in the remediation of warning letters, consent decrees, 483’s or import bans
Regulatory affairs is Regulatory Compliance Associates® Inc.’s backbone and we fully understand the complexities of the pharmaceutical and biopharmaceutical industries. Our expertise spans all facets and levels of Regulatory Affairs, from Regulatory Support for New Products to Life Cycle Management, to other services like Outsourced Regulatory Affairs, Submissions, Training, and more.
As your partner, we can negotiate the potential assessment minefield of pharmaceuticals with insight, hindsight, and the clear advantage of our breadth and depth of knowledge and experience. We offer the following four regulatory affairs services for pharmaceutical companies.
- New Product Support
- Product Lifecycle
- Other Regulatory Services
The regulations process surrounding pharmaceutical companies can be tricky for even the most experienced industry veteran to understand, and just one misstep could mean significant and lasting consequences for your business. At RCA® Inc., we offer the experience and resources necessary to guide you in quality compliance.
- Regulatory Agency Response
- Preparation and Training
- Inspection Readiness
Regulatory Compliance Associates® Inc.’s Quality Assurance services include assessments, strategy, implementations, staff augmentations, and identification of quality metrics to ensure continuous improvement, aligning with your business needs and goals. Our consultants are quality experts with experience spanning major corporations and start-ups. We know firsthand how to achieve, maintain, and improve quality, and we excel in transferring this knowledge to your organization.
- 21 CFR Part 11
- Data Integrity
- Manufacturing Support
- Facility Support
Regulatory Compliance Associates® Inc. has significant experience and a proven approach to managing FDA Warning Letters, Consent Decrees, Remediation and other serious regulatory situations. We know how to partner with executive, legal, and communication teams, and will assist management with a response that will be accepted by the regulatory agency and be realistic to execute.
We can develop a comprehensive proof book of documented objective evidence demonstrating the corrective actions taken to remediate non-compliant issues. In addition, RCA can help prepare a comprehensive strategy to assist in your remediation efforts, drive continuous improvement, and maintain compliance with the regulations.
- Regulatory Action
- Warning Letter
- 483 Observation
- Oversight Services
Regulatory Compliance Associates® (RCA) provides healthcare consulting services to the following industries for resolution of compliance and regulatory challenges:
We understand the complexities of running a life science business and possess areas of expertise that include every facet of R&D, operations, regulatory affairs, quality, and manufacturing. We are used to working on the front lines and thriving in the scrutiny of FDA, Health Canada, MHRA and globally-regulated companies.
As your partners, we can negotiate the potential minefield of regulatory compliance and regulatory due diligence with insight, hindsight, and the clear advantage of our unique expertise and experience.
- Founded in 2000
- Headquartered in Wisconsin (USA)
- Expertise backed by over 500 industry subject matter experts
- Acquired by Sotera Health in 2021
About Sotera Health
The name Sotera Health was inspired by Soteria, the Greek goddess of safety, and reflects the Company’s unwavering commitment to its mission, Safeguarding Global Health®.
Sotera Health Company, along with its three best-in-class businesses – Sterigenics®, Nordion® and Nelson Labs®, is a leading global provider of mission-critical end-to-end sterilization solutions and lab testing and advisory services for the healthcare industry. With a combined tenure across our businesses of nearly 200 years and our industry-recognized scientific and technological expertise, we help to ensure the safety of over 190 million patients and healthcare practitioners around the world every year.
We are a trusted partner to more than 5,800 customers in over 50 countries, including 40 of the top 50 medical device companies and 8 of the top 10 pharmaceutical companies.
To begin the RCA® scoping process today, please enter your information in the blue form below and click the submit button at the bottom of the webpage.