The Metrics of Quality Culture
Share Our Information
The long awaited, anxiously anticipated FDA guidance on quality metrics is now finally distributed for comment. The official title of this guidance for industry is Submission of Quality Metrics Data Guidance for Industry. As a result, many executives believe it will continue to increase the conversation around quality assured best practices and Quality Culture.
FDASIA
The intent of FDA to establish quality metrics first emerged when Congress passed the Food and Drug Administration Safety and Innovation Act (FDASIA) enhancing FDA’s capability to proactively react to, prevent, and alleviate drug shortages.
Title VII
Specifically, Title VII Section 705 of the Act states FDA “shall inspect establishments described in paragraph that are engaged in the manufacture, preparation, propagation, compounding, or processing of a drug or drugs.”
Technical Experts
Section 706 of the same act allows FDA to request certain information from companies in advance of or in lieu of inspections. The guidance goes on to state “records or other information that the Secretary may inspect” under this section. Technical experts involved with “the manufacture, preparation, propagation, compounding, or processing of a drug shall” will be responsible.
Drug Shortages
The intent of FDA to establish quality metrics is to proactively react to, prevent, and alleviate drug shortages. In the Federal Register Notice, FDA asked the industry to assist with “drafting a strategic plan on drug shortages” per the FDA Safety and Innovation Act. This notice asked a series of thought-provoking questions that invited collaboration from across the industry.
Manufacturing Quality
Questions like “What metrics do companies currently use to monitor manufacturing quality?” offered feedback from companies of all sizes. Another question – “How frequently would such metrics need to be updated to be meaningful?” – invited an internal discovery on when assessments should be conducted.
Industry Collaboration
After a years of conformance quality experts actively engaging with the industry in a variety of venues, this new guideline has helped shape a new analysis on quality culture. The culture of a company dictates the veracity of their metrics.
Product Review
The metrics proposed in the guideline are not new to the pharma industry. Many of them are currently being used by companies to internally measure performance. In some cases, the specified metrics are also reported to the agency via annual report or annual product review. The biotech and pharmaceutical industry needs to ensure meaningful culture continuous improvement data to be provided to measure success.
FDA Guidance
The quality culture definition used to determine a company’s well-being is only one example of excellent quality. Creating a culture of Quality dictates the veracity a leadership team’s metrics. The best way to ensure the data has merit is to assess the quality culture of the submitting organization. It is in this area that the FDA guidance lacks clarity.
QA Metrics
The guidance leaves the opportunity open to establish both quality culture meaning and metrics by stating, “these metrics are not intended to be an all-inclusive set”. As innovation grows, there will be new and useful metrics to assess a product and manufacturer’s state of quality.
Pharmaceutical Quality System (PQS)
For example, management commitment to quality is an evaluation factor for the overall health of the PQS [pharmaceutical quality system]. The industry dialog to share benchmarks and standards could provide more streamlined metrics from management team to team. A robust strategy would demonstrate the commitment to measurability while inviting these new evaluation factors.
Quality Metrics
FDA proposes three voluntary metrics to try to get at total quality culture. The first optional metric proposed is intended to measure senior management engagement by assessing whether the head of the quality unit and the head of the operations unit have signed the annual product review (APR) or product quality review (PQR).
CAPA
The second optional metric proposed is corrective action and preventive action (CAPA) effectiveness. The measurement for this metric is to indicate the percentage of corrective actions that required retraining of personnel.
Deviations
The assumption for retraining is the root cause of original deviation was determined to be insufficient or ineffectual training. Another proposed metric is intended to measure a firm’s process capabilities through a checklist series of questions. The real question should be if these new metrics, optimized together, can shed new light on the quality culture.
Management Engagement
Achieving a quality culture requires an environment where responsibility, accountability, and reliability are paramount. Each employee understands the role they play in delivering a high-quality product to the customer. Each function recognizes that sustaining performance over time means process improvement and training.
Employee Development
Management must educate employees and provide the tools where employees perform in an atmosphere that encourages excellence. Assigning the head of quality and the head of operations the task of signing the APR or the PQR does not ensure management engagement nor does it mean that the quality culture is lacking.
Signature of Authority
It is up to an organization to establish the appropriate level of responsibility and signing authority for APR and PQRs. It is up to senior management to provide the people charged with these activities the necessary resources to complete the task in a timely manner with the expectation that they will be held accountable for the contents.
Retraining personnel
The second optional quality culture metric is specific to CAPA. The proposed metric is to report the percentages of corrective actions involving the retraining of personnel. Without context supporting the retraining of personnel, this metric does not offer insight into the true culture of an organization. It could be argued that any CAPA that results in a reduction or elimination of a recurring deviation would require an element of training personnel.
Preventive Action
For instance, retraining of personnel on the CAPA issue, how it was solved, and implementing the preventive action shows management engagement. It should be expected that a majority of CAPAs involve some retraining of personnel. Finally, Quality metrics can measure how often CAPA training is administered to the operations team and the baseline for future employee development.
Critical to Quality (CTQ)
The third quality-culture optional metrics involves trying to use critical quality attributes (CQA) as a key indicator of a quality culture. Of the three optional metrics proposed, this one does provide some measurement of the existence of a quality culture.
FDA Drug Approval
On the surface, the questions just seem to be a regurgitation of information contained in the APR or PQR. Upon closer evaluation, however, it is clear the FDA drug approval process looks to measure whether a company drives for continuous improvement through their review and assessments. Additionally, threshold levels for established CQAs are actively monitored during an FDA inspection.
Continuous Improvement
Continuous improvement programs are, in fact, reliable indicators of the presence of a quality culture. Companies that have established CQAs and linked them to a requirement to issue a CAPA when they exceed the established threshold levels have demonstrated a commitment to continuous improvement.
Measuring intangibles
The establishment of simple quality metrics don’t simply measure the quality of the product. Metrics set a baseline for the quality culture required to assist FDA in establishing a risk-based audit program. The problem is that it can be difficult to measure culture with cold, hard data. The remaining question is: If taken together, are the proposed optional metrics indicative of a quality culture? The answer is subjective and ongoing.
Change Management
Careful thought and consideration should be exercised when determining what to measure and how often to measure. Driving positive change involves how to interpret and communicate the data to the team. Likewise, clarity on interpreting expectations for using the data also drives positive collaboration.
Change Management Process
As a result, management needs to be cognizant of the change management process metrics are reported, they must be developed, evolved, and adjusted over time to maximize their impact on driving positive change. Careful thought and consideration should be exercised when determining what to measure, how often to measure, how to interpret and communicate the data, and what the expectation is for using the data to drive positive change.
Change Management Plan
When choosing metrics for your change management plan, it is important that the architects of the metric are aware of unintended consequences. Additionally, what metrics (if any) may inadvertently drive negative employee behavior. Management attempting to financial incentivize a goal may lead to inappropriate behaviors that do not address the real issue.
Organizational Change
In these cases, it is generally not the metric that will drive the behavior. Instead, your use of behavioral rewards will incite organizational change at a faster pace. Reward for achievement rather than analysis of the real underlying causes will not lead to sustainable positive change. When managed properly, metrics are an important tool to help drive positive change and quality process improvements.
Quality Assessments
Upon observation, an unhealthy quality culture is easy to identify. People in a poor culture do not understand their job and its importance to the business. They often appear stressed, and they hide their mistakes or blame others for their errors. Likewise, there is no evidence of teamwork. People work in silos and rarely, if ever, seek input or advice from others.
Employee Turnover
One metric that could be potentially used to assess a poor culture include employee turnover. Deviations due to human error may demonstrate an employee’s lack of pride delivering a quality performance. For instance, deviation training may be reconsidered if the employee is not a quality culture fit in general.
Cascaded Goals
In contrast, a robust, healthy quality culture can be evidenced by alignment of goals between quality and operations. Work teams who focus on continual improvement promote the attitude of incorporating quality into their daily jobs. Above all, they are not afraid to offer suggestions for improvement to their colleagues or about the process. The metrics chosen must be meaningful and written to provide a clear analysis of ongoing activities.
Continuous Quality Improvement
People understand the importance of their jobs and respect each other and their management. This culture views these inspections as a tool in their continual improvement initiatives. Metrics used to measure a healthy quality culture include a small employee turnover, deviations that identify a root cause other than human error, and employee pride in their performance.
Conclusion
When establishing a metrics program, companies should evaluate numerous data input points including, but not limited to, product-quality attributes, manufacturing site performance, people metrics, and quality-system metrics. For product-quality metrics, companies should consider reporting on batch-specific data. Most importantly, this includes trending drug product, drug substance, and stability test results against complaint rates.
Data Points
Indirect product-quality metrics should include environmental monitoring, water trend results, and yield rates. When establishing site metrics, the company can look at inspection history including internal audit findings and maintenance history such as equipment age versus defect-failure rates. People metrics should consider ongoing job-specific training and education.
Skill Assessment
Skills and experience assessments should be measured against employee turnover rate. Analyzing the turnover rate by job function and site can show quality success or failures. Quality systems metrics will analyze change control, investigation root-cause trends, and release-testing cycle times. There is no set requirement on which metrics a company should track to measure their overall performance.
Operations Management
Each company should determine which metrics to track based on their operations, number of facilities they manage and where they are located. Above all, what types of products they manufacture and the existing culture is foundational. The metrics chosen must be meaningful and written to provide a clear analysis of ongoing activities.
KPI Report
It is important for operations and quality to agree on how to report them to management to avoid overreaction to the data. It is not sufficient to simply report the data. The interpretation of the KPI report data is of crucial importance because it may include a root-cause analysis of its own.
About the Publisher:
About RCA’s Quality Assurance Services
Regulatory Compliance Associates® Quality Assurance services include assessments, strategy, implementations, staff augmentations, and identification of quality metrics to ensure continuous QA improvement, aligning with your QA engineering team and business needs. Quality is something our software quality assurance consultants are experts at with experience spanning major corporations and start-ups. We know firsthand how to achieve, maintain, and improve quality, and we excel in transferring this knowledge to your quality engineering team.
QA Services
- Quality Assurance Consultants – Interim Management, staff augmentation or outsourcing quality and regulatory management has become an increasingly common practice in the life science industry. The quality assurance (QA) and regulatory affairs (RA) functions create extra complexity for large and small life science companies because needs may vary greatly depending on the lifecycle of the organization.
- QA Audit – Virtual gap assessments can assist your team in advance for an upcoming QA inspection by a notified body. These proactive organizations turn to RCA for outsourcing Ex-FDA experts who can describe exactly what a notified body looks for.
- Quality Assurance in Software Testing – Cybersecurity is becoming more essential to our industry based on updated FDA cybersecurity guidance. Pharmaceutical manufacturers partner with RCA for their cybersecurity FDA strategy and how to stay ahead of hackers.
QA Process
Early in a company’s lifecycle, strategic leadership is needed for product filings, regulatory pathway, manufacturing setup, and quality management systems creation. As the company matures, QA planning often transitions to a continuous improvement stage.
- Total Quality Assurance
- Quality Assurance Methodologies
- QA Procedures
- Project QA
- Product QA
- Supplier QA
- Digital QA
- Agile QA
Outsourcing QA management can often provide a more strategic perspective when you’re challenged across changing quality control in engineering teams. Our pharmaceutical quality assurance experts bring a wealth of quality assurance management needed to help you cut cost via unique QA methodologies.
QA Planning
- Quality Control Audit
- QA Procedures
- QA Reporting
- QA Responsibility
- SOP Quality Control
- QA Best Practices
- QA Programmer
- QA Project Manager
- QA Software Tester
- SQA Tester
- Software QA Analyst
- Software Quality Analyst
- QA Testing Services
- Standard Quality Control
- Quality Control Tester
- Quality Assurance Analyst
- Data Quality Analyst
Software QA
- Software QA Outsourcing
- Agile Quality Management
- Web Quality Assurance
- Software Quality Control
- Quality System Automation Using Software
- Quality Systems Implementation
- Quality System Remediation / Improvement
- Quality Control Procedures
- Computerized System Validation Standard Operating Procedure (SOP) – Our computer validation experts and software engineers have extensive experience with the GAMP5 validation model to ensure that you have the appropriate level of documentation.
- Risk Management ICH Q9 – We can provide you with the staff and tools needed to implement or remediate your quality systems to current industry standards and train your personnel to manage it.
QA Teams
- Corrective and Preventive Action (CAPA) and Complaint Systems – We manage implementation and remediation of CAPA systems and Complaint Handling Systems.
- 21 CFR Part 210 and 211
- Quality Control Assurance
- EU Good Manufacturing Practice (GMP)
- Quality Assurance Best Practices
- QA Testing Process
- Performing Quality Control
- QA Workflow
- Quality Culture Guidelines
- QA process
- QA Tester
- QA Analyst
- QA Management
- Laboratory Support
- SOP Development
- Method Transfer
- Method Validation
- Data QA
About RCA
Regulatory Compliance Associates® (RCA) provides FDA inspection consulting services to the following industries for resolution of compliance and regulatory challenges:
We understand the complexities of running a life science business and possess areas of expertise that include every facet of R&D, operations, regulatory affairs, quality, and manufacturing. We are used to working on the front lines and thriving in the scrutiny of FDA, Health Canada, MHRA and globally-regulated companies.
As your partners, we can negotiate the potential minefield of regulatory compliance and regulatory due diligence with insight, hindsight, and the clear advantage of our unique expertise and experience.
- Founded in 2000
- Headquartered in Wisconsin (USA)
- Expertise backed by over 500 industry subject matter experts
- Acquired by Sotera Health in 2021
About Sotera Health
The name Sotera Health was inspired by Soteria, the Greek goddess of safety, and reflects the Company’s unwavering commitment to its mission, Safeguarding Global Health®.
Sotera Health Company, along with its three best-in-class businesses – Sterigenics®, Nordion® and Nelson Labs®, is a leading global provider of mission-critical end-to-end sterilization solutions and lab testing and advisory services for the healthcare industry. With a combined tenure across our businesses of nearly 200 years and our industry-recognized scientific and technological expertise, we help to ensure the safety of over 190 million patients and healthcare practitioners around the world every year.
We are a trusted partner to more than 5,800 customers in over 50 countries, including 40 of the top 50 medical device companies and 8 of the top 10 pharmaceutical companies.
To begin the Regulatory Compliance Associates® scoping process today, please enter your information in the blue form below and click the submit button at the bottom of the webpage.