Risk Evaluation Strategy (REMS) Audits: How to Execute a Successful Program

Risk Evaluation and Mitigation Strategy (REMS) programs are critical components of drug safety, mandated by the FDA for medications that carry serious safety concerns.

These programs are designed to ensure that the benefits of a medication outweigh its risks. REMS audits play a crucial role in verifying the effectiveness and compliance of these programs, serving as a vital link in the chain of patient safety.

Having planned, scheduled, resourced, and executed large, multi-site REMS audit programs for major pharma firms, we're sharing our playbook for success. Contact us if you need expert REMS audit support.

The Purpose of REMS Audits

First, a quick refresher on the basics. REMS audits serve multiple essential purposes in the pharmaceutical industry:

• Verify compliance with FDA requirements
• Evaluate the effectiveness of the REMS program
• Identify areas for improvement in patient safety and education
• Maintain public trust in pharmaceutical companies and their products

These audits are not merely regulatory exercises; they are fundamental to ensuring patient safety and the integrity of the healthcare system. By systematically examining each aspect of a REMS program, audits can uncover potential weaknesses or gaps that might otherwise go unnoticed, allowing for timely interventions and improvements.

Types of REMS Programs

REMS programs can vary significantly based on the specific risks associated with a medication. Some programs may be relatively straightforward, while others can be highly complex, involving multiple stakeholders and intricate protocols.

For example, we work with a pharmaceutical company on an ongoing REMS program to mitigate the risk of birth defects associated with a particular drug. This program is comprehensive and multi-faceted—cycling annually and requiring:

• Mandatory patient enrollment before receiving medication
• Specific education for healthcare providers about the risks
• Regular pregnancy testing for female patients of reproductive age

In this case, the REMS audits need to verify not only that these processes were in place but also that they were being consistently and correctly implemented across various healthcare settings. This level of complexity underscores the importance of a well-planned and executed audit program.

Planning Your Audit Program

Effective planning is the cornerstone of a successful REMS audit program. This phase sets the tone for the entire audit process and can significantly impact the quality and usefulness of the audit outcomes.

Let's break down the anatomy of an effective REMS audit program at a high level.

1. Define the audit scope and objectives

The first step in planning your audit program is to clearly define its scope and objectives. This involves a thorough review of your REMS program and careful consideration of what aspects need to be audited and why.

We suggest starting this process by listing all components of your REMS program that will be subject to audit.

This might include:

• Patient enrollment processes
• Healthcare provider education programs
• Medication dispensing procedures
• Patient follow-up and monitoring systems

For each of these components, set specific, measurable goals for the audit. These goals should align with your REMS program's overall objectives and provide clear indicators of program effectiveness.

For instance, in the audit project we referenced earlier for a birth defect risk REMS program, we helped establish the following objectives:

• Verifying 100% enrollment of female patients before medication dispensing
• Confirming that 95% of prescribers had completed the required REMS education
• Ensuring all participating pharmacies had up-to-date REMS materials

These objectives were not arbitrary; they were carefully chosen to reflect the critical aspects of the REMS program and to provide meaningful insights into its functioning.

In addition to these specific objectives, it's important to determine key performance indicators (KPIs) that will be used to measure the success of your audit program. These KPIs might include metrics such as the percentage of compliant sites, the number of critical findings, or the time taken to implement corrective actions.

2. Determine audit frequency and timeline

The frequency and timing of your audits can significantly impact their effectiveness. While annual audits are common, the optimal frequency may vary depending on the complexity of your REMS program and the level of risk associated with the medication.

When establishing your audit timeline, consider the following:

• Create an annual audit cycle that aligns with your company's fiscal year or regulatory reporting periods
• Develop a month-by-month timeline for the entire cycle, including preparation, execution, and follow-up phases
• Allow flexibility for potential rescheduling or follow-up audits

For example, when working with a multinational pharmaceutical company, we implemented a six-month audit window from January to June. This extended timeframe allowed for both remote and on-site evaluations across multiple locations, providing ample time for thorough assessments while also accommodating potential scheduling conflicts.

It's also crucial to consider seasonal factors that might affect audit execution. For instance, avoid scheduling on-site audits during periods of known staff shortages or high workflow in healthcare settings, such as flu season for outpatient clinics.

3. Choosing between internal and external auditors

The decision to use internal auditors, external auditors, or a combination of both is a critical one that can significantly impact your audit program's perceived integrity and effectiveness.

• Internal auditors bring deep knowledge of your company's processes and culture, which can be invaluable in understanding the nuances of your REMS program. However, they may lack the fresh perspective that external auditors can provide, and their findings might be perceived as less objective.
• External auditors, on the other hand, offer an independent viewpoint and often bring broader industry experience. Their involvement can lend additional credibility to the audit process, especially in the eyes of regulators and the public.

For example, in our ongoing REMS audit program for a medication with potential birth defect risks, we provided external auditors to ensure unbiased evaluations. This decision was driven by the need to maintain public trust and comply with FDA requirements for third-party validation. The external auditors brought specialized expertise in REMS audits and were able to provide benchmarking insights based on their experience with similar programs at other companies.

4. Selecting and preparing qualified REMS auditors

Your REMS audit program's success hinges heavily on your auditors' quality and preparedness. Whether internal or external, these individuals need to possess not only the right skills and experience but also a deep understanding of your specific REMS program and its context within the broader regulatory landscape.

Speaking from experience, we know firsthand that the process of selecting auditors should be rigorous and systematic.

Create a detailed auditor qualification checklist that includes:

• Relevant educational background (e.g., pharmacy, medicine, public health)
• Years of experience in REMS audits or related fields
• Familiarity with FDA regulations and guidance related to REMS
• Understanding of healthcare delivery systems and pharmacy operations
• Strong analytical and communication skills

We highly recommend prioritizing auditors who have specific REMS audit experience, particularly with programs similar to yours. Their familiarity with common challenges and best practices can be invaluable.

Auditors must also understand your company's unique processes and culture. For long-term projects, consider the benefits of continuity. For example, in our multi-year REMS audit support program, we carefully selected auditors who had previously worked on similar projects with the client. This approach ensured familiarity with the company's processes, reduced ramp-up time, and allowed for more insightful audits that could track progress over time.

5. Providing the necessary training and resources

Even the most experienced auditors need proper ramp-up and resources to perform effectively in your specific REMS audit program — as each is different, and those differences have outsized impact.

Develop a comprehensive training program that covers:

• Detailed overview of your REMS program, including its goals, components, and known challenges
• Review of relevant FDA guidelines and how they apply to your program
• Company-specific protocols and systems related to REMS implementation
• Audit methodologies, tools, and reporting requirements

Very critically, create a simple resource library that auditors can reference throughout the audit process. This should include:

• Your REMS program documentation
• Relevant FDA guidelines and communications
• Company policies and standard operating procedures
• Any existing audit checklists and templates or material that needs to appear in new ones
• Previous audit reports and corrective action plans

Regular refresher training sessions are also important, particularly when there are changes to the REMS program or relevant regulations.

For example, in a complex REMS program involving multiple stakeholders, we implemented a system of constant communication to ensure auditors had access to up-to-date training materials, audit agendas, and necessary documentation throughout the project. This ongoing support was crucial in maintaining consistency across audits and allowing auditors to adapt to any changes in the program or regulatory environment.

Planning REMS Audit Program Logistics

The logistical planning phase of a REMS audit program is critical to its success. This stage involves making key decisions about audit types, scheduling, and preparing the necessary tools. Proper logistical planning ensures that audits are conducted efficiently, consistently, and with minimal disruption to the audited sites.

Let's break the logistical elements into their component parts and explain them one by one.

Determining audit types

One of the first logistical decisions in planning your REMS audit program is determining whether audits will be conducted on-site, remotely, or through a combination of both approaches. This decision impacts resource allocation, timelines, and the depth of the audit process.

When assessing each audit location, consider factors such as:

• The complexity of the REMS program at that site
• The site's history of compliance
• The availability of electronic records
• The need for physical inspections or in-person interviews
• Travel costs and time constraints

We always suggest creating a set of criteria for deciding between on-site and remote audits. For instance:

• High-risk sites or those with a history of non-compliance may require on-site audits
• Sites with fully digital record-keeping systems might be suitable for remote audits
• New sites or those undergoing significant changes may benefit from on-site evaluations

Sit down and develop specific protocols for each audit type to ensure consistency and thoroughness, regardless of the audit format.

For example, one of our audit support engagements involves both inpatient and outpatient pharmacies. The audit team conducted a mix of on-site and remote audits, tailoring the approach to each location's specific needs and constraints.

• With their complex workflows and higher-risk patients, inpatient pharmacies were prioritized for on-site audits.
• In contrast, many outpatient pharmacies, with their more standardized processes and electronic record-keeping, were effectively audited remotely. This flexible approach allowed for comprehensive coverage while optimizing resource use.

Scheduling REMS audits

Effective scheduling is crucial for the smooth execution of your audit program. It requires balancing the need for thorough audits with the practical constraints of time, resources, and site availability.

When creating your audit schedule:

• Develop a detailed master schedule for all audits, considering the audit type (on-site or remote) for each location
• Allow adequate time for each audit, typically 3-4 days per location for on-site audits, including travel time
• Build in buffer days to accommodate unexpected delays or the need for rescheduling
• Consider the workload and availability of both the audit team and the sites being audited

In practice, this careful scheduling pays off. In the project involving multiple pharmacy audits, we allocated four days per pharmacy for travel and audit execution. This generous time allocation provided flexibility to deal with unexpected issues while maintaining overall efficiency. When one pharmacy needed extra time due to a complex issue uncovered during the audit, the buffer in the schedule allowed for this without disrupting the entire audit timeline.

Preparing audit tools and checklists

Standardized audit tools and checklists are essential for ensuring consistency across audits and comprehensiveness in evaluating each aspect of the REMS program.

When preparing your audit tools:

• Develop standardized audit tools tailored to your specific REMS program
• Create detailed checklists for each aspect of the REMS to be audited, ensuring all critical elements are covered
• Design user-friendly templates for audit reports that facilitate clear communication of findings

Here are a few best practices for developing these tools we've learned over time:

• Align checklists with FDA guidance and regulations relevant to your REMS program
• Include both quantitative and qualitative assessment criteria
• Design tools that are flexible enough to accommodate site-specific variations while maintaining overall consistency
• Regularly review and update your tools based on audit experiences and any changes to the REMS program or regulations

We developed a comprehensive audit checklist for the birth defect prevention REMS program we supported that included sections on patient enrollment verification, prescriber education documentation, and pharmacy dispensing protocols. This standardized tool ensured that every audit, whether conducted on-site or remotely, covered all critical aspects of the REMS program consistently.

Conducting the Audits

The audit execution phase is where your planning and preparation are put to the test. This stage requires a balance of adherence to established protocols and flexibility to address unexpected situations.

Consistency in audit execution is key to obtaining reliable, comparable results across different sites and over time.

When conducting audits:

• Adhere closely to the predetermined schedule and methodology
• Use the standardized checklists and tools consistently across all audits
• Document any deviations from the plan and the reasons for them

However, rigid adherence to the plan should not come at the expense of addressing important issues that arise during the audit. Auditors should be empowered to explore significant findings, even if they fall somewhat outside the scope of the original plan.

For example, when we faced the need to reschedule certain audits due to site-specific requirements, they maintained flexibility while ensuring all necessary audits were completed within the project timeline. In one instance, a pharmacy undergoing renovation requested a delay in their on-site audit. The audit team quickly pivoted, conducting a partial remote audit to cover time-sensitive elements and rescheduling the on-site portion for after the renovation. This adaptive approach ensured comprehensive coverage without compromising the integrity of the audit process.

Maintaining open communication

Clear, consistent communication is vital throughout the audit process. It ensures that all stakeholders are informed, issues are addressed promptly, and the audit proceeds smoothly.

Establish a clear communication chain that includes regular check-ins between auditors and project managers (e.g., daily during audit periods), a system for rapid escalation of significant findings or issues, and clear lines of communication with all stakeholders.

Here are a few best practices from our REMS process:

• Hold brief daily debriefs with the audit team to discuss findings and adjust plans as needed
• Provide the client with regular status updates
• Establish a single point of contact for the client to streamline communication

We implement a tiered communication system. Auditors hold daily internal debriefs, project managers provide bi-weekly updates to the client, and establish a rapid escalation protocol for critical findings. This system ensures that, for example, when one auditor uncovers a significant dispensing error at a pharmacy, the information is quickly communicated to the client, allowing for immediate corrective action.

Documenting audit findings thoroughly

Thorough, clear documentation of audit findings is crucial for the credibility and usefulness of the audit. When documenting findings we always suggest using standardized reporting templates to ensure consistency across audits, ensuring all observations are backed by concrete evidence, and clearly classifying findings based on severity and potential impact on REMS effectiveness.

A few of our audit documentation best practices

• Using clear, objective language to describe observations
• Including specific examples to illustrate findings
• Clearly distinguishing between facts and auditor opinions or recommendations

For example, we developed a tiered classification system for audit findings: Critical (requiring immediate action), Major (requiring prompt attention), and Minor (opportunities for improvement). This system helped prioritize findings and guide corrective actions. In one audit, a critical finding related to incomplete patient counseling was immediately escalated, leading to rapid implementation of additional staff training.

Post-Audit Activities

The work doesn't end when the auditor leaves the site. Post-audit activities are crucial for ensuring that the audit findings lead to meaningful improvements in the REMS program. A thorough review process ensures that audit reports are accurate, clear, and actionable.

Implement a multi-stage review process:

1. Initial review by the lead auditor
2. Peer review by another experienced auditor
3. Final review by the project manager

Make sure all reports use consistent terminology and formatting, and verify that all findings are clearly described and supported by evidence.

Holding closeout meetings

Closeout meetings are an opportunity to ensure all stakeholders understand the audit findings and next steps. When conducting closeout meetings:

• Schedule a formal meeting with all key stakeholders
• Prepare a presentation summarizing key findings and recommendations
• Allow ample time for questions and clarifications

At the conclusion of a multi-site REMS audit project, we coordinated a comprehensive closeout meeting with the client's senior management, quality assurance team, and key site representatives. The meeting included a presentation of overall trends identified across sites, highlighted best practices observed, and outlined recommendations for program improvement.

Developing a post-audit action plan

Translating audit findings into concrete improvements is the ultimate goal of the audit process. Create a template for corrective and preventive action (CAPA) plans and assign clear responsibilities and deadlines for each action item. Follow this up by establishing a process for tracking and following up on action items.

We suggest prioritizing actions based on risk, setting realistic timelines, and ensuring action plans address root causes rather than just symptoms.

For example, when we identified inconsistencies in patient education across multiple sites, we worked with the client to develop a comprehensive CAPA plan. This plan included updating patient education materials, implementing a standardized training program for healthcare providers, and establishing a quarterly review process to ensure ongoing compliance. Clear ownership and deadlines were assigned for each action item, with a tracking system put in place to monitor progress.

Gather feedback

Continuous improvement of the audit process itself is key to maintaining an effective REMS audit program. To gather comprehensive feedback:

• Design a feedback survey for all involved parties (auditors, auditees, client stakeholders)
• Conduct a post-project review meeting to discuss lessons learned
• Use feedback to refine and improve the audit process for future cycles

For example, after completing a REMS audit project $10,000 under budget and receiving positive client feedback, we conducted a thorough review to identify successful strategies. This review included surveys of all team members and client stakeholders, as well as a detailed analysis of project metrics. The insights gained, such as the effectiveness of their mixed on-site and remote audit approach, were documented and incorporated into planning for future audits.

Final Thoughts and Next Steps

Implementing a robust REMS audit program requires meticulous planning, experienced personnel, and adaptable execution. 

The key to success lies in thorough preparation, consistent execution, clear communication, and a commitment to turning audit findings into tangible improvements. By investing in a well-designed REMS audit program, companies can enhance patient safety, maintain regulatory compliance, and build trust with healthcare providers, patients, and regulatory authorities.

Remember, each REMS program is unique, and audit approaches should be tailored to the specific needs and risks of each program. Regular review and refinement of your audit process, based on experience and feedback, will ensure your REMS audit program remains effective and efficient in the face of evolving regulatory landscapes and healthcare environments.

Plan and execute your REMS audits with The FDA Group.

Implementing a robust REMS audit program is crucial for ensuring patient safety, maintaining regulatory compliance, and safeguarding your company's reputation. But you don't have to navigate this complex landscape alone. 

Here are a few of the reasons firms work with us to audit their REMS programs:

• Proven Expertise: With years of experience in conducting REMS audits across various therapeutic areas, we bring unparalleled knowledge and insights to your program.
• Tailored Approach: We understand that each REMS program is unique. Our team crafts customized audit strategies that align with your specific needs and objectives.
• Regulatory Insight: Stay ahead of FDA requirements with our up-to-date knowledge of regulatory expectations and industry best practices.
• Efficiency and Cost-Effectiveness: Our streamlined processes and experienced auditors ensure thorough audits without unnecessary delays or expenses.
• Comprehensive Support: From planning and execution to follow-up and improvement strategies, we're with you every step of the way.

Fill out our quick contact form below, and we'll reach out to discuss how we can support your specific needs. Whether you're looking to overhaul your entire audit program or need support with specific components, we're here to help and respond within one business day.