Utilizing Data and Key Performance Indicators to Successfully Execute a Phase III Influenza Treatment Clinical Trial
SGS was recently engaged in a global Phase III influenza treatment clinical trial, providing both overall project management and study monitoring. The trial was one of the largest influenza treatment trials conducted to date, and as a result presented several challenges, due in part to the seasonality of influenza and the geographic imbalance in the outbreaks. This paper will address how SGS utilized constant data tracking to mitigate the disease-specific challenges and successfully execute this Phase III clinical trial.
The study team focused on integrating quality into every aspect of the trial. SGS tracked several key performance indicators (KPIs) to measure and monitor the quality of study conduct in four main areas:
- Start-Up Timeline Adherence
- Site Engagement
- Influenza Positivity
- Data Monitoring
The trial ran over multiple flu seasons in more than 150 sites throughout several countries. In order to ensure the opportunity to recruit subjects year-round, the distribution of sites spanned both hemispheres. Enrollment into the study alternated between the two hemispheres, occurring from November through March in the Northern Hemisphere and from April through August in the Southern Hemisphere (Figure 1). The sites screened for subjects with signs and symptoms of uncomplicated influenza.
Start-Up Timeline Adherence
Due to the seasonality of influenza, sites may only have a period of six to eight weeks to actively recruit for the clinical trial in a given year. This challenge is compounded by the significant seasonal variability of outbreaks and peak activity, which remains unpredictable. Per Figure 2, the flu season in the United States historically occurs in February; however, historical records show seasonal peaks are possible as early as October or November. Therefore, it was very important to prepare sites for initiation early enough that the peak of flu season was not missed while also taking into account that sites should not actually recruit for the study until flu activity was confirmed in their areas.
For this study, the target date for initiation of the majority of sites was mid-November during flu season in the Northern Hemisphere. This allowed sites to be open and ready to start enrollment during the time period with highest flu incidence. In order to ensure that study start-up activities were progressing quickly enough to achieve this goal, the SGS study management team tracked two relevant KPIs:
- Percentage of contracts to be executed by November 1st
- Time from Institutional Review Board (IRB) approval to Site Initiation Visit (SIV) for each site
To monitor the KPI related to site contracts, SGS tracked the number of Clinical Trial Agreements signed each week as well as the cumulative number of contracts near final negotiations. These values were then used to project the total number of agreements expected by the first of November, the goal of which was 90% of all sites. The metric for the timing of SIVs was tracked on a site-by-site basis with a target of initiating each site within 10 business days of IRB approval. Monitoring this KPI allowed the SGS team to ensure that the opening of sites was done promptly following all required start-up activities and that it was not delayed due to staff availability or scheduling issues. Although the primary objective was to initiate sites within ten days following IRB approval, flu incidence rates in the respective areas were used to determine if there was a need for a more rapid initiation of each site on a case-by-case basis.
A critical factor to the success of any given clinical trial is the performance and commitment of the study sites. While the sponsor and CRO can provide planning and careful monitoring, the sites are vital contributors to studies because they drive recruitment and overall data quality. In this particular study, promoting a strong commitment of the sites was especially important given the short window to recruit subjects and the need for prompt attention to data queries. The SGS study management team worked diligently to keep sites engaged in both recruitment activities and ongoing data cleaning through regular monitoring visits and frequent phone or email contact. SGS tracked three relevant metrics to examine site performance:
- Number of days from site activation to first subject enrolled at each site
- Average number of days to complete electronic Case Report Forms (eCRFs) following a subject visit
- Average number of days to resolve queries
Sites were considered active and ready to recruit following the SIV and once the presence of influenza was confirmed in the local community. It was essential to know that enrollment was progressing in areas where influenza was prevalent so that sites did not miss the transient flu activity in their areas. The KPI for first subject enrolled looked at the number of sites which enrolled a subject within one week of activation. The overall goal was to ensure that at least 75% of sites were enrolling within this timeframe. SGS Clinical Research Associates (CRAs) followed up closely with sites that had difficulty meeting this goal and provided support and suggestions to facilitate recruitment.
Once a site did enroll subjects, the SGS team calculated the number of days from each subject visit to completion of the respective eCRFs by site staff. The goal for this metric was to see data entered within five days. Additionally, SGS monitored a KPI on data queries in order to assess sites’ attentiveness to pending data clarifications in the Electronic Data Capture (EDC) system. We tracked the total number of days that queries remained open and expected to see at least 90% of queries closed, including additional source data verification by the CRA as needed, within 30 days of initial generation. The metrics on data cleaning allowed the study management team to identify sites that needed more frequent follow-up between interim monitoring visits.
Recruitment was based on the presence of signs and symptoms of influenza (versus a confirmed diagnosis of flu by means of a rapid antigen or polymerase chain reaction test). The study team took several measures, both at the site and study levels, to maximize the flu positivity rate of enrolled subjects. Again it was important for the study team to use metrics to track overall quality. In this case, the two metrics tracked were:
- Overall study-wide flu positivity
- Site-level flu positivity rates.
The goal for flu positivity was 45% across all sites. By calculating this value, the management team was able to confirm that the targeted activation of sites based on the presence of influenza in the community was, in fact, effective. The site-level KPI was a bit different in that we tracked the number of flu positive subjects enrolled at each site and monitored for multiple negative subjects in a row. If a site enrolled four consecutive flu-negative subjects, that site was placed on an enrollment hold pending retraining and reconfirmation of local influenza activity. The retraining for the site included topics such as inclusion/exclusion criteria, collection of nasopharyngeal swabs, and proper handling and shipping of samples. Following enrollment of eight consecutive flu-negative subjects, a site was directed to permanently halt recruitment. This was an unfortunate but necessary step in order to ensure optimal use of investigational product, which meant providing drug to sites that were successfully enrolling flu-positive subjects.
In addition to tracking metrics on flu positivity, SGS implemented community surveillance plans to serve as tools to help sites confirm local influenza activity. These were site-specific plans which required each site to identify a combination of public health surveillance sources and local certified laboratories that could provide data on confirmed influenza cases. The site then needed to obtain documentation from one of the identified sources to serve as evidence of local flu activity. The sponsor encouraged the use of local laboratories, such as those embedded within hospitals, to provide data as this generally resulted in more contemporaneous, localized flu confirmation versus relying on public health surveillance data. A completed community surveillance plan verified by the Principal Investigator was required in order for a site to be considered activated and ready to recruit.
With highly-engaged sites working diligently to recruit flu-positive subjects, SGS was tasked with providing thorough clinical trial monitoring of a vast amount of data on a very tight timeline.
Achieving the goal for database lock required a very large, dynamic team of CRAs as well as close monitoring of CRA workload by the study management team. Not only was there a substantial amount of data in this trial, there were two main added challenges specific to the indication. Due to the seasonality of influenza, enrollment in the Northern Hemisphere was concentrated during the months of December, January and February. This spike in enrollment resulted in a large surge of eCRFs generated in a short period of time, as shown in Figure 3.
Additionally, the data was not evenly dispersed across all sites. The top 15 enrolling sites in the U.S. recruited approximately half of all enrolled subjects, meaning about 10% of the sites generated roughly 50% of the data (Figure 4). Because of the uneven distribution of data, it was not effective to use the traditional model of allocating a certain number of sites to each CRA. Therefore, the SGS team carefully tracked the number of subjects assigned to each CRA. If the workload of one CRA became too high, it was necessary to quickly readjust the distribution of sites and subjects.
In addition to watching the CRA workload, the study team tracked the percent of data that was Source Data Verified (SDVed), at both the study and site levels, on a near-daily basis to ensure the monitoring continually progressed even as enrollment was occurring very rapidly. If the percent of data SDVed at one or more sites dropped significantly, the team arranged for CRAs to attend visits as co-monitors to assist with data verification. Assessing these metrics and adjusting the allocation of resources frequently was essential to locking the database promptly after completion of the study.
The SGS study management team dedicated a great deal of time to tracking and calculating multiple metrics on a frequent, often daily, basis. The information obtained through analysis of these KPIs was crucial to steering the day-to-day study action plan and to guiding the trial to a successful database lock. SGS applies this practice across all therapeutic areas, defining relevant, indication-specific metrics to improve overall study quality and efficiency.
Proje ct Manager – Clinical Research
SGS Life Science Services