« Back

February 26, 2016

Accrual Index, A Novel Way of Measuring the Timeliness of Clinical Study Enrollment
By Rhonda Kost

Investigator-initiated studies that enroll (accrue) research participants more slowly than expected delay scientific discovery. Recent studies documenting the failure of many clinical trials at the national level to accrue participants on time, or to ever reach study endpoints, led to calls for accountability in study accrual from the Institutes of Medicine and funding agencies such as NIH/NCATS/CTSA. These agencies would like to standardize the evaluation of accrual success across the CTSAs nationally, but no validated measure of this success has yet been developed.

To speed study accrual in protocols conducted by investigators in the Rockefeller Center for Clinical and Translational Science (CCTS), Dr. Rhonda Kost, Clinical Research Officer, andmembers of Clinical Research Recruitment and Outreach Support Service (CRROSS), Lauren Corregano MSW, and Katelyn Bastert MS, have developed data-driven recruitment practices and documented the positive impact of focused data capture and analysis on recruitment success. They have demonstrated the benefits of incorporating recruitment planning early in the Protocol Navigation process to optimize the many factors that impact on the ability to recruit participants into a study.

Estimating the availability of the target population is a core element of assessing recruitment feasibility estimated through review of prevalence data, prior recruitment experience, and the presence of eligible volunteers registered in the Research Volunteer Repository. The recruitment staff initially focused on reducing the time from activation of recruitment to the first participant screening visit which in 2014 was a median of just 10 days. More recently, they devised a new measure, the Accrual Index (AI) to provide near real-time assessment of whether study enrollment proceeding in alignment with the recruitment plan predictions. A paper describing the AI was recently published in the journal Clinical and Translational Science (8:655, 2015).

Summary of the steps in the development of the AI:
Formulating a Measure I– Participants: The Accrual Target is the number of evaluable participants needed to complete the study, captured as the sample size from the calculation in the IRB approved protocol performed to judge how many individuals need to be studied to have a good chance of obtaining a statistically valid result. A coarse measure of enrollment progress toward the completion goal can be expressed as

Percent Accrual=Evaluable Patients/
Accrual Target
Percent Accrual is useful to illustrate how study enrollment fared at the end of the study, but the absence of a time indicator limits its use in judging the status of studies that are still open to enrollment.
Formulating a Measure II – Adding Time: Since the inception of the CCTS, the CRROSS team has increasingly refined its approach to estimating how long it will take to complete accrual into a study, assuming the availability of the population has been properly assessed. From 2007-2010 CRROSS estimated burdens and incentives for participants as key factors in accrual success; in 2011-2012 it began to incorporate specific investigator availability information; and in 2013-2014 it delved deeper to incorporate predictable lags on recruitment timelines such as leaves of absence, vacations, delays for assay refinement, predictable months of recruitment doldrums, and required FDA review periods, as well as other factors. To obtain a measure of the impact of these improvements in enrollment prediction, CRROSS developed a simple measure to compare its recruitment performance across studies spanning a broad range of accrual targets and anticipated durations.
The measure: The AI places the progress of accrual in the context of the anticipated time frame at any moment during study conduct and is calculated from the equation:

Progress toward Accrual Target/ Time Elapsed Relative to Total Target Time = AI

When accrual progress is on-time the AI =1. When accrual lags, AI is less than 1, and when accrual is ahead of schedule, AI is great than 1. The simplicity of the Index makes it possible to rapidly assess the effectiveness of recruitment strategies across studies and across time. Thus, every study can be immediately evaluated throughout the life of the study rather than having to wait for its completion.
The full equation is below:

(Evaluable Subjects Enrolled/Accrual Target)/((Days since Recruitment Start)/(Predicted Number of Days to Accrual Completion))

Using data captured in the IRB approval protocols, iRIS study management data, and the recruitment core’s database, CRROSS has evaluated AI for every study in which recruitment was initiated at the CCTS between 2007-2014. The major trend in the past two years has been toward a narrower range of AIs for open and completed protocols centered around an AI of 1, which indicates both the increasing precision the CRROSS group has achieved in predicting how long it will take to accrue research participants and the success of the recruitment strategies developed by investigators in collaboration with CRROSS staff. For example, more than 50% of studies actively enrolling, or completing/closing enrollment in the year 2014 had AIs between 0.65 and 1.2.

We also created a dashboard (below) to assess accrual in real time for studies active at the CCTS.



The dashboard displays the AI for the prior month, the AI for current month, the trend in AI (change over time), and uses conditional formatting to create a visual alert for studies that are on-time or ahead of schedule (green), on the border of timeliness (yellow), or behind schedule (red) for timely accrual.
Although the PTAC is incorporated into the AI, staff finds it useful to include the percent PTAC elapsed on the dashboard for sorting studies according to how far along they are in the study life cycle. This type of dashboard is reviewed by the Clinical Research Office’s recruitment team monthly, Investigators can anticipate reviewing the dashboard and AI data for their studies regularly with the recruitment team. The dashboard is reviewed at intervals by the Senior Staff as part of the review of hospital operations. Starting in 2016, the ACCTS will review the dashboard as part of its accountability and charge to oversee the use of resources and services provided to investigators in alignment with the CCTS mission to accelerate the completion of translational research.