Biostatistics, Epidemiology, and Research Design Trailblazer Awardee: Neal Montgomery, Ph.D.

Project Title: Bayesian Methods to Improve the Design and Analysis of Feasibility Studies

Neal Montgomery, Ph.D., is an assistant professor in the Department of Biostatistics and Data Science at the University of Kansas Medical Center. A Kansas City-area native, Montgomery earned his bachelor’s degree in mathematics from the University of Kansas and his master’s and doctoral degrees in biostatistics from KU Medical Center.

Now, his methodological research focuses on early-stage clinical trials, especially pilot and feasibility studies. These studies are designed to answer a practical yet essential question: before researchers invest in a large, definitive clinical trial, can the trial actually be conducted?

Montgomery’s Biostatistics, Epidemiology, and Research Design Trailblazer Award focuses on developing better statistical methods to answer that question.

“When we’re talking about the research pipeline, a lot of times people think about the late-stage definitive clinical trials,” Montgomery said. “These are the trials that are to be kind of the definitive word on whether some sort of intervention works.”

Those large trials are important, but many do not succeed as planned. Some fail to recruit enough participants, some struggle to retain participants, and some stop early without collecting enough data to answer the clinical question. When that happens, the consequences extend beyond the research team. Funding, staff time, and participant effort may be spent without producing a clear answer.

“That’s a huge misuse of resources,” Montgomery said. “It is a huge waste of participants' time. Patients take time out of their schedule; they accept some sort of risk, usually to participate in a clinical trial, and then you do not actually answer the clinical question.”

Pilot and feasibility studies are often conducted before a large clinical trial to help determine whether the larger trial can be completed. These studies may examine recruitment, retention, treatment fidelity, acceptability, and other outcomes that affect the feasibility of a larger trial. But Montgomery said the way these studies are commonly analyzed is often too simplistic for the decisions they are expected to support.

A common approach is known as a “traffic-light” system, where individual outcomes are categorized as “go,” “amend,” or “stop.” For example, a study might set a retention threshold and decide that retention above a certain level allows the trial to proceed, while a lower level requires the design to be amended or stopped. While intuitive, that approach often evaluates outcomes separately, even when those outcomes are related.

Recruitment, retention, treatment fidelity, participant burden, and early signals of whether an intervention may work can influence one another. If an intervention is difficult to follow, fidelity may be lower. If researchers increase participant burden to improve fidelity, retention may decline. If retention is lower than expected, the future trial may need to recruit more participants than originally planned. Looking at each factor in isolation can miss the larger picture.

“There’s a functional relationship between the efficacy, the recruitment rate, and the retention rate,” Montgomery said. “Analyzing those independently is an inappropriate way to do it, in my opinion. You are throwing away data.”

Montgomery has developed a Bayesian statistical method to estimate the probability that a future randomized controlled trial will be feasible, based on pilot data. Bayesian methods allow researchers to update probability estimates as data are observed. In this case, the method can evaluate multiple feasibility outcomes together and anchor the decision to the planned definitive trial, rather than relying only on rule-of-thumb thresholds.

Through his BERD Trailblazer Award, Montgomery will extend that method to incorporate efficacy data. In pilot and feasibility studies, researchers are generally advised not to draw formal conclusions about whether an intervention works, as these studies are usually too small and variable for that purpose. Montgomery agrees with that caution. His goal is not to use pilot data to prove efficacy, but to use those early signals to make better decisions about whether a future trial is feasible.

For example, if a small pilot study shows acceptable recruitment and retention but only a weak signal of efficacy, the planned larger trial may require a different design, more sites, or a larger sample size than originally expected. Conversely, a stronger-than-expected signal of efficacy could change how researchers interpret recruitment or retention data for future trials.

“I think we can use efficacy data to help us make better decisions about feasibility,” Montgomery said.

The project has two major goals. First, Montgomery will extend his existing Bayesian methodology to incorporate continuous and binary efficacy outcomes from one- and two-arm studies. He will use simulation studies to test how well the method performs and how sensitive it is to underlying assumptions. Second, he will create tools that make it easier for researchers to use the methods, including an R package, a user-friendly Shiny application, and tutorials designed for non-statistical audiences.

That software component is central to the project’s potential reach. Many early-stage study teams lack extensive statistical or programming support. By creating easier-to-use tools, Montgomery hopes to make more rigorous feasibility analysis available to a wider range of research teams.

The work may also support reviewers of grants and manuscripts. Montgomery said reviewers often see preliminary data but may lack clear tools to evaluate whether those data truly support moving forward with a larger trial. Future software could help reviewers input preliminary data summaries and estimate the probability that a proposed definitive trial is feasible.

“If we could get buy-in from researchers, the NIH, and reviewers, I think it would greatly reduce the number of trials that are proposed that shouldn’t be proposed,” Montgomery said. “It would reduce the number of trials that fail for avoidable things like poor recruitment, poor retention.”

Montgomery sees the project as part of a broader effort to increase rigor and reduce waste across clinical research. The method is not disease-specific; it could be applied to pilot and feasibility studies in cancer, Alzheimer’s disease, obesity, implementation science, nephrology, and other areas where researchers need to decide whether and how to move from early-stage work to a definitive trial.

If successful, this BERD Trailblazer Award will lay the groundwork for a larger NIH methodological grant. The long-term goal is to build a more comprehensive set of tools for designing, analyzing, and reviewing pilot and feasibility studies.

“If this caught on, if this became widespread, I think it would greatly improve the rigor,” Montgomery said. “It would reduce waste, and it would reduce the misuse of participants’ time.”

For Montgomery, the aim is not to make research more difficult for its own sake. It is to make early-stage studies more useful, so that when large clinical trials do move forward, they are more likely to recruit the participants they need, answer the questions they were designed to answer, and respect the time and contribution of the people who participate.