In 2002, the Oakland Athletics team (the A’s) broke the world record, winning 20 consecutive baseball games with a limited budget and no star players. How?
In 2003, author Michael Lewis released his book Moneyball: The Art of Winning an Unfair Game, which explores how Billy Beane, the General Manager of the A’s, used a sophisticated statistical approach called sabermetrics to analyze and obtain undervalued players. He assembled a winning team at a fraction of the cost of hyped players instead of choosing players based on preferences, personal judgement, and gut feeling.
The success story of Moneyball stands as the base of inspiration for this case study, where the goal was to use statistical modeling to analyze digital biomarkers*, informing how a clinical trial with a limited budget can be successful.
Evaluating the Impact of Clinical Trial Technologies
The process of choosing biomarkers and digital biomarkers, and understanding their impact, is complex. Measuring the performance of a drug is one of the biggest challenges in clinical trials. Many recent Phase 3 trials in neurodegenerative disorders (Alzheimer’s, Parkinson’s, and Huntington’s disease) were terminated because they failed to reproduce the same results from Phase 2. In these drug trials, reliance on rating scales led to failures.
According to the Michael J Fox Foundation, Parkinson’s Priority Therapeutic Pipeline Report 2019, one of the obstacles in Parkinson’s disease drug development is that outcome measures lack precision. This leads to unreliable results, requiring large cohort sizes and extended follow-ups. Even experts in the field say that “The assays are simply not sensitive enough, and they create false signals,” and that if we want a new treatment for Alzheimer’s Disease or Parkinson’s Disease, “we need better measures.”
Subsequently, understanding digital biomarkers in clinical trials and quantifying their impact is crucial to optimizing clinical performance.
Hiro Mori, former Head of Early asset Technology at UCB, the Belgian biopharmaceutical company, took notice of this dilemma and wanted to evaluate the impact of different technologies in clinical trials. He explains that technologies are available but are often considered research gadgets or added as an afterthought without a business case. However, qualified digital biomarkers such as SV95C (Stride Velocity 95th centile) for Duchenne MD are emerging with credible promise.
Hiro underlined that digital biomarkers with higher accuracy would improve patient selection and outcome detection, and that quantitative simulation of study design would allow stakeholders to:
Identify the technology-enabled measurements with meaningful impact
Ground technology scouting conversations on the clinical performance benefits
Focus on and allocate resources to enable technology-inclusive study design ahead of time
Invest in digital projects upon evaluating their pipeline asset contribution business case
Clinical trials have a high level of uncertainty and risk. While innovations such as novel biomarkers introduce more uncertainty, they can also optimize clinical performance. To understand how to make technology adoption decisions, UCB and Captario partnered to quantify the effects of clinical trial technologies, using Captario SUM®, a SaaS analytics and simulation tool.
Quantifying the Impact of Clinical Trial Technologies – Case Study
A generic prototype was built in the platform Captario SUM®, as a proof of concept to evaluate the impact of:
Being selective of patients in a clinical trial
Using a specific outcome measurement as an endpoint for the clinical results
A hypothetical model based on a clinical study for Parkinson’s Disease was used as an experiment to:
Evaluate the technology DaTscan for patient selection by determining whether a patient has dopamine deficiency or not
Evaluate the accuracy of the outcome measurement using an SV95C-like tool.
The results from the simulation based on this hypothetical study show that the trial outcome for these Parkinson’s patients was optimized when DaTscan was used to choose patients with dopamine deficiency (patient selection) and when the treatment effect was analyzed after using an SV95C-like endpoint (outcome measurement accuracy). The trial study was optimized either by increasing the probability of study success (keeping the number of patients constant) or by decreasing the sample size (keeping the probability of success constant), as seen in the graphs below.
Illustration of technology’s potential impact on the probability of study success (PoSS).
Illustration of technology’s potential impact on sample size.
More importantly, however, this case study showed that it is possible to quantify and evaluate the impact of technologies to optimize clinical performance.
Conclusion
This case study is a prime example of how statistical modeling and simulation can provide insight into new and better opportunities. Inspired by Moneyball and how Billy Beane used statistics to buy baseball wins rather than specific players, the same approach can be applied in clinical trials. The focus shouldn’t be on purchasing specific technologies but on achieving clinical study success. Like Billy Beane, this case study utilized statistical modeling as an affordable way of optimizing the best approach to achieve the desired outcome.
“It’s all about evaluating skills and putting a price on them. Thirty years ago, stockbrokers used to buy stock strictly by feel. Let's put it this way: Anyone in the game with a 401(k) has a choice. They can choose a fund manager who manages their retirement by gut instinct or one who chooses by research and analysis. I know which way I'd choose.” – Billy Beane
For further in-depth analysis, this case study can be extended to a holistic model of the entire development process using the Captario SUM® platform. This tool enables project and portfolio leaders to analyze the effects of technologies and other decisions made in the clinical phase on Market, NPV, ROI, launch date, and other metrics to answer strategic questions such as
How much faster will we reach the market?
Will our order of entry to the market affect our market share?
How much will the effects of the drug influence the price and thus sales and NPV?
What is the overall return on investment of different choices?
If you want to know more about this case study, the prototype, and the methodology, you can read about it in the Digital Biomarker Journal. You can also reach out to Hiro Mori, Stephanie Mardini, or Stig-Johan Wiklund.
About UCB
UCB is a Belgian biopharmaceutical company focused on creating value for people living with severe diseases in immunology and neurology now and into the future. They work with stakeholders to address the unmet needs of patients and caregivers, helping them to achieve their goals and to live the lives they want. UCB is on a journey to become the patient-preferred biopharma leader by delivering medicines and solutions that improve lives while creating value for society. Their commitment to value is a promise to bring together the talent, expertise, tools, and scientific know-how required to serve patients in need.
References
Presentation by Hiro Mori that refers to The first frontier: digital biomarkers for neurodegenerative conditions.
Presentation by Hiro Mori that refers to E. Ray Dorsey at the Innovation Forum Digital Biomarkers at Basel Life 2017, Basel, September 12
* Biomarker – measuring indication of a biological state (for example, blood pressure) * Digital biomarkers – quantifiable physiological and behavioral data that are collected and measured by digital devices such as portables, wearables, implantables or digestibles