Improving trial study start time using Google’s Gemini AI technology.
Today’s process is completely manual and takes several weeks possibly months, just to complete the first step. We approached this project in iterations, solving for the first step in the start-up of a trial study.
Implementing Google’s Gemini AI technology gave the team the ability to review the data AI parsed from the documents and either approve, add or edit. This enabled the team to move to the next step in the trial study start-up process, saving time and money.
Take a manual, human-centric process and use AI to automate processes with human oversight.
Who will be using our Sponsor Protocol AI tool?
-
Research Study Management
Team of people who work with study coordinators to get information from the Sponsor Protocol document to downstream teams. Manage the entire lifecycle of a trial study start-up.
-
Study Coordinator
Assist the Research Study Management team with all downstream activities.
-
Upper Management, Directors
Oversea the progress of a trial study and report to senior leadership. Want to know the status of a trial study and any road blocks the team might be facing.
Pivot and shift as needed to get the job done.
I served as the Product Manager and UX Designer on this project, working side-by-side with our Subject Matter Expert. Given the nature of the business, I took an unconventional process to gather functional requirements. I used a reactionary method to low-fidelity wireframes rather than how I typically uncover current day processes and where we can make improvements.
Product Goals and Road Map
Long Term Goals and Vision
We want this product to serve as the central location for study activation. A place where end-users can plan the necessary steps for Trial Study Activation, review and track each step.
Align this product to the various works streams for the applicable stages of every Study Trial start-up such as revenue, lab, pharmacy, imaging, etc.
Immediate Goals and Vision
This product will still serve as the single source of truth for study activation however we will break it down to capture every workflow our end-user takes. And with each iteration, we get closer to housing everything a trial study start-up takes to launch.
Challenges
Every sponsor uses a different protocol template such as Investigator Initiated Trial Studies
Different Trail Study types need different information (questions) answered)
Device Trial Studies are not as straight forward as Drug Trial Studies
For example, long term sample storage and the possibility of future research would not be necessary for a device trial study but would for a drug study if, for example, it related to an explanted valve and we needed information about tissue.
User Research
Rather than following the users manual process today, I started from our goal in this first iteration and backed out from there. The users goal, is to create a document called Summary Logistics Sheet. This document contains information about how the trial study needs to be set-up. The information that populates the Summary Logistics Sheet comes from the Sponsor Protocol document (details about the drug the trial study sponsor wishes to test).
Today’s Process
What is a sponsor protocol document?
A sponsor protocol document is provided by the drug sponsor of the trial study. For example, Pfizer has a drug they would like to test and bring to market, Pfizer will provide the parameters of how the study will be setup and guide to how study is being done.
UX AI Principles
What are UX principles when using AI technology to solve user problems? As we ideate, we want to use AI to improve the users experience, not replace the person.
Empowerment
Enhance users’ skills and decision-making without replacing them. Enable the user to do the work they want to do, not the work they are burdened by doing. Ensure accessibility for a diverse range of abilities.
Explainability & Resources
Highlight the factors and criteria that correlate to the output. Cite sources.
Ethics & Bias
Plan, anticipate, and design for likely bias. Ensure the AI is used ethically and respects user privacy.
Control
Users should feel in control of the AI and be able to override or change decisions made by the AI if necessary.
Feedback
Collect feedback from users to ensure that we are listening and learning how to improve our models. Explain how their feedback will improve the AI.
Simplicity
Help users by utilizing relevant context and common conventions to reduce cognitive load. Do not overwhelm users with information.
Transparency & Limitations
Help users understand generally how the AI was trained. Be clear about limitations of the AI.
Outputs & Errors
Anticipate and mitigate errors to avoid undermining user trust and confidence. Plan for a variety of potential unintended use cases.
Working closely with our SME, we looked at the Sponsor Protocol document and identified what questions our user is currently asking themselves when they review this document today. Remember, they manually read this 200-300 page Sponsor Protocol PDF document in order to populate the Summary Logistics Sheet.
The result, we identified 53 question/answer sets the user would use the LLM to inform the Summary Logistics Sheet. In addition, some of these question/answer sets had follow-up questions and required branching logic.
How do we use Gemini AI to collect content from the Sponsor Protocol document and populate the Summary Logistics Sheet?
“How do we streamline the question/answer sets so they are applicable to every study type?”
Ideate
Once the Summary Logistics Sheet is complete, the next step the user takes is to work with downstream teams to complete further tasks in the trial study start-up process.
But is the Summary Logistics Sheet really complete and ready to go, with the first response from the LLM? Does the user need to edit the LLM response in any way?
A sponsor protocol document is provided by the drug sponsor of the trial study. For example, Pfizer has a drug they would like to test and bring to market, Pfizer will provide the parameters of how the study will be setup and guide to how study is being done.
Gaining feedback from stakeholders in an unconventional way, allowed me to build the artifacts we need to keep communication moving throughout this product’s development.
Now that the LLM is pulling our data, what would the RPS do next?
Parse data from document using Gemini AI technology
4. Ability to leave communicate with other team members.
2. Ability to edit the LLM data pull
5. Birds-eye view of progress.
3. Ability to run pre-defined and custom follow-up questions against the same Sponsor Protocol document to gather further information
6. Version control and document repository
Often times there are new versions of the Sponsor Protocol document that need to be run and amend the previous LLM response. How do we track those updates?
UX Ideate Phase
During the user research phase of this product development, we uncovered that users at different clinic sites approach the process differently.
Furthermore, different types of trial studies will require different ways of working. For example, one study type might not require further downstream team discussions based on the information provided in the sponsor protocol document and others would.
Streamline processes
“Asking, what would the impact be if we released this set of functionality for every study type?”