Virtual Clinical Trials: Challenges and Opportunities

Workshop Highlights 2019

The Forum on Drug Discovery, Development, and Translation of the National Academies of Sciences, Engineering, and Medicine hosted a workshop in Washington, DC, Virtual Clinical Trials: Challenges and Opportunities, held November 28–29, 2018. This workshop examined the current clinical trials infrastructure and potential opportunities for supporting the practical implementation of virtual clinical trials. Workshop participants discussed inefficiencies of the current clinical trial enterprise; the boundaries of what might be considered a virtual clinical trial; the opportunities to expand access for patients; perspectives of using digital health technologies in clinical care and observational and interventional studies; the impact of using digital health technologies on access and equity to clinical trials; the policy landscape governing clinical trials; and possible opportunities for future action.

Defining "Virtual Clinical Trials"

Workshop participants used a variety of terms to refer to clinical trials in which all or part of the study incorporates digital health technologies and enables remote participation. Clay Johnston, Dell Medical School, The University of Texas at Austin, observed that an adequate umbrella term is not easy to identify. He commented that terms such as “decentralized,” “remote,” or “site–agnostic” may only describe some types of trials that incorporate digital health technologies. Ray Dorsey, University of Rochester, added that many study activities still require a centralized location. Additional terms referred to by workshop participants include “direct-to-participant,” “location–variable,” and “mobile.” Each of these terms highlights different aspects of how digital health technologies may be incorporated into study design.

Opportunities to Improve Clinical Trials

According to Dorsey, clinical trials, as currently conducted, are expensive, inefficient, and inaccessible. Furthermore, he added that they fail to represent the patient population adequately. He envisions that in the near term, clinical trials could be conducted centrally (at one trial site), at multiple individual trial sites, and/or remotely (via digital health technologies), depending on the type of data needed. Dorsey provided examples that illustrate how virtual clinical trials can increase participant access and geographic representation, improve participant experience, and enhance recruitment of patient subpopulations.

Donna Cryer, Global Liver Institute, stated that a quality clinical trial is one that generates the minimal amount of credible, replicable, and evaluable data needed to answer meaningful questions with the least time and cost burden on participants. She expressed hope that as more virtual clinical trials are conducted and patient communities are better engaged, the quality of endpoints and outcome measurements will improve such that trials can more effectively address questions about a patient’s quality of life. Cryer emphasized that these types of trials offer opportunities to foster ongoing relationships with participants, better understand clinical conditions longitudinally, and generate new and relevant research questions.

Craig Lipset, Pfizer, Inc., described a 2011 study conducted by Pfizer, REMOTE, which was designed to validate available virtual technologies by repeating a standard brick-and-mortar clinical trial that Pfizer had conducted for Detrol, a drug used to treat overactive bladder. REMOTE was discontinued due to recruitment issues, but it successfully demonstrated the ability of researchers to remotely screen patients, seek informed consent, monitor safety, and capture required data to demonstrate safety and efficacy. According to Lipset, the study did not operate at the limit of available technology at the time it was conducted, nor did it require new legislation, safe harbor, or guidance from regulators. Lipset emphasized that moving beyond such pilot programs may require a change in will and culture by stakeholders across the clinical trial enterprise.

Lessons Learned from Virtual Clinical Trials

  Jenny Bollyky, Livongo, described three challenges associated with remote management of chronic conditions: (1) attributing digitally collected data to the individual under observation, (2) providing meaningful data for the participant being monitored (e.g., instant feedback on blood glucose levels for people living with diabetes), and (3) coordinating care for people with chronic conditions. Leveraging real-world evidence in a research setting may be complicated by the Hawthorne effect—the alteration of behavior by the subjects of a study due to their awareness of being observed, noted Bollyky.

  Joshua Denny, Vanderbilt University Medical Center, discussed how longitudinal research on large cohorts that aggregates data from a variety of sources (e.g. electronic health records, participant provided data, collected biospecimens, claims data), can facilitate trials. He focused on three examples: All of Us, U.K. Biobank, and Project Baseline. The combination of data that emerge from cohort studies, noted Denny, could be leveraged to conduct a wide range of clinical research activities, including the identification of new disease targets and pharmacogenomic discovery, testing for adverse drug events, and identifying disease subtypes. Furthermore, Denny highlighted that large, diverse observational cohort studies can enable direct and targeted recruitment of diverse populations, in addition to more intelligent trial design.

Steven Cummings, San Francisco Coordinating Center, discussed what he termed “direct-to-participant” (D2P) trials, which he defined as having no physical clinical sites, and thus no geographic limits on recruitment. He argued that the term ‘D2P trials’ more aptly captures the importance of building relationships with participants and that this approach can simplify trial design (i.e., improve enrollment, screening, data collection and reporting, etc.) thus increasing the likelihood of trial success. He added that recruitment from trusted communities and providers could lend D2P trials more success than solely web-based recruitment.

Kimberly Hawkins, Sanofi Genzyme, highlighted opportunities that decentralized trials provide, including increased geographic flexibility and reduced burden for participants; continuous data collection for faster and more accurate detection of health signals; and improved long-term follow-up with participants. Hawkins also listed challenges that decentralized trials can pose, such as operational (e.g., integration of emerging types of data and maintaining a temperature-controlled supply chain), regulatory (e.g., endpoint and digital health technology validation), and change management (e.g., integration of decentralized trials into medical product development).

Wendy Weber, National Center for Complementary and Integrative Health, discussed the National Institutes of Health’s Health Care Systems Research Collaboratory, and its role in supporting embedded clinical trials, or research that engages health care delivery systems as partners. Such trials can leverage electronic health records (EHRs), which can provide a cost-effective source of information collected during routine care. However, as Weber mentioned, EHRs may not always reflect the schedule of data collection necessary for a clinical trial nor include the type of data needed. As such, embedded pragmatic trials may work best when the outcome of interest is captured in the EHR and when the trial itself does not place undue burden on patients and clinicians during routine health care visits, noted Weber.


Noah Craft, Science 37, emphasized that successful D2P trial design should fit participant needs. Adrian Hernandez, Duke University School of Medicine, added that the field needs to develop the science of patient engagement to better understand patient preferences and participation in clinical research. Jon White, Office of The National Coordinator for Health Information Technology (ONC), stated that ONC and the Centers for Medicare & Medicaid Services have issued proposed rules for how the federal government will regulate health information systems to make EHR data more readily available through application program interfaces. Josh Rose, IQVIA, said that flexibility is key when interacting with participants—some clinical trial participants want more in-person interaction while others prefer to use just the technology.


Access and Equity

Silas Buchanan, Institute of eHealth Equity, emphasized the importance of engaging directly with community members when deploying digital interventions. Building a network of partnerships and leveraging trust brokers within the community can be instrumental in the success of public health campaigns. Using his social impact firm, Institute for eHealth Equity, as an example, Buchanan provided key lessons learned for how virtual clinical trials can be designed and positioned to increase inclusion of underrepresented populations and how to address the unique socioeconomic factors those populations face, if specific trial design considerations call for it. A strong ethos of community engagement is key to the introduction of digital health tools, emphasized Buchanan, as is acknowledgement of histories of discrimination and transparent discussion about power and responsibilities.

Sherine El-Toukhy, National Institute on Minority Health and Health Disparities, discussed the inclusion of minority populations in research and commented on barriers to participation, such as skepticism about the researchers and value of the study. She suggested that while health information technology may reduce health inequities, it can unintentionally exacerbate existing disparities or create new ones. She emphasized that understanding and prioritizing the target population’s needs, values, and preferences is critical for designing culturally and linguistically appropriate clinical trial recruitment material.

Will McIntyre, The Michael J. Fox Foundation, pointed out that trial participation often reflects the urban–rural divide. He then discussed how synergy between the research and technology sectors could equip study participants with technologies that will better enable those living in rural areas to connect with studies.

Sally Okun, PatientsLikeMe, commented on how virtual clinical trials can use unique designs to create new insights and increase participation rates of those who are typically excluded from clinical trials. ALSUntangled, a research consortium, seeks to understand the efficacy of alternative and off-label treatments for amyotrophic lateral sclerosis (ALS) and engages in patient-driven inquiry. In one its studies, ALSUntangled expanded its inclusion criteria such that those with more advanced forms of ALS could participate.

Participant engagement could be improved by returning individual research results to participants—a trend that reflects a broader cultural shift from paternalism to partnership in medicine and research, proposed Kathy Hudson, People-Centered Research Foundation. However, doing so may need to be balanced with the risk of damaging the integrity of the study, added Okun and El-Toukhy. Buchanan highlighted the importance of creating a trustworthy mechanism for returning data to participants. Hudson added that for widespread culture in the pharmaceutical industry to change, there needs to be evidence that involving participants improves outcomes and a continued expectation from the U.S. Food and Drug Administration (FDA) that clinical trials focus on patient partnership.

Policy Considerations

  Leonard Sacks, FDA, highlighted the opportunities to use mobile technologies and engage local providers to promote inclusivity and convenience for trial participants, and for gathering information on real-world patient experience. These opportunities will require policies and regulations to address patient safety, privacy, the integrity of the data produced by remote technologies, and the responsibilities of the investigators involved in technology–enabled decentralized trials. Ensuring participant safety in a decentralized trial is no different than in a traditional clinical trial, said Sacks. However, he noted that digital health technologies are creating opportunities for greater safety oversight by replacing episodic monitoring with continuous monitoring of variables, such as blood glucose levels, heart rate, and rhythm. At the same time, he cautioned, it is important to ensure that technology failure does not jeopardize participant safety or the integrity of the data and that technical support is available for when a digital health technology malfunctions.


  Leanne Madre, Clinical Trials Transformation Initiative (CTTI), described lessons learned from CTTI’s Decentralized Clinical Trials Project. CTTI has identified multiple benefits to decentralized trials, such as faster trial participant recruitment; improved retention; greater control, convenience, and comfort for participants; and increased participant diversity. To achieve these benefits, CTTI issued six categories of recommendations: (1) Engage early with FDA and those who have already conducted a decentralized trial can be important for developing the trial protocol and trial success; (2) Maintain licensed investigators in each active trial site or use investigators licensed in multiple states, given state-by-state variations in regulations governing physician licensure; (3) Use mobile health care providers to facilitate participant protocol contributions, such as blood draws or administration of the investigational product, as a decentralized trial can cover a wide geographic area; (4) Review laws governing D2P shipment of drugs as these laws can also vary from state to state; (5) Consider differences between a standard and decentralized trial when delegating responsibilities to investigators, sub-investigators, and local providers; (6) Ensure that trial participants and trial staff are aware of procedures related to adverse events.


  Deven McGraw, Ciitizen Corporation, discussed the importance of protecting participant data and privacy, as well as policy mechanisms in the United States and Europe that govern these protections. Privacy, McGraw stated, is about enabling appropriate data use with good data stewardship that engenders trust among trial participants. She added that investigators must make and keep commitments to trial participants concerning how their data will be used and disclosed, and be transparent about how data will be used. In the context of a virtual or decentralized trial, the Health Insurance Portability and Accountability Act (HIPAA) covers identifiable data collected—even from a mobile health technology—if the investigator is a HIPAA-covered entity. However, it is less clear if HIPAA covers data that reside in consumer mobile devices. McGraw mentioned that recent developments in privacy law, such as the California Consumer Protection Act and the European Union’s Global Data Protection Rule, are now requiring more explicit forms of consent for data reuse.


            Matthew McIntyre, 23andMe, discussed policy and regulatory challenges in developing informed consent processes for remote studies that involve passive data collection—data collection in which information is automatically gathered, often without the awareness of the research participant. While privacy considerations for passive data collection primarily draw on HIPAA, participants may have concerns that go beyond the risk of de-identification, such as who will have access to their data and how their data will be used. This is true for paradata—additional data collected along with passive data, such as time stamps and geolocation. As the quantity of paradata collected makes de-identification a challenge, McIntyre said that one solution is data minimization. However, this comes at the expense of limiting quality control and oversight of protocol compliance. McIntyre indicated that the incorporation of passive data in trials may require new policies for mixed uses and sources of data, in addition to dynamic ways to acquire informed consent and inform participants about data collection.


Reflections on the Workshop and Potential Future Directions

Concluding the workshop, participants reviewed key themes discussed during the workshop. Johnston emphasized the importance of using human-centered design and seeking input from participants early in the trial design process. Kelly Simcox, Sanofi, pointed out the need to have more visibility regarding current virtual clinical trials so that lessons learned are shared and D2P trials can move beyond the pilot-stage and towards mainstream clinical trial methodology. Rebecca Pentz, Emory University School of Medicine, and Hudson emphasized the importance of relaxing exclusion criteria to increase equitable participation in trials. They also highlighted the importance of community engagement, and how returning data to community members in real time can empower participants and help build trust in the trial process. John Wilbanks, Sage Bionetworks, noted the disconnect between the technology needed to support virtual clinical trials and the current policy environment. He emphasized the importance of understanding consent as an ongoing relationship—not a one-time transaction—especially when digital health technologies are passively collecting data.


Johnston emphasized the need for development of an ontology to describe the variety of studies occurring in the virtual clinical trial space. While regulatory issues were raised as a barrier to conducting virtual clinical trials, Hudson suggested that policy analysis and advocacy could facilitate implementation of reform. Craft went further to propose that the federal government could carve out an exemption to state laws for telemedicine-based clinical research. Johnston and Cummings mentioned that virtual clinical trials may require greater inclusion of participants in study design, governance, as well as drug safety monitoring boards. Resources that provide guidance for meaningful engagement and bidirectional training for a study team will also be important to engage participants meaningfully, noted Hudson.


Virtual clinical trials are not a “one-size-fits-all” model and only a fraction of clinical trials might be considered fully virtual. In the near term, Hawkins suggested that digital health technologies may only be applicable in a few settings, such as disease areas for which telemedicine is already an accepted practice or for evaluating medical products with a known safety profile and endpoints that can be measured remotely. However, in the longer term, Lipset and Dorsey highlighted that virtual clinical trials have the potential to streamline the process of drug development and may offer new opportunities for a modern, more patient-centric clinical trial enterprise.


For more information, visit

Virtual Clinical Trials: Challenges and Opportunities: Proceedings of a Workshop can be purchased and downloaded from the National Academies Press, 500 Fifth Street, NW, Washington, DC 20001;

DISCLAIMER: This Workshop Highlights was prepared by Eeshan Khandekar and Carolyn Shore as a factual summary of what occurred at the workshop. The statements made are those of the rapporteur or individual workshop participants and do not necessarily represent the views of all workshop participants; the planning committee; or the National Academies of Sciences, Engineering, and Medicine.

SPONSORS: This workshop was supported by Abbvie Inc.; American Society for Microbiology; Amgen Inc.; Association of American Medical Colleges; AstraZeneca; Burroughs Wellcome Fund; Critical Path Institute; Eli Lilly and Company; FasterCures; Foundation for the National Institutes of Health; Friends of Cancer Research; GlaxoSmithKline; Johnson & Johnson; Merck & Co.; National Institutes of Health: National Cancer Institute, National Center for Advancing Translational Sciences, National Institute of Allergy and Infectious Diseases, National Institute of Mental Health, National Institute of Neurological Disorders and Stroke, Office of Science Policy; New England Journal of Medicine; Pfizer Inc.; Sanofi; Takeda Pharmaceuticals; and U.S. Food and Drug Administration: Center for Drug Evaluation and Research, Office of the Commissioner.