Abstract
Background Given the swift advancements in artificial intelligence (AI), the utilisation of AI-based clinical decision support systems (AI-CDSSs) has become increasingly prevalent in the medical domain, particularly in the management of cerebrovascular disease.
Aims To describe the design, rationale and methods of a cluster-randomised multifaceted intervention trial aimed at investigating the effect of cerebrovascular disease AI-CDSS on the clinical outcomes of patients who had a stroke and on stroke care quality.
Design The GOLDEN BRIDGE II trial is a multicentre, open-label, cluster-randomised multifaceted intervention study. A total of 80 hospitals in China were randomly assigned to the AI-CDSS intervention group or the control group. For eligible participants with acute ischaemic stroke in the AI-CDSS intervention group, cerebrovascular disease AI-CDSS will provide AI-assisted imaging analysis, auxiliary stroke aetiology and pathogenesis analysis, and guideline-based treatment recommendations. In the control group, patients will receive the usual care. The primary outcome is the occurrence of new vascular events (composite of ischaemic stroke, haemorrhagic stroke, myocardial infarction or vascular death) at 3 months after stroke onset. The sample size was estimated to be 21 689 with a 26% relative reduction in the incidence of new composite vascular events at 3 months by using multiple quality-improving interventions provided by AI-CDSS. All analyses will be performed according to the intention-to-treat principle and accounted for clustering using generalised estimating equations.
Conclusions Once the effectiveness is verified, the cerebrovascular disease AI-CDSS could improve stroke care and outcomes in China.
Trial registration number NCT04524624.
WHAT IS ALREADY KNOWN ON THIS TOPIC
Artificial intelligence-based clinical decision support systems (AI-CDSS) have been developed and deployed in stroke medical care. However, the efficacy of the tool in the real world was seldom reported.
WHAT THIS STUDY ADDS
We designed the GOLDEN BRIDGE II trial to investigate the effects of the cerebrovascular disease AI-CDSS on stroke outcomes and the quality of stroke care.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
The GOLDEN BRIDGE II trial will produce objective data on whether cerebrovascular disease AI-CDSS improves the quality of stroke care and outcomes among patients who had an ischaemic stroke.
Introduction
In China, the incidence of stroke has increased over the past three decades, becoming the primary cause of death.1 2 Advance prevention strategies and treatment tools for patients who had a stroke, which will ultimately enhance stroke care and improve outcomes, are in high demand.3 4
Artificial intelligence (AI) technology has been widely used in the medical field. The AI-based clinical decision support system (AI-CDSS) is a novel practice and application of AI technology in cerebrovascular diseases.5–7 AI-CDSS is a computer system that can help physicians make clinical decisions and diagnose diseases by matching the clinical information of patients with the disease database, which provides helpful knowledge that can be used to optimise the diagnosis and treatment according to evidence-based guidelines, and thereby improve care quality and patients’ prognoses.8 9 Therefore, it is of great importance to developing AI-CDSS systems in aiding the decision-making process for clinical diagnosis and treatment, which will ultimately help to achieve the goal of precision medicine.
In our study, we developed the cerebrovascular disease AI-CDSS to provide clinical guideline-based interventions, including AI-assisted imaging analysis, auxiliary analysis of aetiology and pathogenesis, and guideline-based treatment recommendations for patients who had an acute ischaemic stroke (AIS). We designed a clustered-randomised controlled study to investigate the effects of the AI-CDSS (the cerebrovascular disease AI-CDSS) on the patient’s early recurrence of composite vascular events and on stroke care quality.
Methods
Study design
The GOLDEN BRIDGE II trial is a multicentre, open-labelled, cluster-randomised multifaceted intervention study designed to evaluate the effects of cerebrovascular disease AI-CDSS on new composite vascular events (including ischaemic stroke, haemorrhagic stroke, myocardial infarction or vascular death) at 3 months in patients who had an AIS. The cerebrovascular disease AI-CDSS will provide AI-assisted imaging analysis, auxiliary stroke aetiology and pathogenesis analysis, and guideline-based treatment recommendations for patients in the intervention group. On the other hand, patients in the control group will receive usual care. All patients will be enrolled sequentially and receive follow-ups at 3, 6 and 12 months after stroke onset. The flow chart of the study is shown in figure 1. The study was registered on ClinicalTrials.gov (NCT: 04524624).
The flow chart of this study. AI-CDSS, artificial intelligence-based clinical decision support system.
Participants
To ensure diversity, the geographical location and grade of hospitals were considered. The included hospitals cover the eastern, central and western regions of China, with proportional recruitment of secondary and tertiary hospitals. A total of 80 hospitals located across 23 provinces, autonomous regions or municipalities in mainland China were included. The criteria for hospital inclusion and exclusion are shown in table 1.
The inclusion and exclusion criteria for the hospitals
This trial is specifically designed for patients over the age of 18 with AIS confirmed by imaging of the brain within 7 days of symptom onset, and excludes those with the diagnosis of transient ischaemic attack (TIA), haemorrhagic stroke or non-cerebrovascular diseases. The inclusion and exclusion criteria for the participants are shown in table 2.
The inclusion and exclusion criteria for the participants
Intervention
The clustered randomisation will be completed before each hospital is enrolled in the study, and the eligible participants with AIS in the intervention group will receive the AI-based analysis and the recommended treatment provided by the cerebrovascular disease AI-CDSS. The specific AI-CDSS intervention protocol includes AI-assisted imaging analysis, auxiliary analysis of ischaemic stroke aetiology and pathogenesis, and guideline-based treatment recommendations for acute and secondary prevention.
The AI-CDSS obtains patients’ clinical data and imaging data from the hospital information system and uses the method of stroke image automatic interpretation technology based on deep learning combined with a stroke clinical knowledge base to provide assistant clinical decisions including imaging diagnosis, aetiological classification and treatment decision of ischaemic stroke (figure 2 and online supplemental material).
Supplementary data
The framework for cerebrovascular disease clinical decision support system (CDSS). EMR, Electronic Medical Record; HIS, Hospital Information System; LIS, Laboratory Information Management System; PACS, Picture Archiving and Communication System; TOAST, Trial of Org 10172 in Acute Stroke Treatment; CISS, Chinese Ischemic Stroke Subclassification.
The neurologists in the intervention hospitals will complete training on cerebrovascular disease AI-CDSS before the start of this study. A 2-week transition phase of AI-CDSS intensive intervention will be completed before the patients are enrolled. All participants in the hospital of the control group received normal routine diagnosis and treatment (figure 3).
Intervention flow chart. CDSS, clinical decision support system.
Randomisation
The computer-generated random number sequence will be used for cluster randomisation at the hospital level, and two independent biostatisticians (YJ and H-QG) will be aware of it. Participating hospitals are randomly grouped on a 1:1 basis. To prevent imbalance, hospital location and grade are matched during randomisation. A confirmation letter will be provided to each hospital 1 month before the intervention’s implementation to assure readiness. If the chosen hospital could not complete the study, it was replaced with another one of equivalent capacity and economic–geographic region strata.
Primary outcome
The primary outcome is a new composite vascular event (including ischaemic stroke, haemorrhagic stroke, myocardial infarction or vascular death) at 3 months after stroke onset.
Secondary outcomes
The secondary outcomes include: (1) the composite measure score of medical service indicators for AIS care quality, (2) a new composite vascular event at 6 and 12 months after stroke onset, (3) disability based on the modified Rankin Scale (mRS=3–6) at 3, 6 and 12 months after stroke onset and (4) average hospitalisation days, average total hospitalisation cost, average hospitalisation drug cost and average hospitalisation examination cost.
Safety outcomes
The primary safety outcome refers to the occurrence of moderate or severe bleeding events as defined by the Global Utilisation of Streptokinase and Tissue Plasminogen Activator for Occluded Coronary Arteries (GUSTO) within 3 months after stroke onset. The secondary safety outcomes include severe or moderate bleeding (as defined by GUSTO) within 6 and 12 months. In addition, all bleeding events (including severe or moderate bleeding) within 3, 6 and 12 months are included.
Study organisation
The International Academic Steering Committee is comprised of the cochief investigators and members. It provides the scientific basis and strategic guidance for the trial, overseeing research design, execution and publication-related activities. Additionally, the Academic Steering Committee chairs meetings of the Senior Management Group. The Executive Committee, comprised of experienced experts guided by the International Academic Steering Committee, aims to monitor the trial’s progress, ensure participants’ safety and provide scientifically grounded decisions on whether to terminate, modify or continue the trial. The Data Safety Monitoring Board regularly monitors the progress of the study to ensure adherence to the highest ethical and patient safety standards. Furthermore, it provides written recommendations to the Chair of the Trial Steering Committee following each meeting. The Quality Improvement Committee, consisting of clinical specialists and experts in stroke quality improvement, is responsible for supervising adherence to stroke care quality standards.
Data collection
The executive committee has approved the case report form.10 11 Baseline data include demographic information, medical history, medication history, prestroke mRS Score, time of stroke onset, baseline National Institutes of Health Stroke Scale (NIHSS) Score and imaging data. Trained clinical research coordinators at each site are responsible for collecting this information on admission. Subsequently, data collected at discharge including final aetiological classification, in-hospital treatments, in-hospital complications, NIHSS Score at discharge, mRS score at discharge, medications for secondary prevention at discharge, hospitalisation costs and other data. A face-to-face interview is conducted at 3 months after stroke onset, followed by concentrated telephone follow-ups at 6 and 12 months after stroke onset. The data collected at 3, 6 and 12 months include current mRS Score, medication adherence, stroke recurrence, all-cause mortality and other clinical events. All interviewers will be oblivious to the patients’ cluster assignment.
Sample size
The main null hypothesis of this study was that, compared with those who receive traditional routine diagnosis and treatment, there would be no difference in the incidence of new vascular events observed at 3 months among patients who had a stroke treated with interventions recommended by the cerebrovascular disease AI-CDSS.
According to the findings of the third China National stroke Registry study, patients who had an AIS or TIA had a 3-month new composite vascular event rate of 6.4%.10 We assumed that the rate of 3-month new composite vascular events would be reduced to 4.7% (relative decrease of 26%) by using multiple quality-improving interventions with AI-CDSS, which decrease rate was made based on data from the GOLDEN BRIDGE trial.3 We initiated the trial with a target sample size of 21 689 participants from 80 hospitals with a 2-sided significance level of 5%, with 80% power to see a reduction of 26% in new composite vascular events, 10% of people not being followed up with and an intraclass correlation coefficient of 0.01.
Statistical analyses
In this study, efficacy analysis will be performed using intention-to-treat analysis.12 Categorical variables are typically presented in the form of counts and percentages, whereas continuous variables are commonly displayed as means and SD or alternatively as medians accompanied by IQRs. For continuous variables, t-tests or Wilcoxon Rank Sum tests will be used, and for categorical variables, the χ2 t-test or Fisher’s exact test will be used. The primary outcome will be tested using a mixed effects logistic regression with a random effect or generalized estimating equation for the cluster (hospital). The methodology employed for the primary outcome will be used for the analysis of the categorical secondary outcomes. Mixed effects linear regression model incorporating a random effect or generalized estimating equation for the cluster (hospital) will be employed to analyse continuous secondary outcomes. All statistical analyses will be performed by using SAS V.9.4 software.
Discussion
The AI-CDSS has experienced several stages of development and has the ultimate goal of assisting with actual clinical care scenarios.13 The AI-CDSS has been widely used in the research of cerebrovascular diseases, including the early identification of patients at high risk, decision-making regarding acute thrombolysis, automatic classification of disease aetiology and decision-making regarding secondary prevention.14–20
Nagenthiraja et al developed the MRI-based automated software tool “Computer-Based Decision-Support System for Thrombolysis in Stroke” (COMBAT Stroke) to assess volumes and ratios of mismatches between perfusion weighted imaging (PWI) and diffusion weighted imaging (DWI) and validated decisions made by COMBAT Stroke by referencing the decisions in the clinical setting.21 Additionally, the COMPuterised decision Aid for Stroke thrombolysiS (COMPASS) decision aid tool developed by Flynn et al can assist clinicians in making specific clinical decisions about thrombolytic therapy, with a numerical and graphical presentation of the results of risk prediction.22
A portable decision system for stroke classification, named iTOAST, was created by Nam et al in 2012.23 This system can provide classification results based on the answers to only six questions. Compared with the classification results from the stroke experts, the kappa coefficient of the iTOAST system was 0.79, which is more accurate and convenient than that in the traditional TOAST. However, the systems described above are still highly dependent on the expertise of the evaluators. Moreover, AIS patients were categorised using the technology of machine learning and natural language processing.24 The kappa score of the machine classification using the combined data of radioactivity reports and disease course records was 0.57, which is quite close to the accuracy of human classification.25 Evidence from these studies suggests that using AI to aid in ischaemic stroke aetiology classification is doable. To date, no research efforts have been made to develop an automatic system of accurate and efficient classification for ischaemic stroke aetiology by comprehensively analysing neuroimaging and clinical information based on AI technology.
The Self-management TO Prevent (STOP) Stroke Tool is an AI-CDSS tool developed by using an integrated model. The goal of this system is to provide health management for veterans, promote evidence-based management by automatically providing clinicians with guidelines for the secondary prevention of stroke based on electronic medical records, develop a web-based system for stroke self-management and facilitate patient-centred decision-making.26 27 The potential of AI-CDSS for secondary stroke prevention will soon become clear. The AI-CDSS has also been shown to drastically enhance the quality of healthcare according to a systematic review.13
The application of AI-CDSS in clinical practice still faces multiple challenges, such as (1) how to develop an advanced strategy for the integration of electronic medical records to achieve automatic acquisition and sharing of clinical information; (2) how to construct a standardised clinical knowledge base to provide evidence-based guidelines; and (3) how to combine the output information of AI-CDSS with clinical practice in diagnosis and treatment to establish an effective human–computer interaction.
Although many challenges remain, AI-CDSS has initiated a predictable trend in the development of stroke management in clinical practice. Prior studies of performance improvement systems/quality feedback such as Get With The Guidelines-Stroke (GWTG-Stroke) and the GOLDEN BRIDGE-AIS have demonstrated improved ischaemic stroke care and outcomes. The incidence of new vascular events over a 3-month period was reduced by 26% thanks to a multifaceted quality improvement intervention.3 This present trial may provide insights into further enhancements that can be achieved with AI-CDSS. Our study employed the cerebrovascular disease AI-CDSS to enhance the quality of stroke care and the clinicians’ ability to diagnose and treat ischaemic stroke. If the efficacy is well established, the cerebrovascular disease AI-CDSS could become an important and effective tool for the management and quality improvement of stroke care in China. It could help improve outcomes in the absence of stroke experts by providing individualised recommendations to generalists caring for patients who had a stroke.
Data availability statement
Data are available upon reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants. The protocol of this study was approved by the central institutional review board at Beijing Tiantan Hospital, Capital Medical University and participating hospitals (KY 2020-016-02). Participants gave informed consent to participate in the study before taking part.
Acknowledgments
We thank all investigators and participating hospitals of the GOLDEN BRIDGE II trial.
Footnotes
X @hqgu, @yilong, @braindoc_mgh
ZL, XZ, LD and JJ contributed equally.
Contributors XZhang, LD and JJ analysed and interpreted the data and drafted the manuscript. CW, CD, MW, MX, YZ and MH assisted to promote the project progress. HG and YJ completed the statistical work. XM, XG, KD, XZhao, YilongW, LL, HL, YX, EP, GCF, LHS, ZL and YongjunW conceived and designed the research.
Funding This trial was funded by the National Key R&D Programme of China (2022YFC2504902), the National Natural Science Foundation of China (92046016), Beijing Municipal Administration of Hospitals’ Mission Plan (SML20150502), Ministry of Industry and Information Technology of the People’s Republic of China(2020-0103-3-1), CAMS Innovation Fund for Medical Sciences (2019-I2M-5-029), and the Beijing Ande Yizhi Technology Co., Ltd.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
References
