Abstract
Background and objective The injury of the cholinergic white matter pathway underlies cognition decline in patients with silent cerebrovascular disease (SCD) with white matter hyperintensities (WMH) of vascular origin. However, the evaluation of the cholinergic white matter pathway is complex with poor consistency. We established an intelligent algorithm to evaluate WMH in the cholinergic pathway.
Methods Patients with SCD with WMH of vascular origin were enrolled. The Cholinergic Pathways Hyperintensities Scale (CHIPS) was used to measure cholinergic white matter pathway impairment. The intelligent algorithm used a deep learning model based on convolutional neural networks to achieve WMH segmentation and CHIPS scoring. The diagnostic value of the intelligent algorithm for moderate-to-severe cholinergic pathway injury was calculated. The correlation between the WMH in the cholinergic pathway and cognitive function was analysed.
Results A number of 464 patients with SCD were enrolled in internal training and test set. The algorithm was validated using data from an external cohort comprising 100 patients with SCD. The sensitivity, specificity and area under the curve of the intelligent algorithm to assess moderate and severe cholinergic white matter pathway injury were 91.7%, 87.3%, 0.903 (95% CI 0.861 to 0.952) and 86.5%, 81.3%, 0.868 (95% CI 0.819 to 0.921) for the internal test set and external validation set. for the. The general cognitive function, execution function and attention showed significant differences among the three groups of different CHIPS score (all p<0.05).
Discussion We have established the first intelligent algorithm to evaluate the cholinergic white matter pathway with good accuracy compared with the gold standard. It helps more easily assess the cognitive function in patients with SCD.
WHAT IS ALREADY KNOWN ON THIS TOPIC
Injury of the cholinergic pathway is one of the important causes of cognitive decline in patients with silent cerebrovascular disease (SCD). However, there is a lack of rapid and accurate methods to evaluate the cholinergic pathway.
WHAT THIS STUDY ADDS
We developed an intelligent algorithm that can segment white matter hyperintensities and evaluate cholinergic pathway injury. We validated its accuracy and applied it to assess cognitive function in patients with SCD.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
The intelligent algorithm we established helps to quickly and accurately assess cholinergic pathway injury and reduce labour costs. It provides a new method for imaging assessment of cognitive impairment in patients with SCD.
Introduction
Silent cerebrovascular disease (SCD) is an age-related disease with a high prevalence. About 25% of people over 80 years old have more than one silent cerebral infarction.1 2 SCD is one of the most common causes of cognitive impairment, and it accounts for 36%–67% of vascular dementia.3 White matter hyperintensities (WMH) is one of the imaging manifestations of SCD.4 Previous studies have shown that the progression of WMH is closely related to the decline of cognitive function.5–7
Furthermore, specific neural pathways have different effects on cognitive function. The injury of the cholinergic pathway is an important mechanism of WMH affecting cognitive function. A diffusion neuroimage study found that the integrity of the cholinergic white matter pathway is significantly reduced in all stages of Alzheimer’s disease.8 Several studies suggested that impairment of the cholinergic pathway may also be associated with cognitive dysfunction in Lewy body disease and Parkinson’s disease dementia, although these three diseases have different pathological mechanisms.9–11 In order to facilitate the quantitative evaluation of the cholinergic white matter pathway, Bocti12 et al developed a new Visual Rating Scale: The Cholinergic Pathways Hyperintensities Scale (CHIPS) to assess WMH within cholinergic pathways. Several subsequent studies13–15 have found that CHIPS are correlated to cognitive function. Qiu13 et al found that time-based prospective memory, Montreal Cognitive Assessment (MoCA) score and CHIPS score were negatively correlated in patients with WMH, suggesting that WMH may be involved in retrospective memory by affecting the central cholinergic pathway.
Therefore, quantitative analysis of WMH within cholinergic pathways is very important for evaluating cognitive function. The calculation of the CHIPS is usually time-consuming due to its highly complicated nature. Therefore, some algorithms of automatic WMH segmentation have been developed.16 Lesion Segmentation Toolbox17 from the Statistical Parametric Mapping (SPM) was a commonly used tool. On this basis, some researches have improved the algorithm for WMH segmentation. Jiang et al presented a cluster-based pipeline for extracting WMH.18 However, the results of previous studies were not consistent to be used as the basis of the scoring algorithm.19 As far as we know, there is no intelligent WMH segmentation algorithm addressing WMHs in the cholinergic pathway. Therefore, we chose deep learning based on U-net of convolutional neural networks to extract complex features. The intelligent algorithm realised WMH segmentation and CHIPS scoring. Based on this algorithm, we analysed the relationship between the WMH within the cholinergic pathway and cognitive function in patients with SCD.
Methods
Study population
This study continuously collected patients with SCD with WMH of vascular origin at the neurology department of XXX Hospital. The inclusion criteria were: (1) Aged 55 years to 85 years. (2) MRI showed WMH of vascular origin (high signal on T2-Fluid attenuated inversion recovery (FLAIR) in the white matter area). Patients with stroke, demyelination disease, brain tumours, autoimmune encephalitis, Alzheimer’s disease and other imaging manifestations of SCD: new subcortical small infarction, lacunae, severe perivascular space and cerebral microhaemorrhage were excluded (figure 1).
Flow chart of the selection process of patients. (A) Internal set, include training and test set; (B) external validation set. DICOM, Digital Imaging and Communications in Medicine; SCD, silent cerebrovascular disease; WMH, white matter hyperintensities.
Standard protocol approvals, registrations and patient consents
Cognitive measurement
Cognitive function was assessed with the following neuropsychological tests. (1) General cognitive function: Mini-Mental State Examination20 (MMSE): Included orientation, memory, attention, language and visuospatial function. The score ranges from 0 to 30, with higher scores associated with better cognitive function. (2) MoCA:21 Included executive function, memory, attention, language and visuospatial function. The score ranges from 0 to 30, with higher scores associated with better cognitive function. (3) Trail Making Test22 (TMT): Patients were asked to connect 25 numbers as requested. Used to evaluate executive function. (4) Symbol Digit Modality Test23 (SDMT): Used to assess attention. The subjects were asked to convert symbols into numbers, and the more they did the better their cognitive function. (5) Auditory Verbal Learning Test:24 Used to evaluate memory function. The more words recalled the better the cognitive function. (6) Verbal Fluency Test:25 Subjects were asked to list as many animals as possible in 1 min. Used to evaluate language function.
MRI acquisition
In training and test sets, MRI scans were performed on two MRI machines with different parameters. The first was a 3.0T scanner (United Image). The scan protocol of T2 FLAIR was as follows: repetition time was 8000 ms, echo time was 104 ms, number of slices was 19, slice thickness was 5 mm and spacing between slices was 6.5 mm. The second was a 1.5T scanner (General Electrics). The scan protocol of T2 FLAIR was as follows: repetition time was 8602 ms, echo time was 162.2 ms, number of slices was 16, slice thickness was 5 mm and slice gap was 3 mm.
In external validation sets, MRI was performed on Philips 3.0T MRI scanner. Scanning parameters of T2 FLAIR were as follows: repetition time 8000 ms, echo time was 120 ms, number of slices was 21, slice thickness was 6 mm and spacing between slices was 7 mm.
Cholinergic Pathways Hyperintensities Scale
CHIPS was a Visual Rating Scale evaluating cholinergic white matter pathways. Bocti et al 12 developed it based on published immunohistochemical tracers of human cholinergic pathways, and superimposed it on structural MRI scans. Four axial images of the adjacent layers of the lateral ventricle and the third ventricle were taken by anatomical markers, and the cholinergic medial pathway (cingulate) and lateral pathway (capsule externa and centrum semiovale) were overlapped on CHIPS. The degree of WMH in each region was assessed according to the location of 10 regions (left and right cingulate, capsule externa anterior, capsule externa anterior, capsule externa posterior, centrum semiovale anterior and centrum semiovale posterior) (figure 2). The severity of WMH lesions was classified into three grades, 0 for normal, 1 point for involving less than 1/2 of one region and 2 points for involving more than 1/2 of one region. Due to the different distribution of cholinergic fibres in each layer, each layer has a different weighting coefficient, ranging from 1 to 4. Each hemisphere scores a maximum of 50 points. A total of 100 points on both sides. CHIPS≤4 represents mild cholinergic white matter pathway injury, while 5≤CHIPS≤15 represents moderate and CHIPS≥16 represents severe.
Cholinergic Pathways Hyperintensities Scale on T2-FLAIR MRI. (A) Low external capsule; (B) high external capsule; (C) corona radiata; (D) centrum semiovale; (E) coronal view of above slices (A–D). FLAIR, Fluid attenuated inversion recovery
WMH intelligent algorithm
A deep learning model based on a convolutional neural network was used to segment and predict the WMH. With a ratio of 7:3, patients’ MRI data was randomly divided into training sets and test sets. The specific implementation steps of the algorithm are as follows. (1) Data cleaning: The collected brain image data were classified according to different categories; (2) data labelling: According to the cleaned data, the disease conditions in the brain image are labelled, so that the model can learn efficiently and accurately; (3) data preprocessing: Skull was removed using FMRIB Software Library (FSL) Brain Extraction Tool (BET)26 and skull-stripping images reoriented to the standard Montreal neurological institute (MNI) space with SPM; (4) model training: U-net27 convolutional neural network is used to learn the data after the doctor’s labelling, and a relatively mature and usable preliminary model is obtained. We used classical two-dimensional U-net architecture with 19 layers with 3×3 convolution kernels. The model is trained with Dice loss function. We implemented the networks in Python V.3.6 using Pytorch V.0.8.0. The model was trained on an NVIDIA Tesla V100, taking 2 min per epoch. We used the Adam optimiser with 10−4 learning rate. Data augmentation was applied with a torch tool box using translation (x/y-offset ∈(−10, 10)), rotation (θ ∈(−10, 10)). The total training epochs number was 300; (5) model testing: Take the learnt model to perform segmentation prediction of WMH on the test set; (6) model generalisation: Evaluate model generalisation performance using external validation sets.
The CHIPS scoring algorithm is based on CHIPS template. It is distributed in blocks in the standard space, occupying 20 areas. Each area has a different weight. WMH is matched to the standard space with the template. The specific scoring algorithm is as follows. (1) Register the individual spatial image and calculate the distribution of WMH in standard space. (2) The standard spatial template contains 20 regions, each spanning several layers. The volume of WMH in each layer of each region is calculated, and the region with the highest proportion is selected as the result of this region. (3) Judge the results obtained in each area. If it is less than threshold A, it is counted as 0 points; if it is greater than threshold B and less than threshold C, it is counted as 1 point; if it is greater than threshold C, it is counted as 2 points. By learning the data scored by the doctors, set threshold A: 0.003, threshold B: 0.03 and threshold C:0.25. The specific parameters of A, B and C were determined through the grid search optimisation method. The goal of optimisation is to maximise the accuracy of the final scoring. (4) Calculate the product of each zone’s fraction and zone coefficient, and sum (figure 3).
The pipeline of Intelligent algorithm. (A) Input T2-FLAIR image; (B) image preprocessing: remove skull and reoriented to standard MNI space; (C) model training: using classical 2D U-net architecture with 19 layers with 3×3 convolution kernels; (D) the result of WMH segmentation; (E) CHIPS templates in standard space. Divided into low external capsule, high external capsule, corona radiata and centrum semiovale. (F) Match the segmented WMH back to the standard space, and combining the CHIPS template. The grid search optimisation method was used to get the CHIPS score. CHIPS, Cholinergic Pathways Hyperintensities Scale. FLAIR, Fluid attenuated inversion recovery. MNI, Montreal neurological institute. WMH, white matter hyperintensity; 2D, two-dimensional.
The intelligent algorithm can be acquired from Google Drive (https://drive.google.com/file/d/1oz-DQssQPY_axlAB8tqN71sds67UC7pZ/view?usp=share_link).
WMH manual assessment
Two neurologists calculated CHIPS scores and performed manual segmentation of WMH on the FLAIR images using 3D Slicer (V.4.8). If there was a disagreement between the two neurologists, it was resolved by including the third neurologist. The segmentation of WMH and a score of CHIPS by these three doctors were regarded as the gold standard. Neurologists were blind to the segmentation of each other and intelligent algorithms. Meanwhile, in order to verify the instability of manual CHIPS scores, we invited seven additional neurology residents to calculate CHIPS scores.
Statistical analysis
The normality was tested with the Shapiro-Wilk test. Continuous data with a normal distribution are expressed as the mean±SD. Data with non-normal distribution are presented as medians with IQRs. The correlation between CHIPS scores measured by doctors and intelligent algorithm was described with Spearman’s correlation coefficient and evaluated by Bland-Altman analysis. Sensitivity, specificity and the area under the receiver operating characteristic curve (AUC) were used to evaluate the diagnostic accuracy. The comparisons among three groups were done by one-way analysis of variance then followed by post hoc Bonferroni test. Independent sample t-tests were used in the analysis between two groups. Statistical analyses were performed with SPSS software, V.19.0 (IBM Corporation, Armonk, New York, USA). P value<0.05 was considered statistically significant.
Data availability
We take full responsibility for the data, the analyses and interpretation and the conduct of the research. We have full access to all of the data. We have the right to publish any and all data, separate and apart from the guidance of any sponsor.
Result
Demographic and clinical characteristics
For training and test sets, a total of 626 patients with SCD were enrolled in this study from 1 January 2018 to October 2019. Among them, 111 patients were excluded due to accompanying diseases and 51 patients were excluded due to the unaccomplished cognitive test. Finally, 464 patients were included in the statistical analysis. With a ratio of 7:3, patients’ MRI data was randomly divided into training sets and test sets. Training sets contained 325 patients, while test sets contained 139 patients. A total of 157 patients with SCD were enrolled for the external validation set. According to the including and exclusion criteria, 100 patients were included in the statistical analysis. The demographic characteristics and risk factors of small cerebral vascular disease of training, test and external sets are shown in table 1.
Clinical characteristics of patients
The establishment and verification of intelligent algorithm
Due to the complex calculations of CHIPS, the differences between evaluators were always large. We invited seven resident neurologists to score the CHIPS. The Kendall’s W coefficient of the CHIPS score of 7 residents was 0.555 (p<0.001), indicating poor consistency. Hence, it is necessary to establish the intelligent algorithm to reduce the differences of manual evaluation.
Intelligent CHIPS scores were significantly correlated with CHIPS scores measured by senior neurologists in both training set (r=0.941, 95% CI 0.933 to 0.958, p<0.0001) and test set (r=0.910, 95% CI 0.880 to 0.932, p<0.0001), as well as in external validation set (r=0.846, 95% CI 0.785 to 0.901, p<0.0001). In the training set, Bland-Altman analysis revealed a bias of −3.527 (95% CI −13.06 to 20.41) for the CHIPS score of intelligent algorithms and senior neurologists, while in the test set, bias was 4.631 (95% CI −15.56 to 25.58). The bias of the external validation set was −4.170 (95% CI −12.17 to 20.51) (figure 4).
Linear regression analysis and Bland-Altman analysis of CHIPS score measured by a doctor and intelligent algorithm. (A) Training set, (B) test set, (C) external validation set. CHIPS, Cholinergic Pathways Hyperintensities Scale.
In the test set, the sensitivity and specificity of the intelligent algorithm were 91.7%, 87.3%, AUC=0.903, (95% CI 0.861 to 0.952) for the assessment of moderate and severe cholinergic white matter pathway injury. While in the external validation set, the sensitivity and specificity of the intelligent algorithm were 86.5%, 81.3%, AUC=0.868 (95% CI 0.819 to 0.921). The confusion matrix is present in figure 5.
Confusion matrix of intelligent algorithm detecting moderate and severe cholinergic white matter pathway injury. (A) Training set, (B) test set, (C) external validation set.
The intelligent algorithms also showed stability in magnetic resonance with different parameters. Intelligent CHIPS score had a good correlation with manual CHIPS score on both 3.0T MR (r=0.945, p<0.0001) and 1.5T MR (r=0.896, p<0.0001). The intelligent algorithms took 4.33±2.06 s to complete CHIPS score of a patient, while the clinician took 123.50±88.68 s. Intelligent algorithms significantly reduced time (p<0.0001).
Association between cholinergic white matter pathways and cognitive functions
The patients of training and test sets were categorised into three groups based on the CHIPS score of intelligent algorithms, mild (group 1), moderate (group 2) and severe (group 3). The comparison of cognitive function among three groups is shown in table 2. There was no significant difference in age or gender among groups (both p>0.05) MMSE (p=0.019), MoCA (p=0.015), TMT (p=0.029) and SDMT (p=0.016) had significant differences among the three groups of different CHIPS scores. The scores of intelligent CHIPS were negatively correlated with MMSE, MOCA and SDMT scores (r=−0.330, p<0.0001; r=−0.309, p<0.0001, r=−0.290, p<0.0001) (figure 6). The correlation between cognitive function and CHIPS of different regions was shown in online supplemental figure 1. MMSE and MoCA were negatively correlated with WMH in cingulate and posterior of capsule externa. SDMT was negatively correlated with WMH in the cingulate, posterior of capsule externa and centrum semiovale, while TMT was correlated with WMH in cingulate.
Supplementary data
Linear regression analysis of the cognitive scale and CHIPS score measured by an intelligent algorithm. (A) MMSE, (B) MoCA, (C) SDMT. CHIPS, Cholinergic Pathways Hyperintensities Scale; MMSE, Mini-Mental State Examination; MoCA, Montreal Cognitive Assessment; SDMT, Symbol Digit Modality Test.
Comparison of cognitive function among different groups of CHIPS score
For the external validation set, the scores of intelligent CHIPS also show a significant correlation with MMSE, and MOCA (r=−0.278, p=0.013; r=−0.341, p=0.008).
Discussion
To the best of our knowledge, it is the first study that establishes the intelligent algorithm specific to the evaluation of the cholinergic white matter pathway. We verified that there is good agreement between the intelligent algorithm and senior neurologists. The algorithm accurately and quickly completed WMH segmentation and CHIPS scoring. Meanwhile, the intelligent algorithm can be adapted to different magnetic resonance machines with different scanning parameters. Using this algorithm, we investigated the correlation between cholinergic pathway damage and cognitive function, especially attention and executive function, in patients with SCD.
SCD leads to insidious cognitive decline. As an imaging manifestation of SCD, WMH has been proven to be associated with cognitive function by many studies. Furthermore, the quantitative evaluation of these imaging manifestation has been paid more and more attention, and precise imaging helps to understand the mechanism of disease.28 29 WMH in special neural pathways can lead to cognitive decline. Cholinergic pathways are thought to be important in cognitive function.30 CHIPS is considered to be a good method for assessing the injury of the cholinergic pathway, but the evaluation is complex and time consuming. We established an intelligent CHIPS scoring algorithms, and it had good consistency with manual scoring. Moreover, it significantly reduced the evaluating time and variance in manual assessments, and showed stability in magnetic resonance with different parameters. Several previous studies used different kinds of tools, such as statistical parametric mapping (SPM),31 k-Nearest Neighbor32 and neural networks,33 to predict the segmentation results of WMH and brain region division. Our algorithm is based on U-net architecture. In order for the detection of both large lesions and small lesions, we ensembled multiple model predictions. We replaced the convolutional kernels in the initial two layers with those of varying sizes to extract features more efficiently from different receptive domains.
Our study had more detailed conclusions that intelligent CHIPS were correlated with scores of MMSE, MoCA, TMT and SDMT. We suggested that WMH in the cholinergic pathway mainly correlated with attention and executive function. Some previous studies have found a correlation between CHIPS and cognitive function.34 35 Nemy36 et al found that the integrity of the cholinergic pathway has a greater effect on memory and attention than the WMH load and volume of white matter in the patient with small vessel disease. Swartz37 et al evaluated the impact of WMH involving cholinergic pathways in patients with vascular and mixed dementia. Executive function and visual-spatial attention were more severely impaired in patients with moderate and severe impairment than in patients with mild impairment of the cholinergic pathway, although overall cognition and memory impairment were comparable. The correlation between cognitive function and WMH in the cholinergic pathway was shown not only in vascular disease, but also in other disease cause dementia, such as Alzheimer’s disease,14 Parkinson’s disease38 and diffuse Lewy body disease.9
Intelligent algorithms can finely delineate brain regions in cholinergic pathway. We found that the cingulate gyrus of the cholinergic pathway of closely related to cognitive function, including general cognitive function, attention and executive function. We inferred that the cingulate gyrus might be an important region of cognition. Some previous studies had similar evidence. In a community cohort of 819 people over the age of 65, WMH volume in the cingulate gyrus was found to be significantly correlated with Modified Boston Naming Test performance.39 Wertz et al 40 evaluated the work in science, technology, engineering and mathematics of 178 healthy volunteers (aged 16–32 years old) and found that white matter integrity of the cingulate gyrus was associated with creative cognition.
Previous studies have suggested a possible mechanism that injury of the cholinergic pathway affected cognitive function. An acetylcholinesterase positron emission tomography (PET) study41 assessed cortical acetylcholinesterase activity in patients with age-associated WMH. They found that periventricular WMH was significantly associated with lower cortical acetylcholinesterase activity, which supported the disruption of cholinergic fibres by WMH. Lim42 et al included 80 patients with a mean age of 78.4±6.5 years and a median MMSE score of 17. It was found that the CHIPS score was positively correlated with putamen and globus pallidus volume. WMH in the cholinergic pathway was negatively correlated with the thickness of the temporal cortex and orbitofrontal cortex. WMH in the cholinergic pathway may lead to the volume and structure changes of cortical and subcortical structures inactivated by cholinergic neurons.
CHIPS score can reflect the relationship between WMH and cognitive function in patients with SCD, but due to complicated calculation, it usually takes much time and varies greatly among evaluators. The application of intelligent CHIPS algorithm can greatly reduce labour costs and promote the application of CHIPS in clinical practice.
Data availability statement
Data are available upon reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
The study was approved by the Ethics Committee of Zhongshan Hospital affiliated to Fudan University and in accordance with the Declaration of Helsinki (Approval No: B2018-155). All enrolled patients signed the informed consents.
Footnotes
Contributors BF enrolled the patients, analysed the data and did the writing. YC built the algorithm and analysed the data. YL enrolled the patients. GZ enrolled the patients. AG enrolled the patients. JL helped build the algorithm. SW enrolled the patients. HW designed the study and guided in writing. JD designed the study and guided in writing. Besides, JD was guarantor that accepted full responsibility for the work and the conduct of the study, had access to the data, and controlled the decision to publish. XW guided in study design. All authors gave final approval of the manuscript for publication.
Funding The study was funded by Shanghai Municipal Committee of Science and Technology (Code 20Z11900800).
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
References
