KEYWORDS
Dementia, cognitive impairment, research with vulnerable populations, clinical trials
INTRODUCTION
Research with dementia patients brings about some unique challenges that may hamper the generalisability of the study findings. In research with elderly patients age,1 comorbidities and sensory impairment are often used exclusion criteria.2,3 Regarding multimorbidity, it is estimated that 55-98% of patients older than 65 have two or more chronic conditions.4 Most people suffering from dementia are over 70 years of age. Thus, a substantial proportion of patients with dementia are excluded from study participation. As a consequence, the participants in dementia research may not represent the general dementia population. If the validity and generalisability of the findings from biomedical research are weak, patients cannot benefit from the findings of these studies.
The population of patients with dementia is challenging: the group is heterogeneous with regards to the type of dementia, severity of disease and presence of comorbidities. It also means that adjusted research methods, such as subgroup analysis, are required and extrapolation of the findings remains uncertain.2
The aim of this study is to analyse study and population characteristics of dementia research protocols in the Netherlands between 2006 and 2015. Particularly, we analysed eligibility criteria in order to get a clearer picture as to whether the general population and the research population are concordant.
MATERIALS AND METHODS
Search strategy
We searched for dementia research protocols on ToetsingOnline.nl, the Dutch online assessment portal of the Central Committee on Research Involving Human Subjects and of the accredited Medical Research Ethics Committees. The ToetsingOnline database contains all biomedical studies conducted in the Netherlands that are reviewed by a Research Ethics Board. The trial data are self-reported by trial sponsors or investigators. Each record contains a set of data elements describing the study’s purpose, design, eligibility criteria, location, sponsor and other protocol information, although not all fields are mandatory and publicly accessible. In March 2015, we searched for all approved protocols regarding dementia between 2006-2015 including the term dementia, cognitive decline, Alzheimer’s disease, Parkinson’s dementia, Lewy body dementia, familial dementia, frontotemporal dementia, and vascular dementia.
Data extraction
Data extraction was conducted by using a data extraction form. This form was developed in order to standardise data extraction, on the basis of a pilot assessment of a random selection of 30 protocols by the two investigators together (KJ and RB). All remaining protocols were scored by two researchers (KJ and RB) independent of each other. Disagreements that arose were solved by discussing the protocol together. The main outcome measures were: type of dementia (Alzheimer’s disease, vascular dementia, Parkinson’s dementia, familial dementia, frontotemporal dementia, Lewy body dementia, mild cognitive impairment (MCI)), type of study (observational, drug intervention, other interventions), expected number of participants, and their age, comorbidities (somatic, psychiatric, neurological), competence, drug use and living situation.
Data synthesis
The eligibility criteria used in the study protocols were grouped into themes. For classifying data, we made the following choices: we included protocols that also studied MCI in the group of dementia protocols. In most cases, there was no distinction made between MCI and mild dementia, therefore we assumed that these protocols included a combination of MCI patients as well as patients with dementia. Furthermore, we considered use of medication an exclusion criterion if any medication use was mentioned as an exclusion criterion in the protocol. The same way of reasoning was used for any sensory impairment, any somatic, any neurological and any psychiatric comorbidity. We did not score substance abuse (34 in total) as a psychiatric exclusion criterion, Alzheimer in Down syndrome patients is scored as Alzheimer research (2 studies in total), the living environment criterion was divided into dependent (institutionalised patients, being taken care of by care professionals 24/7) and independent (either living at home or at an assisted-living facility, care by a proxy and under supervision of a GP).
Descriptive analysis
We excluded studies that investigated interventions for proxies, studies not primarily focused on dementia, or prolongations of an earlier study, because the eligibility criteria were not described in the prolongation protocol.
In the analysis we focused on the description of the type of studies and on the eligibility criteria for participants. Descriptive statistics were used to describe the study and participant characteristics. Categorical variables were reported as proportions and continuous variables as ranges or absolute numbers. Due to the descriptive nature of the study, formal statistical comparisons were not made.
RESULTS
Search results
The combination of search terms yielded 150 protocols. The duplicates were removed and 135 distinctive research protocols remained. From these 135 protocols, 20 studies were excluded. Thus, 115 studies remained of which 12 were drug studies with healthy volunteers and 103 with dementia or MCI patients (figure 1).
Characteristics of the protocols
Dementia protocols
The total number of participants between 2006 and 2015 (excluding healthy volunteers) was n = 26,422, ranging from 12 to 2400. In comparison, in the year 2014 alone, 427,500 research participants were included in any study to any disease in the Netherlands.5 A substantial proportion of the dementia protocols (36%) concerned relatively small studies, enrolling 100 subjects or less. Almost half the studies are mono-centre studies, a third of the studies included participants from at least one country outside of the Netherlands (table 1). Of the 103 studies with dementia patients, 30 were drug trials, 29 other intervention studies and 44 observational studies. In total 35% of the studies were financed by the industry (table 1). Of the studies sponsored by the industry, 62% concerned drug-intervention studies.
Healthy volunteers
The studies with healthy volunteers were 11 drug-intervention studies and one observational study. The number of participants ranged from 4 to 74, with a total of 422 participants. All of these studies but one were financed by the industry and focused on Alzheimer’s disease. These protocols with healthy volunteers are not further described or analysed in this paper.
Number and type of studies over the years
A notable trend is that the total number of research protocols seems to be increasing, in total 11 protocols were reviewed in 2006-2007, compared with 34 in 2014-2015; especially drug trial research has grown tremendously over the past years (table 1 and figure 2). The industry has initiated more research trials in the last few years: 18 trials in 2014-2015 compared with one in 2006-2007. In all publication years, Alzheimer’s disease was the most researched specified type of dementia.
Type of dementia
What is remarkable is that a substantial proportion of studies (32%) do not specify the type of dementia studied (in figures and tables labelled as all dementias), while the types of dementia vary tremendously in terms of severity, symptoms and needed care. MCI/prodromal dementia composes 12% of the studies; 9% studied two or more types of dementia, all of these included Alzheimer’s disease. Of the studies focusing specifically on one type of dementia, Alzheimer’s disease is the type of dementia most often studied in terms of number of trials (84%) and in expected number of participants (13,011).
By contrast, only a small number of studies focused on vascular and Lewy body dementia; familial dementia was not studied in any of the protocols (figures 3 and 4). Most drug trials and observational studies concerned Alzheimer’s disease, and most non-drug interventions were aimed at an unspecified group of dementia patients (table 1). 
Eligibility criteria used in the Dementia study protocols
Age
Regarding age, we found that 60% of the studies use age as an eligibility criterion, either an upper limit alone (7 protocols), a lower limit alone (28 protocols) or an age range (27 protocols). The range of the upper limit is 60-100 years, with an average of 83.7 years. The lower age limit ranged from 18-65 with an average of 49.1 years.
Competence
Of the studies, 24% noted competence of the research participant in their inclusion criteria, while 49% demanded the consent of the patient (implying participant’s competence). The consent of a proxy was required in 24% of the protocols, 23% asked for both proxy and informed consent. In approximately half of the studies (51%), having a proxy was required to be included in the research, even if their consent was not necessary.
Living situation
Dementia patients living in nursing homes were explicitly excluded from 22% of the studies. Only a small proportion of the studies (13%) focused explicitly on patients living in nursing homes due to dementia. All other studies either recruited people living independently or did not mention the living situation as an eligibility criterion. Due to other recruitment demands, patients living in nursing homes were nevertheless excluded from these studies. For instance, in 23 studies cognitive screening tools were used with scores implying mild or moderate dementia.
Dementia-screening instrument
A dementia-screening instrument, such as the Mini Mental State Examination (MMSE) and Clinical Dementia Rating (CDR) was used in 62% of the protocols. MMSE is most often used, and there is a large variety in the range set for eligibility, ranging between 10-30. Some studies set no lower limit at all for the MMSE score, but these studies required that the patient should live independently, thereby implicating a MMSE score of at least 10. Most studies consider patients with a MMSE score of 10 or less as severely demented.6
Dementia severity
Severely demented patients were excluded from most protocols: 16 protocols focused on MCI or mild dementia and 52 protocols excluded patients with a CDR score > 2 or MMSE < 10. Of the remaining 35 protocols, five required competence of the research participant and seven required that the participant was living independently, which are unlikely conditions for severely demented patients. In the remaining 23 protocols (22%), severely demented patients may be enrolled unless they have non-eligible comorbidities.
Comorbidity and medication use
Concerning comorbidities, 22% of the studies noted a visual or hearing impairment as an exclusion criterion. Medication use was stated as an exclusion criterion in 38% of the protocols. In 54% of the studies, patients with a psychiatric disorder were excluded. Somatic comorbidities were indicated as an exclusion criterion in 54% studies; 56% excluded patients with neurological conditions. In 9% of the protocols all these five exclusion criteria were noted and 14% noted none of these exclusion criteria. The most often mentioned exclusion criterion in observational studies is a neurological condition other than dementia, while in both drug studies and other intervention studies the most often used exclusion criterion is a somatic condition.
Ambiguous criteria
A remarkable finding is that 15% of the dementia studies explicitly state very ambiguous exclusion criteria, such as ‘Any other condition that in the opinion of the investigator would complicate or compromise the study’, or ‘investigator’s uncertainty about willingness, ability, or medical status of the patient to comply with protocol requirements’ which leaves much room for interpretation by the researcher without the further intervention of a Research Ethics Board. Most of these studies were drug trials initiated by the industry.
DISCUSSION
This analysis provides a first snapshot of the landscape of dementia research and of dementia research participants as listed on ToetsingOnline in the Netherlands. The results of these research studies provide the basis for treatment and prevention for Dutch dementia patients. From this report of research trials in dementia patients, several noteworthy observations emerge.
Study characteristics
There is a discrepancy between the focus of the research trials and the prevalence rate of the different types of dementia. The estimated prevalence of Alzheimer’s disease, as reported in the literature, varies between 30 and 75% of all dementia patients.7-9 The WHO estimates that Alzheimer’s disease accounts for approximately 41% of all dementias and vascular dementia for 32%.8 Stevens et al. reported a prevalence of 31% Alzheimer’s disease, 22% vascular dementia, 3% Parkinson’s dementia, 8% frontotemporal dementia and 11% Lewy body dementia.9 As our data have shown, a disproportionate number of research trials, which specified the subtype of dementia, focus on Alzheimer’s disease (figure 4).
Mixed pathologies are common in practice, and it is not always easy to distinguish clinically between the types of dementia. This is especially true for Alzheimer’s disease and vascular dementia, and Alzheimer’s disease and Lewy body dementia.7,10 The nine study protocols that studied two or three types of dementia did aim to differentiate between subtypes of dementia. The 33 studies that enrolled patients with all types of dementia did not make that distinction, disregarding the necessity of an appropriate diagnosis of type of dementia to tailor future cure and care. Although different types of dementia are described in the literature, it is not always possible to distinguish the specific types of dementia in a single patient. In the studied protocols, it was not always described on what ground a dementia subtype was diagnosed; since the goal of our study is to sketch the landscape of scientific research regarding dementia, we have followed the assumptions made regarding subtypes of dementia.
In addition, our study suggests that the number of industry-sponsored trials has increased over the past years. These mostly focused on drug trials concerning Alzheimer’s disease and MCI. Not many trials focused on vascular dementia, Parkinson’s dementia, Lewy body dementia and none on familial dementia. The number of participants in these few studies was also fairly low, implying these types of dementia are comparatively understudied in the Netherlands. The Lewy body dementia and vascular dementia studies were conducted as mono-centre studies in the Netherlands, thus for each of these subtypes of dementia, only one single institute has studied these conditions in the past ten years. Although small trials are necessary in some cases (e.g., early-phase drug studies, trials of rare/orphan diseases), obtaining clinically meaningful and generalisable information from small studies may be difficult.
Clinical research is reported to undergo the same globalisation process as other industries and sciences, especially in the realm of clinical trials.11,12 Our data showed that 32% of the studies enrolled patients in at least one country outside of the Netherlands. Cooperation between centres (multicentre research) is considered beneficial, because it contributes to the generalisability of the patient population. Multicentre research can also contribute to the inclusion of sufficient participants, which might be a challenge in a population as heterogeneous as dementia patients. However, the living and care conditions vary tremendously in different countries, which can complicate multicentre international research in patients suffering from dementia.
Representation of dementia patients
The discovery of effective interventions to prevent or delay disability in older persons is a public health priority. In order to let the growing number of dementia patients benefit from the findings in research, it is necessary that the results of the research trials can be extrapolated to the general population of dementia patients.
In the Netherlands, most people with advanced stages of dementia live in nursing homes, which is approximately 25% of all dementia patients.13 We have seen that the dementia research protocols mainly focus on mild/ moderate dementia, as can be concluded from the MMSE/ CDR scores used in the analysed protocols as well as by the requirement that people should still live at home. To be living independently at home, one would expect a MMSE score of approximately 15 or more. Most patients suffering from advanced dementia will not be living at home independently. When a patient is only eligible for enrolment in studies if living independently at home, it is safe to assume that he or she will not be suffering from advanced dementia. Dementia patients living in nursing homes differ in relevant aspects from patients living at home, concerning the severity of the dementia and the care needed.
Many of the findings obtained in independently living patients cannot be extrapolated to severely demented patients. Since the severely demented patient group requires and receives the most intense care, one would expect a large proportion of the observational or care research to be conducted in this group.
The need for assistance with daily living, impaired cognition and incontinence can affect both the efficacy and the risks of a particular intervention and also the ability of a patient to implement a treatment or successfully complete self-management tasks.2
Elderly patients typically have concomitance of multiple illnesses, as a result of two processes: the association between age and incidence of degenerative diseases and the development over time of complications of the existing diseases. Comorbidity is considered one of the hallmarks of geriatric patients, and a fundamental component of their complexity. Sensory impairment is prevalent among the elderly; in people aged 70 years or older, approximately 24% to 36% suffer from visual impairment or blindness and one-third of all people over the age of 65 experience disabling hearing loss.14,15 Somatic multimorbidity is prevalent in 55-98 patients aged 65 years or older.4 As shown in the results, most research protocols incorporate exclusion criteria regarding somatic comorbidities or sensory impairment. The research participants are generally required to be healthy and not sensory deprived, whereas the average dementia patient has several comorbidities, including sensory deficits. Therefore, dementia patients included in research protocols do not seem to represent the average patient population suffering from dementia. Excluding patients with comorbidities limits the external validity and might not truly represent the wider spectrum of patients seen in clinical practice. To the degree that it is clinically feasible, studies should include multimorbid individuals of all ages reflective of the general dementia population. A possible solution to the limited external validity of randomised controlled trials (RCTs) is the implementation of pragmatic studies (or real-life studies), which are gaining widespread recognition and support among clinicians and are of particular interest for policy-makers.17-19 Pragmatic studies are designed to evaluate the effectiveness of interventions in the full spectrum of real-life settings in order to maximise applicability and generalisability, as opposed to the optimal situations created in RCTs. Therefore, these studies are suitable for including a large number of participants, have a small number of eligibility criteria to allow a variety of patients in the trial, have patient-centred outcomes, and use clinical interventions similar to those used in routine care.18,19
A surprising finding that deserves attention is the frequent mention of ambiguous exclusion criteria. These criteria offer researchers too much freedom to selectively exclude potential research participants without the intervention of a Research Ethics Board. The selective exclusion of eligible research participants is, however, problematic for both scientific and ethical reasons. It results in an arbitrary selection of participants and limits both the internal and external validity of the study. Preventing eligible patients from participating in research is also known as gate keeping,20 and withholds the choice to participate from research participants.
Limitations
A limitation inherent to the use of the research registry includes missing data; for example the phase of the drug trial, information regarding the informed consent process and the competence of the research participant were not included in the publicly accessible part of the registry. Therefore, we could not provide a further analysis of the mismatch between the necessity for informed consent and the apparent lack of attention for the research participant’s competence. Furthermore, the registration of biomedical research trials in the web portal ToetsingOnline has been compulsory since the end of 2011, while before that time it was voluntary; it is thus unclear how many data are missing from the years before 2011. Finally, since ToetsingOnline is a prospective register, the number of participants is based on an anticipation of the researchers and does not necessarily correspond with the actual number of enrolled participants.
Concluding remarks
In our study we found that the distribution of dementia research over the different types of dementia does not correspond with the prevalence of these dementia types in clinical practice. Furthermore, we found that the research population is not representative of the larger population of people suffering from dementia. Therefore, the possibility to extrapolate research findings of drug, intervention and observational studies to the patient population is limited. Furthermore, the exclusion of dementia patients in the more advanced stages of dementia in research studies means that this group of patients cannot benefit from possible therapeutic effects of the studies and may not profit from developed interventions and new insights, because this group differs significantly from the group of research participants. Moreover, ambiguously formulated exclusion criteria should always be avoided and should not be accepted by Research Ethics Boards, because these criteria limit the internal and external validity of the research.
A greater number of dementia patients could derive benefit from research if the research agenda were more closely aligned with disease prevalence. Lewy body dementia, familial dementia and vascular dementia are understudied compared with their disease prevalence and require more attention. In order to improve the generalisability of the research findings to the broader dementia population, it is important that the research participants reflect the population of patients. This is important for both intervention as well as observational studies. Regarding the extrapolation of research results of intervention studies, we encourage the conduct of ‘pragmatic studies’ in order to extend the applicability of RCT results to real-life settings. Our study may be useful to stakeholders, including policy makers, academic centres, industry, and investigators, and aid future decision-making regarding the conduct of trials in dementia patients. A better understanding of which conditions and populations are insufficiently addressed in the current research practice should provide guidance to organisations on how to allocate and prioritise available resources.
DISCLOSURES
No conflict of interest was declared by the authors.
K.R. Jongsma and S. van de Vathorst received a research grant from the Netherlands Organization for Health Research and development with grant number: 40-41500- 98-99002.
The funder had no influence on the design, methods, subject recruitment, data collections, analysis and preparation of the paper.
REFERENCES