KEYWORDS
Atrial fibrillation, anti-thrombotic treatment, cardiology/ standards, practice guidelines as topic, primary stroke prevention
INTRODUCTION
Prevalence of atrial fibrillation (AF), a common arrhythmia, has increased over the last 30 years.1 In 2006 its prevalence ranged from 0.7% (age group 55-59) to 17.8% (age group > 85). AF increases the risk and severity of stroke.2,3 Antithrombotic therapy with oral anticoagulation (OAC) decreases this stroke risk, but at the same time increases the risk of major bleeding.2
National and international clinical practice guidelines on AF management provide guidance on how to weigh these risks against expected benefits, and include recommendations on antithrombotic treatment.3,4 Yet, antithrombotic treatment is often not in line with these recommendations.5,6 For example, a study in the Netherlands estimated that less than 50% of AF patients received treatment according to national guidelines.6
Various types of barriers may thwart physicians in following guidelines in clinical practice, such as lack of familiarity with the guideline’s content, lack of skills or resources to change current practice or patients not reconciling with guideline recommendations.7 Suboptimal guideline features (i.e. format and content) may also hamper implementation.7,8 Lugtenberg et al. reported this as one of the barriers that hindered general practitioners (GPs) in following the Dutch College of General Practitioners (NHG) AF guideline.9
Improving these features may positively affect guideline use.8,10,11 Therefore, this study systematically appraised the format and content of the NHG AF guideline4 to identify suboptimal features that may hamper its use in Dutch primary care. We focused on the guideline section related to prescription of OACs for stroke prevention in AF patients. The results of this appraisal may contribute to improving features of future AF guideline versions, as well as to developing tools and strategies for AF guideline implementation.
MATERIAL AND METHODS
The NHG AF guideline
The NHG aims to promote evidence-based primary care by bridging the gap between theory and practice. With 12,000 members,12 they cover around 80% of all Dutch GPs13 and nurse practitioners.14 The NHG has developed over 100 guidelines covering the diagnosis and treatment of acute and chronic conditions, with ten guidelines related to cardiovascular diseases. In the current study, we reviewed the 2013-updated version of the NHG guideline on diagnosis and treatment of patients with atrial fibrillation.4 It consists of 34 pages of background information on AF, recommendations for practice, endnotes and references. We focused on three key recommendations: (I) Eligibility criteria for OACs; (II) Which type of OAC to prescribe: coumarin derivatives versus new OACs (NOACs); and (III) Type and dosage of coumarin derivatives. The recommendations are displayed in table 1.
Systematic appraisal of guideline features
To systematically appraise the features of the NHG AF guideline, we used the Appraisal of Guidelines, Research and Evaluation (AGREE) II tool15 and the GuideLine Implementability Appraisal (GLIA) tool.16 Both tools are publicly available, and have previously been used for guideline appraisals.17-19
The AGREE II tool focuses on assessing the methodological rigour and transparency with which a guideline has been developed. It contains 23 items grouped in six domains (table 2, first column). Each item reflects a statement that refers to the guideline as a whole (e.g., ‘Key recommendations are easily identifiable’), and is scored on a seven-point Likert scale, ranging from 1 (strongly disagree) to 7 (strongly agree).15
To identify obstacles to implementation of the guideline’s key recommendations on OAC prescription (table 1), we completed GLIA appraisals. GLIA consists of 21 items in eight dimensions that – in contrast to AGREE II — are scored at the level of individual recommendations (table 3, first column). Each item is formulated as a question (e.g., ‘Is justification for the recommendation stated explicitly?’) with response categories ‘yes’, ‘no’, ‘not applicable’, and ‘unsure’. We did not assess GLIA’s global dimension that appraises the guideline in its entirety and largely overlaps with the AGREE appraisal.
Data collection and analysis
Following the AGREE II and GLIA manuals,20,21 our appraisal panel consisted of four experts, representing a mix of clinical and methodological guideline expertise: one general practitioner (WL), one expert on antithrombotic treatment and stroke prevention in AF patients (DA), and two experts on guideline development and implementation (AB, SV). Panel members first individually performed the appraisals, using the online AGREE (www.agreetrust. org) and GLIA (eGLIA; http://nutmeg.med.yale.edu/glia) tools. They also provided additional information in free text fields to explain their scores. The appraisal process was primarily informed by the guideline document itself, but when necessary, extra information was collected from: i) the NHG website; ii) the booklet on NHG guideline development procedures;22 and iii) a structured interview with two members of the NHG AF guideline development group. The appraisal coordinator (AB) then summarised the results as input for a group discussion based on which panel members could alter their scores when they considered this appropriate (e.g., to correct for available data that were overlooked during the initial appraisal). We discussed every item for which scores differed by more than one point, and every item for which the NHG development group members provided additional information.
AGREE II domain scores were calculated by summing up the individual appraisers’ scores for each item within a domain (i.e., obtained score), and then standardising this as a percentage of the possible maximum score for that domain,20 as follows:
As a result of the consensus procedure for GLIA scores, features were categorised as optimal (‘Y’ in table 3) or suboptimal (‘N’ in table 3). Per recommendation, we calculated the percentage of suboptimal features as follows:
RESULTS
AGREE II appraisal of overall guideline features
Appendix 1 presents the individual appraisers’ scores before and after group discussion. The group discussion resulted in 24 out of 92 (26%) scores being changed. The main reasons for appraisers to change their scores were: correction for available data that were overlooked during the initial appraisal (8 of 24 changes; 33%); a change of opinion following clarification of other appraisers’ opinion (7 of 24; 29%); correction for additional information provided by the NHG guideline development group members (6 of 24; 25%). After the group discussion, standard deviations of item scores ranged between 0 and 1.6, with the majority (75%) being 1 or lower.
Table 2 presents the final item and domain scores assigned during the AGREE appraisal of the guideline. Domain scores ranged from 52.8% for ‘Stakeholder involvement’ to 87.5% for ‘Editorial independence’.
GLIA appraisal of key recommendation features
Table 3 displays suboptimal features at the recommendation level, which may hinder the guideline’s implementation and applicability in practice. The percentage of suboptimal features for the three key recommendations on OAC prescription (R1-3) ranged from 24% to 45%. The panel considered the domains ‘Effect on process of care’ and ‘Measurability’ optimal across all recommendations.
We found that all recommendations suffered from suboptimal decidability due to ambiguous or unclear conditions for when to apply a recommendation. For example, not all GPs may agree on what could be considered ‘high’ (R2) levels of medication adherence, or ‘relative contraindications’ for prescribing (R3 - referenced table). This, together with the lack of detail on how to prescribe NOACs (i.e., suboptimal executability of R2), led the panel to score many of the computability features as problematic. This reflects the panel’s expectation that operationalising the recommendations in an electronic information system may be difficult.
The strength of recommendations, and thus the degree to which they applied to all patients, was often unclear, resulting in a suboptimal flexibility score. This stemmed from a lack of standardisation of how recommendations were formulated. Terminology ranged from ‘absolute contraindications of NOACs are…’ (R2) to ‘Coumarin derivative dosing […] should aim’ (R3), and from ‘Recommend oral anticoagulants …’ (R1) to ‘… consider NOACs only…’ (R2). In some cases, actions were only suggested indirectly: ‘…coumarin derivatives are preferred’ (R2); ‘…a loading dose is given for the first days’ (R3).
DISCUSSION
In this study we systematically appraised features of the NHG AF guideline to identify guideline-related barriers that may hamper optimal prescription of oral anticoagulants in Dutch primary care. The editorial independence of the guideline development group scored highly; scores for all other aspects of the guideline’s methodological quality were acceptable. At the recommendation level, the main implementation obstacles were the lack of explicit statements on the quality of the underlying evidence, lack of clarity around the strength of recommendations, and suboptimal computability hampering operationalisation of recommendations in electronic systems.
The scores for the NHG AF guideline were high in comparison with those assigned in other, similar guideline appraisal studies. For example, the systematic review of guideline appraisal studies by Alonso-Coello et al.23 summarised the methodological quality of 626 guidelines, and reported mean AGREE II scores that were lower for all six domains. The study by Sabharwal et al.17 appraised 101 cardiac clinical practice guidelines. Compared with the NHG AF guideline, they found higher mean scores for the ‘Scope and Purpose’ domain (64% and 85%, respectively) and for ‘Clarity of Presentation’ (67% versus 82%), but lower scores for the four remaining domains.
Suggestions to improve the NHG AF guideline
1. Extend stakeholder involvement in the guideline development process
The AGREE II domain ‘Stakeholder involvement’ obtained the lowest score of all domains (53%). This was partly due to the lack of representation of a wide range of stakeholders in the guideline development group, which consisted of general practitioners and an epidemiologist. Other relevant disciplines, such as neurologists and representatives of anticoagulation clinics, were consulted at the external review stage. Yet, AGREE II and other accepted guideline standards advocate multidisciplinary development groups because they tend to generate more balanced views than single-speciality groups.24,25 Inviting representatives of other disciplines as members of the NHG guideline development group would be one way to increase multidisciplinary involvement. An alternative approach may be to consult the current panel of external reviewers earlier on in the process, for example when setting the guideline scope, selecting or rating the evidence, or when formulating the recommendations.
The low ‘Stakeholder involvement’ score also stemmed from the apparent limited extent to which patient experiences and expectations informed the NHG AF guideline. The national patient organisation was invited for external review, but it was unclear if and how their suggestions were addressed. Similar to the approaches for increased stakeholder involvement described above, patient involvement may be facilitated by having patient representation in the guideline development group, or by formal patient consultation at earlier stages of guideline development.26 Alternatively, a literature review or patient interviews could inform a guideline section summarising patient views on and experiences with antithrombotic treatment for stroke prevention.
2. Standardise layout and language of key recommendations
Although the key recommendations are clearly presented in the online summary of the guideline (https://www.nhg. org/standaarden/samenvatting/atriumfibrilleren), they remain hidden in the full text version. Using a specific font, framing them in a box, or (if possible) presenting them as a flowchart would support distinguishing key recommendations from other types of information in the guideline. Additionally, applying standardised language may help guideline users to recognise recommendations as such, whilst aligning interpretations of whether recommendations should be considered relevant for all patients. The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) framework27 proposes standardised guideline language: recommendations can be either strong (level ‘1’) or weak (level ‘2’), which translate into the phrases ‘we recommend’ or ‘we suggest’, respectively. Adopting GRADE as part of the guideline development methodology also ensures systematic assessment of the quality of the underlying evidence, with the letters ‘A’ (indicating high quality) to ‘D’ (very low quality) explicitly communicating evidence quality at the recommendation level. For future versions of the AF guideline, the NHG’s updated procedure booklet (published January 2015) includes guidance on how to use the GRADE framework for assessing and summarising quality of the underlying evidence.
3. Suggest criteria for monitoring the guideline’s use in practice
Facilitating local or regional monitoring of the guideline’s use in practice requires clearly defined criteria derived from the guideline’s key recommendations. For the NHG AF guideline, examples of criteria may be ‘the percentage of female patients who are prescribed oral anticoagulants and are aged 65 years or older’, or ‘the percentage of patients on coumarin derivatives with an INR between 2.0 and 3.0’. Suggesting monitoring criteria as part of the guideline would provide a suitable starting point for developing audit and feedback, which Dutch GPs considered an encouraging strategy to improve guideline adherence.28
4. Prepare electronic implementation in parallel with the guideline development process
Recent studies have focused on providing GPs with clinical decision support systems (CDSS) to improve primary care stroke prevention in AF patients.29-31 These systems use decision rules to evaluate the current treatment of the patient and, if necessary, recommend the GP to modify it. Creating these decision rules involves translating guidelines into a format that is interpretable by a computer. The GLIA dimensions ‘Decidability’ and ‘Computability’ relate to obstacles for electronic implementation, i.e. translating guideline recommendations into actionable, computable decision rules. In the current study, the NHG AF guideline scored poorly for these dimensions, indicating presence of ambiguous terms. Although this guideline was included in CDSSs for Dutch GPs,26,29 the suboptimal computability hampered interpretation and translation of individual guideline statements into electronic decision rules. Especially the lack of clear definitions for certain contraindications and the unavailability of structured data to identify contraindications in electronic health records required input from an expert group of clinicians to fully convert the guideline into unambiguous decision rules.26 Based on this finding we suggest involving a CDSS specialist when formulating recommendations for future updates of the NHG AF guideline. By preparing electronic implementation in parallel with the guideline development process, vague and inconsistent recommendations can be identified and resolved before publication.30 This may not only improve overall implementability of the guideline in practice, but also facilitate the development of effective CDSS interventions.
In conclusion, this study provides pointers for optimising future versions of the NHG AF guideline. Future research should investigate whether applying these suggestions indeed positively affects implementation of the guideline in primary care, which in turn may improve the adequacy of antithrombotic treatment and stroke prevention in patients with atrial fibrillation.
ACKNOWLEDGMENTS
We would like to thank Maureen van der Donk and Wim Opstelten for providing additional information on the NHG guideline development process.
DISLOSURES
Sabine van der Veer is funded by the European Renal Association – European Dialysis Transplant Association (ERA-EDTA) for a research fellowship on guideline development and implementation. Wim Lucassen is appointed member of the Authorisation Committee of the NHG.
REFERENCES
