KEYWORDS
Acute care, out-of-hours service, readiness for practice, resident
INTRODUCTION
The importance of timely identification and adequate resuscitation of critically ill patients, as a means to save lives and reduce the number of intensive care admissions, has long been established.1 Residents play a crucial role in this primary assessment, as they are often the first to see the patient.2,3 This is certainly the case during out-of-hours service, when bedside supervision is often not immediately available. Concerns have been raised about potential negative effects of this practice on patient safety. Especially junior residents, having limited clinical experience, may well lack the skills and competencies required to deliver out-of-hours service, and acute care in particular, at a high standard.4-7
Several studies have been published discussing the general transition from medical school to clinical practice.8-10 Residents’ perceived readiness specifically for out-of-hours service, however, remains underexposed. Previous research demonstrates the challenges junior residents face in this context and emphasises that providing a high standard of care to critically ill patients requires more than knowledge and skills.11-13
The Dutch medical curriculum14 shows much resemblance to other medical curricula.15 A three-year bachelor program consisting of mainly problem-based education is followed by a three-year master program incorporating clinical clerkships in a variety of hospital departments and general practice. Generally, clerks will experience increasing levels of responsibility during their training. After finishing their undergraduate education, most graduates will start a residency training program and work under supervision for 3-6 years before practising medicine independently. Traditionally, residents will start participating in out-of-hours service within their first months of clinical practice.
The primary goal of this national exploratory study was to assess whether or not Dutch residents feel sufficiently prepared to provide out-of-hours service at the time of their first shift. Secondly, we aimed to identify factors influencing perceived readiness. 
MATERIALS AND METHODS
This national exploratory study targeted Dutch residents of all levels of experience and different medical specialties, actively involved in out-of-hours service. Residents not involved in out-of-hours service, or not participating in the acute care setting whilst providing out-of-hours service, were excluded. Residents still within their first year of specialty training were labelled junior residents, residents in higher years of specialty training were labelled senior residents.
Data collection
In collaboration with the Dutch Federation of Young Medical Specialists, a link to an online questionnaire was made available from 21 September to 10 November 2015. All active federation members (n = 2626, September 2015) received an email inviting them to participate. Also, the questionnaire could be accessed by non-members through the Dutch Federation of Young Medical Specialists website, which was repeatedly announced on social media and in national medical journals.
Questionnaire
The questionnaire contained a total of 194 questions (a combination of previously validated questionnaires and newly composed questions), focussing on residents’ working conditions. Fourteen newly composed questions were relevant to this study (table 1). Question formats were multiple choice, numerical response, yes or no, or free text. Ethical approval for this study was obtained from the Dutch Society for Medical Education Ethical Review Board (file number: 342).
Data analysis
Current medical specialty was categorised into surgical, non-surgical, supporting and other specialties. The total number of months of clinical experience prior to the first out-of-hours shift was categorised into 0 to < 3 months, 3 to < 6 months, 6 to < 9 months, 9 to < 12 months and > 12 months, which corresponds to the quarterly assessment of junior residents, as is common in most first year residency training programs. Using SPSS 22 descriptive statistics, overall baseline characteristics were collected. Residents who felt sufficiently prepared were compared with residents who did not, exploring both individual characteristics and environmental factors. Chi square tests and independent samples T-tests were performed as appropriate. A p-value < 0.05 was considered statistically significant.
RESULTS
A total of 1220 residents from over 30 different medical specialties responded. After excluding residents who were not involved in out-of-hours service (n = 81) or who did not actively participate in the acute care setting whilst providing out-of-hours service (n = 179), a total of 960 residents remained for final analysis (figure 1).
Baseline characteristics
The 960 residents had a mean age of 32.5 years (SD ± 3.5); 695 residents were female (72.4%, table 2). On average, residents had completed their undergraduate medical education 6.7 years earlier (SD ± 3.0). Residents had chosen a surgical specialty in 28.5%, a non-surgical specialty in 57.6%, and a supporting specialty in 9.1%. The remaining 4.8% selected ‘other’. Eighteen percent of responding residents were labelled junior residents. Overall, 42.6% currently worked in a university medical centre, 47.3% in a district general hospital, and 10.1% worked elsewhere.
Residents’ readiness for out-of-hours service
Of the residents, 614 (64.0%) felt sufficiently prepared for their first out-of-hours shift, 346 residents (36.0%) did not (table 3). The two groups were subsequently compared, exploring both individual characteristics and environmental factors. The difference in mean age (32.3 and 32.8 years, respectively), although statistically significant (p = 0.030), was minimal. No influence of gender (p = 0.328) or chosen medical specialty (p = 0.139) was observed. The percentage of junior residents was significantly higher amongst residents who felt sufficiently prepared (20.2%) compared with residents who did not (14.2%, p = 0.020). A total of 631 residents (65.7%) were enrolled in out-of-hours service within their first three months of clinical practice. Prolonged clinical experience prior to the first out-of-hours shift, however, was associated with an increased sense of readiness (p < 0.001). Furthermore, the percentage of residents who had received additional training targeting out-of-hours service was higher amongst residents who felt sufficiently prepared (46.3%) compared with residents who did not (30.3%). Attending targeted training was associated with a significant increase in perceived readiness (p < 0.001). The majority of residents who felt sufficiently prepared had undergone some form of assessment of relevant skills and competencies prior to their first shift (70.7%). This percentage was lower amongst residents who felt insufficiently prepared (52.3%). Overall, assessment was associated with an increased sense of readiness (p < 0.001). Finally, formal consequences following negative assessment also showed a positive association with perceived readiness (p = 0.001). 
DISCUSSION
This study reviewed residents’ readiness for out-of-hours service as perceived by 960 Dutch medical residents. While 64% of residents felt sufficiently prepared for their first out-of-hours shift, a disturbing 36% did not. This lack of perceived readiness in acute care amongst junior residents was also noted by Tallentire et al. in a systemic literature review following the implementation of Tomorrow’s Doctors in the United Kingdom.16,17 Although hardly an ideal surrogate for actual preparedness, perceived preparedness in acute care is known to impact junior residents’ behaviour and therefore warrants further consideration.11 We found several factors possibly attributing to the primary outcome in our study population.
The percentage of junior residents was higher amongst residents who felt sufficiently prepared compared with residents who did not. Recent revision of the Dutch medical curriculum could partially explain this observation, similar to how the introduction of Tomorrow’s Doctors changed perceived readiness for practice in the United Kingdom.9,16,17 Studies evaluating the impact of Tomorrow’s Doctors suggest that recent changes in UK undergraduate training, while improving readiness in some areas, may have neglected acute care.16 On the other hand, having gained more clinical experience, senior residents could have become increasingly aware of their shortcomings, retrospectively influencing their personal assessment.
Most residents in our study population had their first shift well within their first three months of clinical practice. From an educational perspective, however, prolonging the time to gain clinical experience before engaging in out-of-hours service was associated with an increased sense of readiness. Our results suggest a tipping point somewhere around three months. To the best of our knowledge, no consensus exists regarding the minimally required number of months of clinical experience for new graduates prior to engaging in out-of-hours service, nor has this topic been previously explored. This subject therefore deserves further attention in the future.
The attendance of courses targeting aspects of out-of-hours care, although hardly common practice in either group, was associated with a higher sense of readiness as well, in accordance with previous findings.18,19 Unfortunately, these courses are often optional and/or not easily accessible, due to time-investment issues and costs. Also, courses vary considerably with regards to quality and content, mostly focussing on a specific subset of technical skills. The educational effects of handling the increased responsibility and ‘learning on the job’ during out-of-hours service, however, could be a valid reason for continuing current practice20,21 incorporating several, if not all, of the CanMEDS competencies22 in one complex task. The ideal timing, content and design of a course targeting out-of-hours service, remain to be determined.
Overall, assessment of relevant skills and competencies was associated with an increased sense of readiness. The checklist for new graduates, as mentioned in table 3, was first developed in 2011 and addresses skills and competencies required to provide high-standard care during out-of-hours service. Formal guidelines on how to acquire these skills and competencies to a certain predetermined level, however, are lacking, and the overall familiarity with the checklist seems low.
Finally, consequences following negative assessment, of which the necessity with regards to patient safety should seem self-evident to everyone involved, were associated with an increase in perceived readiness. It is plausible that residents felt reassured, knowing they would not be enrolled in out-of-hours service if their supervising attendants did not judge them competent. This would support the need for a competency-based approach to the timing of the first out-of-hours shift, rather than the now commonly used time-based approach.
There are several limitations to our study. First of all, although all residents in training could participate, only members of the Dutch Federation of Young Medical Specialists were actively approached, possibly introducing bias. Looking at the baseline characteristics, chosen medical specialties, and current site of employment though, our study population appears to be a reliable reflection of the current distribution of medical residents in our country. This survey targeted current working conditions rather than working conditions at the time of the first out-of-hours shift. Organisational aspects of acute care, such as 24/7 coverage of the emergency department by emergency physicians, could therefore not be assessed as contributing factors. Furthermore, asking participants retrospectively about their experiences as a junior resident may introduce recall bias. The tool we used for distribution of our questionnaire and the fact that this study was part of a much larger national questionnaire targeting residents’ working conditions, limited the number of questions we could present to our respondents and impacted the question formats we could use. As a result, we did not generate the level of detail needed for more in-depth analysis. Also, considering that this study was carried out within the Dutch health care system, results may not be totally applicable in other contexts. Finally, most literature focuses on acute care in particular, rather than out-of-hours service in general. Although caring for critically ill patients is a substantial part of the out-of-hours service, this made a comparison between our results and previous literature difficult.
In conclusion, 36% of responding residents felt insufficiently prepared for out-of-hours service at the time of their first shift. Our results emphasise the need for sufficient time to gain clinical experience as a new graduate, and underline the positive contribution of targeted training and assessment of skills and competencies relevant to out-of-hours service. For future research, we recommend conducting a prospective longitudinal study, following new graduates through their first year of clinical practice, with special focus on the first out-of-hours shift. Aside from identifying further challenges junior residents face in this context, this longitudinal study could also shed light on learning processes that occur during out-of-hours shifts, explore the optimal form and duration of clinical preparation for this complex task, and determine the best form of guidance during the first out-of-hours shift. Finally, considering our findings regarding training and assessment, we propose designing an educational program incorporating training and subsequent assessment of all skills and competencies relevant to out-of-hours service.
ACKNOWLEDGEMENTS
We would like to thank Geert Bouwmans for his contribution to the statistical analysis section. Also, we would like to thank the responding residents for their willingness to invest their time to help us explore this important topic.
DISCLOSURES
Conflicts of interest: none declared.
Source of funding: a research grant was made available to the corresponding author by the Department of Research in Learning and Education, Radboud university medical center Health Academy, Nijmegen, the Netherlands. This contribution is the result of a collaboration between the Radboud university medical center and De Jonge Specialist (the Dutch Federation of Young Medical Specialists).
REFERENCES