Emergency Physician Satisfaction and Accuracy of Paramedic Handover Information: A pilot study

  • Teresa M. Chan 1
  • Brendon Trotter 2
  • Serena Sennik 1
  • Caillin Langmann 1
  • Andrew Worster 1
  • Michelle Welsford 1
  1. 1.  Division of Emergency Medicine, Department of Medicine, McMaster University
  2. 2.  St. Joseph's Healthcare Hamilton, Hamilton, ON

Abstract

Effective and accurate communication is of critical importance when transferring patients between healthcare providers. The accuracy of handover information transmission during these encounters has not been well studied. From August 2010 to April 2011, a pilot study was completed to examine physician satisfaction and physician accuracy regarding the performance of prehospital interventions by paramedics. Our findings suggest that physician overall satisfaction (3/5 Likert score) and accuracy (16-44%) were low in our local milieu, suggesting the need for improvement processes.

Introduction

Handover of patient information is a vital component of clinical practice.  Such communication is especially important in high-acuity environments, like the emergency department (ED), where it is often not possible to obtain useful information from patients directly.(Stiell, Forster, Stiell, Walraven 2003) Without an effective prehospital to ED handover process, much of this information may be lost.(Yong, Dent, Welland, 2008; Brenner, Hilton, Carr et al., 2008) Our pilot study’s aim was to determine physician satisfaction with paramedic handover and the accuracy of the physician knowledge of prehospital events. 

Methods

From Aug. 2010 to Apr. 2011, a pilot study was conducted at two medium-volume (~50,000 visits were year) academic emergency departments in Hamilton, ON, Canada.  Research ethics approval was granted by our regional research ethics board, and participants provided written, informed consent.  A group of trained surveyors conducted in-person recruitment in the emergency departments at two sites.  We recruited a convenience sample of 36 emergency medicine physicians and residents (postgraduate year-2 and above) managing three cardinal presentations: chest pain, shortness of breath, and altered level of consciousness/weakness. We asked EPs about: (a) their satisfaction with handover about prehospital interventions (5-point Likert scale), and (b) their knowledge of interventions that were performed in the prehospital phase. Upon completion of each individual survey and satisfaction Likert data were immediately transcribed. The online supplement (part 1) shows the questions asked during each survey. Satisfaction questions regarding involvement of nurses was tracked to determine the number of cases wherein the nurses were involved in the prehospital-to-hospital transfer process.

 

We later extracted the data from the Ambulance Call Records (ACRs).  Up to three extractors located the paramedic records for cases that matched the recorded patient demographics, time and location of arrival.  For missing charts, a second extractor re-attempted a hand-search before declaring the ACR lost.

 

Data Analysis

We measured accuracy rates by comparing the answers rendered by the EPs on the survey to the official ACR documentation.  We calculated the rate of EP accuracy regarding their awareness of prehospital interventions, with the ACR as the reference standard.  If the EP answered that they “did not know” if an intervention was completed, this was considered an inaccurate response.  36 encounters were recorded and nine ACRs were lost to follow-up, thus 27 encounters had complete data. (See the online supplement part 2 for details)

 

Demographics

The demographics of the types of handover encounters are noted in Table 1.     

Table 1: Demographics of Handover Cases

Canadian Triage Acuity Score (CTAS) Proportions (n)

CTAS 1           8% (3)

CTAS 2         42% (15)

CTAS 3         47% (16)

CTAS 4           3% (1)

CTAS 5           0% (0)

Average Ambulance offload delay time

34 minutes (0 to 2:42)

Who was present at handover?

Charge Nurse (RN) present            40%    (14)

Bedside RN present                        80%    (28)

Housestaff Present             11%    (  4)

Attending Present                17%    (  6)

Types of Ambulance Crew

Advanced Care Paramedics            8%     (  2)

Primary Care Paramedics    22%     (  8)

Unknown                               71%     (25)           

Mean age of Patients

69 (Range 32-98)

Enrollment

Enrollment at Site 1                        42% (14)

Enrollment at Site 2                        58%   (21)

 

NB: There was one patient in which two physician encounters were recorded for the same patient (one resident and one attending physician).

 

Overall emergency physician accuracy rates regarding awareness of prehospital interventions were as follows: chest pain 40% (95%CI: 6.7-73%); shortness of breath 33%(18-48%); altered level of consciousness 16% (7.4-25%) (Table 2). See Figure 1 for the cases recruited.

 

Figure 1:  Cases Recruited

 

 

Table 2: Accuracy Rates for Physician Awareness of Prehospital interventions

Item

Average Accuracy

95% Confidence Interval

Chest Pain (n= 5; 5 items in checklist)

40%

6.7-73%

Shortness of Breath (n= 11; 6 items in checklist)

33%

18-48%

Altered LOC (n=11; 6 items in checklist)

16%

7.4-25%

 

The median satisfaction scores around each of the handover process components are shown in table 3. 

 

Table 3: Satisfaction Scores for the Handover Process Components

Handover Process Components

% (n)

Median

IQR (25-75%)

Overall Satisfaction

100% (36)

3

2

Nursing Verbal

53%  (19)

4

1

Nursing Written

72% (26)

4

1

EMS verbal Handover

19%  (7)

4

0

EMS written Handover

2.7%** (1)

4

0

Key for Table 3: Key for satisfaction Likert scale:

1 = Highly Dissatisfied; 2 = Mildly Dissatisfied; 3 = Neutral, No Opinion; 4 = Satisfied; 5 = Highly Satisfied

** This is likely an erroneous score as the investigators did not find or note the ACR in the patient file.

 

None of the paramedic charts (ACRs) were available to physicians at the point-of-transfer of care, though one physician erroneously rated written communication by paramedics.

 

Discussion:

Modern understanding of clinical competency is shifting toward the acknowledgement of the importance of healthcare teams, with less emphasis being placed on the individual (Lingard, 2012). Communication scenarios, such as transitioning patients from prehospital to ED settings, are a critical microcosm to examine team communication competencies.

 

In our pilot study of a convenience sample of EMS patient handovers we identified several issues to conducting a full scale study: these include a low enrolment and high lost-to-follow-up rates regarding paper-based ACRs.

 

Satisfaction scores

Previous studies have looked at satisfaction as a quality measure for handovers in the ED.(Yong, Dent, Welland, 2008; Brenner, Hilton, Carr et al., 2008)  Brenner et al. (2008) concluded that 51% physicians were satisfied with paramedic handover in the ED, even though the physicians only received 44% of the relevant handover data points.(Brenner, Hilton, Carr et al., 2008)  Our physicians reported a low overall satisfaction (average 3.03/5) versus component scores (ranging from 3.84-4.00). We hypothesize this may suggest some systems-level problems about which participants were not surveyed.

 

Of note, our study revealed a high dissatisfaction with EMS patient handover, which is very different from previous studies.  While we are confident in the validity of this finding, the poor precision of our results due to the small sample size and high lost-to-follow-up rate support our intention for a modified, larger study.

 

Accuracy Rate

Our physicians were not accurate about prehospital interventions completed on their patients by the paramedics, which is shown in table 2.  The accuracy scores for the three selected cardinal presentations ranged from 16-40%. These three cardinal presentations were selected because they were associated with the most interventions within our local prehospital paramedic protocols.  We thought that these interventions were deemed to be highly clinically significant as they may immediately affect the ED management (online supplement, part 3). Even considering our small numbers and wide confidence intervals around, physicians were consistently inaccurate about prehospital actions.

Limitations:

This was a pilot study and the emphasis of the study was to determine feasibility of a larges scale project using the same method, as such, the small sample size and high lost-to-follow-up rate suggest that substantial modifications must be taken to ensure better ACR recovery and documentation.  One of the key differences in our study was that the EPs did not have access to the written paramedic records prior to their direct patient encounter.  The ACRs were available 0% of the time in this pilot, compared to a rate of 50% previously reported (Yong, Dent, Welland, 2008), Of note, one of the main limitations was the choice to enroll physicians, rather than patient-information transfer encounters. Since across the two sites, we only had roughly 60 candidate physicians, this decreased our ability to enroll into our study.

Conclusions:

EPs are often be unaware of the interventions performed on patients in the prehospital setting. Physician inaccuracy in our study may be due to the lack of paramedic written record availability. Further study and multidisciplinary action is required to improve this vital clinical handover process.

Acknowledgements:

We would like to acknowledge the following reviewers for their comments and suggestions: Abdulaziz Alali, Aseem Bishnoi, Michael Longeway, Alim Pardhan, Minh Le Cong.  We are incredibly grateful to their contributions as their open peer reviews helped to improve our paper substantially.

References

 

Brenner J, Hilton J, Carr G, et al.  Information transfer from prehospital to ED health care providers.  American Journal of Emergency Medicine (2008) 26, 233–244.  DOI: 10.1016/j.ajem.2007.04.003

 

Lingard, L. (2012). Rethinking competence in the context of teamwork. The Question of Competence: reconsidering medical education in the twenty-first century, 42.

 

Stiell A, Forster AJ, Stiell IG, van Walraven C.  Prevalence of information gaps in the emergency department and the effect on patient outcomes.  Canadian Medical Association Journal. (2003), 169(10), 1023-1028.

 

Yong  G, Dent AW, Weiland TJ.  Handover from paramedics:  Observations and emergency department clinician perceptions. Emergency Medicine Australasia (2008) 20, 149–155DOI: 10.1111/j.1742-6723.2007.01035.x

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Appendix 1:  Survey on EMS-to-ED Handover Communication

Physician Satisfaction

Prompt 1:  Considering all the information you have at this point (prior to speaking with the patient), how satisfied are you with your KNOWLEDGE of the interventions completed in the pre-hospital setting?

 

Overall

1

(Highly Dissatisfied)

2

(Mildly Dissatisfied)

3

(Neutral, 

No Opinion)

4

(Satisfied)

5

(Highly Satisfied)

 

Prompt 2:  Considering all the information you have at this point, how satisfied are you with your knowledge of the interventions in the pre-hospital setting as articulated to you by the Paramedics VERBALLY?

EMS Verbal Handover

1

(Highly Dissatisfied)

2

(Mildly Dissatisfied)

3

(Neutral,    No Opinion)

4

(Satisfied)

5

(Highly Satisfied)

N/A

(didn’t speak with EMS)

 

Prompt 3: Considering all the information you have at this point, how satisfied are you with your knowledge of the interventions in the pre-hospital setting as articulated to you by the Paramedics’ WRITTEN NOTES?

EMS Written Handover

1

(Highly Dissatisfied)

2

(Mildly Dissatisfied)

3

(Neutral,    No Opinion)

4

(Satisfied)

5

(Highly Satisfied)

N/A

(EMS notes were not available)

N/A

(didn’t read EMS notes)

 

Prompt 4:  Considering all the information you have at this point, how satisfied are you with your knowledge of the interventions in the pre-hospital setting as articulated to you by the Nurses VERBALLY?

Nurses

Verbal

Handover

1

(Highly Dissatisfied)

2

(Mildly Dissatisfied)

3

(Neutral,    No Opinion)

4

(Satisfied)

5

(Highly Satisfied)

N/A

(didn’t speak with Nurses)

 

Prompt 5:  Considering all the information you have at this point, how satisfied are you with your knowledge of the interventions in the pre-hospital setting from Nurses’ WRITTEN NOTES?

Nurses

Written

Handover

1

(Highly Dissatisfied)

2

(Mildly Dissatisfied)

3

(Neutral,    No Opinion)

4

(Satisfied)

5

(Highly Satisfied)

N/A

(didn’t read the RN notes)

 

Prompt 6:  Consider the current system for communicating information about pre-hospital interventions.  Do you have any suggestions to improve the process?  (Please write on back of page if necessary)

 

 

 

 

 

 

 

 

 

Appendix 2:  Questions about Interventions asked of Emergency Physicians for each of the three selected “Cardinal Presentations”

 

Chest Pain

Altered Level of Consciousness / Weakness

Shortness of Breath

Was ASA given in the prehospital setting?

Was a Capillary Blood Glucose done by EMS?

Do you know the patient’s initial O2 sat on Room Air (i.e. without supplemental oxygen)?

Was Nitro-spray given in the prehospital setting?

Was an initial Blood Pressure noted to be abnormal?

Was supplemental oxygen or Non-invasive Ventilation (BVM, CPAP) given?

Was a 12-lead ECG done in the prehospital setting?

Was any medication given (Naloxone, IV dextrose, oral carbohydrate, midazolam)?

Were any medications given? (ie inhaled ventolin/epinephrine, IM epinephrine, Nitrospray, etc)

Was there an ST-elevation MI noted prior to arrival at hospital?

Was there any neurological deficits (focal deficits, seizure-like activity) that prompted change in prehospital management (Calling BHP, Calling for ACP assistance, Stroke Bypass)?

Was there any change (improvement, deterioration) in pt symptoms with prehospital interventions?

Was there an abnormal rhythm (SVT, AFib, VT) during transfer?

Was there an abnormal rhythm during transfer?

Was there an abnormal rhythm during transfer?

 

Was a 12 lead ECG done?

Was a 12-lead ECG done in the prehospital setting?

 

 

Showing 5 Reviews

  • Placeholder
    Alim Pardhan
    Originality of work
    Quality of writing
    Quality of figures
    Confidence in paper
    1

    Overall I think it is well written and clearly articulates what you are trying to say

    The paper talks about EP satisfaction on EMS handover but in many cases there were no physicians present at all so this is in fact not only handover, but second hand handover – we may ask different questions etc. Although this is a common practice for the non-CTAS 1/2 patient where we don’t receive handover directly

    It is unclear to me why the number of reports collected was so low over a ten month period, may be worth expanding on that

    It may have been interesting to poll the HHS group to find out in general how satisfied we are with EMS reports


    This review has 1 comments. Click to view.
    • Me
      Teresa Chan

      Thanks for your review, Alim.
      We expanded the explanation re: the low enrolment - we had our index cases be the physicians themselves, which meant, for each shift we could only enrol one patient-EMS-physician encounter.

  • Placeholder
    Abdulaziz Alali
    Originality of work
    Quality of writing
    Quality of figures
    Confidence in paper
    1

    I have no competing interests in relation to this paper
    thank-you for the invitation to review this paper on EMS handover and EM physician awareness of prehospital interventions

    Methods:

    I believe the paper should explain why only 36 the sample size & to include the
     inclusion/exclusion criteria, Also what data did they extract ?!.

    Also I found the best way to measure the accuracy is by comparing two exact answers for example ( did the patient received the intervention ?! Yes or no ). Instead of ER answer 
    that they "did not know" if an intervention was completed & that was considered an inaccurate response.

    Results:

    I found table 1 missing one patient. Total number (35) where the cases recruited are 36 ?! Perhaps this table can be made clearer.

    25% lost to flow-up are high & that could affect the results.


    Limitations:

    I agree with the data that 0% of ACRs were available at time of handover. As I observed that when I did ride-out with 
    Paramedics in Hamilton.


    Conclusions:

    This pilot study it's show how EP's are mainly unaware of prehospital interventions with low overall satisfaction. Yes may be one reason because 0% of written ACRs are not ready in the time of handover.

    But I think it would be nice if we have stander way of transferring the information between the paramedics and physicians or charge nurse to improve the accuracy of handover information.


    This review has 1 comments. Click to view.
    • Me
      Teresa Chan

      Hi there:
      Thanks for your review and corroboration of our findings. I think there is ample room to grow with regards to improving our local practice so that we can better patient care!

  • Placeholder
    Michael Longeway
    Originality of work
    Quality of writing
    Quality of figures
    Confidence in paper
    1

    I have no competing interests in relation to this paper
    thank-you for the invitation to review this paper on EMS handover and EM physician awareness of prehospital interventions

    Methods:

    Good choice for these three cardinal events. Busy calls for Paramedics and often has detailed reports.

    I believe that the inclusion/exclusion criteria needs to be included in this paper. I find it confusing that only 36 patients were included over this period of time at these EDs.

    Results:

    I found table 3 confusing. This was the first place where nurses reports are mentioned and I am not clear where this fits into the question. Perhaps this table can be made clearer.

    Limitations:

    I agree with the data that 0% of ACRs were available at time of handover. This is a common trend in Ontario. The time delay between patient handover and for the ACR to arrive at the hospital has increased since the implementation of electronic patient charting in EMS. A solution to this delay in patient charts should be addressed in future work as it is clear this impacts EPs awareness of prehospital interventions.

    Conclusions:

    The point is made here that the verbal report process is poor. It would be nice to have more accurate data to clearly see where the process is flawed and offer up some areas for improvement. I don't think this answered or identified how to improve.

    This review has 1 comments. Click to view.
    • Me
      Teresa Chan

      Thanks for the kind words and the thorough review. This was a pilot study to determine local quality of handover, a point that was well proven by our results and has affected change.

      The point of this study was not to answer or identify areas for improvement, but to prove a point that there was a need to look at this metric.

  • Placeholder
    Aseem Bishnoi
    Originality of work
    Quality of writing
    Quality of figures
    Confidence in paper
    1

    Methods:

    -      
    define “medium-volume”
    hospitals

    -      
    Inclusion & Exclusion
    criteria are not explicitly stated

    -      
    how were the cardinal
    presentation cases chosen? E.g. chief complaint?  Nurse assessment of the major issue?  EP diagnosis?—this needs to be defined

    -      
    “later extracted the data…” –
    what data did you extract?—needs to be defined.

     

    Data Analysis:

    -      
    Was the ACR considered
    gold-standard?  In itself, this is flawed
    b/c of imperfect retrospective paramedic documentation.  E.g. paramedics may have given a 3rd
    round of salbutamol, but not had time, or neglected to chart it.  In my jurisdiction, this seems to happen
    regularly & becomes evident in the Q&A process.

    -      
    25% Loss to F/U is pretty
    high…impacting the outcome?

     

    Results:

    -      
    Demographics of respondents not
    provided

    -      
    HUGE confidence intervals: e.g.
    Chest Pain: 6.7-73% (woah!)– what does it mean?  The sample size killed your ability to make
    any conclusion from this. Are EP’s aware of prehospital cardiac interventions?

    -      
    Figure 1: excellent tracking of
    patients.  Thank you for including this.

    -      
    Table 3: The layout is
    confusing.  Did 19ppl give Nursing verbal
    a median score of 4/5 (w/ IQR of 1)?  If
    that’s the case, why didn’t the remaining 17ppl  give a score for Nursing Verbal? I’d modify
    this to make it clearer.

     

    Limitations:

    -      
    The 0% written ACR rate is
    fairly accurate where I work too.  However,
    this number will vary depending on the reader / location of practice

     

    Conclusions:

    -      
    EP’s are mainly unaware of
    prehospital interventions because 0% of written ACRs are ready in time.  BUT in addition, your point is well taken: this
    study suggests that EP’s are unaware because the verbal handover process is
    poor – but is this not something we already know?  Did this add to our understanding of why?
    This study sample & data presented is too small & varied to truly make
    conclusions. 

    This review has 1 comments. Click to view.
    • Me
      Teresa Chan

      Thanks for the review Aseem. This was a pilot study, to determine feasibility of launching a larger study. At the end of the day, this paper was not meant to answer the question of why - but to determine that there was an area in need of change locally. We identified two key vulnerabilities, therefore (missing ACRs, and the poor quality of verbal-only information transfer).

  • Placeholder
    Minh Le Cong
    Originality of work
    Quality of writing
    Quality of figures
    Confidence in paper
    1

    I have no competing interests in relation to this paper
    thankyou for the invitation to review this paper on EMS handover and EM physician awareness of prehospital interventions

    I found this paper confusing. For a 10 month period of the pilot survey, only 36 cases were captured. I would have expected more numbers in 2 emergency departments over that period. The paper fails to describe inclusion and exclusion criteria, but does say it was a convenience sample. 
    The lack of availablity of ACR to the EM physician is baffling and likely accounts for the bulk of this trial's findings.
    The generalisability of this trial to other hospitals and EMS organisations is questionable at best.

    The conclusions made are only partly supported by this trial findings. At best it can be concluded an association may occur between lack of ACR and physician unawareness of prehospital interventions


    This review has 1 comments. Click to view.
    • Me
      Teresa Chan

      Thanks Minh for your comments. We have tried to explain that this was a pilot project, but it does highlight how contextual communication can be - and how a system can make or break the generalizability of a study's findings.

      You are correct - depending on the particulars of your prehospital system, this study may or may not be relevant.

License

This article and its reviews are distributed under the terms of the Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and redistribution in any medium, provided that the original author and source are credited.