Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Trends in residents’ factual descriptions six months after refresher training in writing medical incident reports

Medical Simulation Centre, Jichi Medical University, 3311–1 Yakushiji, Shimotsuke-Shi, Tochigi, 329-0498, Japan ↗
Publication year
Digital Object Identifier10.54941/ahfe1003470

Abstract

Many free descriptions in medical incident reports lack factual information. In a previous study, I conducted initial training and refresher training with residents to accurately describe facts. In this study, I compared 23 actual incident reports submitted by residents after initial training, 80 reports 6 months after initial training, 21 reports after refresher training, and 18 reports 6 months after refresher training to verify whether their writing skills were retained. As a result, the description rate of patients and that of medicine and equipment information was significantly highest 6 months after refresher training. Reporter’s action, post-incident response, and original work procedures also maintained a high description rate in incident reports 6 months after refresher training. For these categories, refresher training boosted residents’ writing skills, and the effect persisted 6 months later. For environment, team member's actions, and safety check procedures, the description rate significantly decreased 6 months after initial training, but gradually recovered after refresher training. However, the description rate after 6 months of refresher training is still not high. Further improvement of refresher training is considered necessary for these categories.

INTRODUCTION

Incident reports (IRs) are an important source of information when investi- gating the causes of medical incidents and developing prevention strategies. Most IR systems allow the reporter to enter the date, time, and location of the incident, name of the department, and number of people involved in the inci- dent in a pre-coded format. In addition, specific details of the incident event can be described in a free description field. In medical IRs, it is common to describe what happened, the contributory factors, patient details, patient harms, how incident was discovered, and the post-incident response (Battles et al. 1998; Beaubien & Baker 2002; Holzmueller et al. 2005; Kaplan et al. 1998; Mohsin, Ibrahim & Levine, 2019; Runciman et al. 1993). Contributory factors to incidents have been identified in various fields of medicine (Runciman et al. 1993; Vincent, Taylor-Adams & Stanhope 1998). Kawano (2012) proposed the P-mSHELL model as common contributory

© 2023. Published by AHFE Open Access. All rights reserved. 27

28 Maeda

factor to health care. This model uses seven categories of factors involved in a medical incident, such as patient, management, software, hardware, environment, and liveware (individual/team).

Various types of education on medical IRs have been attempted, for exam- ple, education modules on patient safety (Mohsin et al. 2019), the importance of IRs (Nakamura et al. 2014), reporting systems (Krouss et al. 2019), and error classification methods (Mohsin et al., 2019). These education modules have been effective in removing barriers against submitting IRs (Benn et al. 2009; Evans et al. 2006; Krouss et al. 2019; Pfeiffer et al. 2013; Vincent 2009). However, limited studies have evaluated the quality of the descriptions in post-education medical IRs. There are also limited reports of refresher trai- ning in IRs. In practice, many actual IRs submitted by clinical sites are of poor quality. Specifically, the facts are missing or undermined by the repor- ter’s subjectivity or speculation. If the quality of IRs is low, it is difficult to effectively and efficiently conduct incident investigations based on IRs. As a result, it is difficult to provide sufficient and effective feedback to reporters on many IRs.

PREVIOUS STUDY In our previous study, I organized a method to accurately describe facts, and initial training (IT) was conducted with medical residents during their first week of employment (Maeda et al. 2022b). Nine months later, I conducted a refresher training (RT) to sustain the IT’s effects and verified its short-term effectiveness (Maeda et al. 2023). However, whether RT should be continued permanently remains unclear.

During IT, a fictitious incident video was viewed by 124 residents who had been working at our hospital for one week. The residents then created a fictitious IR about the video based on the fact description method. I recom- mended that residents describe the 5W1H (When, Where, Who, What, Why, and How), things they witnessed, and conversation contents. I requested that they itemize these events in chronological order and in short sentences with clear subjects. This description method was proposed based on our analysis of missing facts and difficult-to-read sentences in actual IRs, in addition to the contributory factors related to medical incidents (Maeda et al. 2021a,b).

RT was conducted online 9 months after IT for 64 residents who had attended IT (Maeda et al. 2022a). During RT, I further recommended that residents use the following nine description categories for IRs: (1) patient (patient’s background), (2) software (safety check procedures), (3) software (original work procedures), (4) hardware (name of equipment used), (5) envi- ronment (the situation and environment wherein the reporter was placed), (6) liveware (individual: reporter’s actions [what did they do?]), (7) liveware (team: team member’s actions), (8) liveware (team: the content of the conver- sation), and (9) post-incident response. These description categories refer to the P-mSHELL model (Kawano 2012).

IT was implemented in April 2021 and RT in January 2022. To examine RT’s short-term effects, the authors compared the description rates of 23

Trends in Residents’ Factual Descriptions Six Months 29

actual IRs submitted by residents in the 4 months immediately following IT (IRs after IT), 80 actual IRs submitted in the 4 months after 6 months of IT (IRs six months after IT), and 21 actual IRs submitted in the 4 months immediately following RT (IRs after RT) for the nine description categories (Maeda et al. 2023).

Our current study is a continuation of this previous study and aims to further investigate the necessity of RT and retainment of IR writing skills based on the nine description categories. Specifically, I compared IRs after IT, 6 months after IT, after RT, and 6 months after RT to verify the quality of the descriptions in the IRs and to discuss the future of RT.

METHODS

In this study, 18 additional IRs submitted in the 4 months after 6 months of RT (IRs 6 months after RT) by 64 residents who had attended RT were analyzed. These are all IRs of residents who agreed to participate in the study. One researcher, a human factors and ergonomics expert (Certified Professio- nal Ergonomist: CPE), read all IRs in all groups (after IT, 6 months after IT, after RT, and 6 months after RT) and evaluated the presence or absence of descriptions based on the nine description categories. Cross-tabulation tables categorizing all IRs by presence/absence of description versus IRs after IT, 6 months after IT, after RT, and 6 months after RT were prepared for each of the nine description categories, and an χ2 test was conducted. A residual analysis was also conducted to identify which cell caused the significant dif- ference. The significance level was set at p = 0.05. SPSS Statistics (ver. 28, IBM, Armonk, NY, USA) was used for statistical analysis.

All actual IRs excluded personal information (hospital ID, patient name, age, gender, IR reporter ID, and reporter’s name) during the analysis phase. The residents were informed that they could opt out of the study at any point; I posted a public information document on our website allowing them to refuse to participate. The study design was approved by the Ethics Review Committee of Jichi Medical University (Approval No. 21-101).

RESULTS

The results of the χ2 test are shown in Table 1. There was a significant bias in the description rate of IRs after IT, 6 months after IT, after RT, and 6 months after RT for four categories (software [safety check procedures]: P = 0.039, hardware: P = 0.029, environment: P = 0.008, and liveware [team mem- ber’s actions]: P = 0.033). Residual analysis showed that the description rate of IRs 6 months after RT for two categories was significantly higher than that of the other IR groups (the value of the adjusted residual; patient 2.2, hardware 2.2).

The description rates of each IR group (after IT, 6 months after IT, after RT, and 6 months after RT) in nine description categories are shown in Figure 1. The description rate of patient and that of hardware was significantly high- est 6 months after RT (both 89%); for these categories, the description rate increased from IT to 6 months after RT. Meanwhile, for four categories, the

30 Maeda

Table 1. Comparison of the description rates between IRs after IT, 6 months after IT, after RT, and 6 months after RT using nine description categories.

Table available in the original PDF paper.

χ2 test Adjusted residual

Description categories

χ2 P V After After IT RT Six months after IT

Six months after RT Patient 6.744 0.081 0.218 -0.6 -1.8 1.0 2.2a

8.383 0.039c 0.243 0.9 -2.8a 1.8 1.2 Software; safety check procedures

7.622 0.055 0.232 1.2 -2.7a 0.8 1.8 Software; original work procedures

Hardware 9.009 0.029c 0.252 -1.0 -1.9a 1.6 2.2a 2.625 0.453 0.136 1.4 -1.1 0.6 -0.4 Environment 11.811 0.008b 0.288 2.1a -3.3a 0.8 1.7 Liveware (Individual)

8.750 0.033c 0.248 2.1a -2.8a 0.8 1.1 Liveware (Team: team member’s actions)

2.212 0.530 0.125 0.2 -0.9 -0.3 1.4 Liveware (Team: content of conversation)

0.449 0.930 0.056 0.1 -0.6 0.3 0.5 Post-incident response Note: aThe absolute value of the adjusted residual is greater than 1.96. bStatistically significant (p<0.01). cStatistically significant (p<0.05).

residual analysis showed that the description rate was significantly lowest 6 months after IT, but 6 months after RT, the rate gradually recovered (the description rate of IRs 6 months after RT; safety check procedures: 39%, original work procedures: 78%, environment: 72%, team member’s action: 67%). For two categories, the description rate remained above 70% from after IT to 6 months after RT (the description rate of IRs 6 months after RT; liveware [individual]: 83%, post-incident response: 78%), which was very high. The same results were obtained for conversation content, but the description rate remained lower than 30%.

DISCUSSION

There were differences in the description rates depending on each of the nine categories.

The description rate of patients and that of hardware was significantly the highest 6 months after RT (both 89%); these rates tended to increase from IT to 6 months after RT. Liveware (individual) and the post-incident response showed no significant change in the χ2 test results and maintained a high description rate from IT to 6 months after RT. A common feature of

Trends in Residents’ Factual Descriptions Six Months 31

Figure 1: Percentage of IRs after IT, 6 months after IT, after RT, and 6 months after RT describing facts related to description categories. Note: * the absolute value of the adjusted residual is greater than 1.96 (description rate significantly lower/higher than in other IR groups).
Original PDF Figure 1:

these categories is that they are also described in the medical records. Medical records are generally described according to SOAP (S: subjective informa- tion, O: objective information, A: assessment, P: planning) (Pearce et al. 2016). Patients mainly include S and O; information on drugs and equipment (hardware) comprises O and P; doctors’ thoughts and actions (liveware [indi- vidual]) include O, A, and P; and post-incident response falls under S, O, A, and P. Residents who had worked for our hospital for more than 18 months had more experience writing medical records and were accustomed to inclu- ding the abovementioned four categories regularly; thus, the effects of IT and RT were considered to be maintained. In addition, although not limited to residents, many of the IRs submitted are either transcribed directly from the

32 Maeda

medical record or partially rewritten. It is possible that the resident wrote the IR based on the medical record, which may have maintained the descri- ption rate in these four categories. This indicates that the training effect is sufficiently sustainable, and no further RT may be necessary.

The description rate of software (original work procedures) decreased significantly 6 months after IT, but it recovered and increased after RT. Ori- ginal work procedures are often clearly stated as protocols in each hospital and residents can refer to them when writing IRs. In other words, once the habit of describing this category in IRs is established, it is easy for residents to implement it. The description rate after 6 months of RT is 78%, which is high enough that further RT may not be necessary.

For environment, liveware (team member’s action), and safety check pro- cedures, the description rate significantly decreased 6 months after IT, and gradually recovered after RT. However, the description rate after 6 months of RT is not high. One reason for this is the strong dependence of these cate- gories on eyewitness memory. Eyewitness memories are difficult to maintain a high degree of reliability (Wixted, Mickes & Fisher 2018) and are easily distorted by the emotions of the parties involved, interventions from others, etc. (Fukushima & Itsukushima 2018). With only two training sessions in IT and RT, it may be difficult for residents to recall their eyewitness memory and describe them in their IRs. Contents of conversation, which have always had a description rate of less than 30% since IT, are also considered to be strongly dependent on eyewitness memory. Therefore, further RT is needed for these four categories.

The interval and frequency of RT after IT should be considered for these description categories in which trainees show low description rates. Because RT in this study was conducted 9 months after IT, it is possible that the skills to describe eyewitness memory decay. For example, it is recommended that the interval between IT and RT on resuscitation should be no longer than 7 months (Woollard et al. 2006). In the future, the impact of shortening the interval between IT and RT in this study should also be examined. Describing eyewitness memories can still be difficult for residents 6 months after RT. For this category, at least the IT and RT content needs to be improved. For exam- ple, the Self-Administered Interview© (SAI©) questionnaire (Gabbert, Hope & Fisher 2009) may be applied in future RTs to acquire eyewitness memories. This is used in criminal investigations and can provide more information than the free recall of eyewitness memories (Matsuo & Miura 2017). In health- care, it is expected that tools like SAI© will be developed to support accurate IR descriptions and collect information on the environment, team members’ actions, safety check procedures, and contents of conversation; and that IT or RT will educate residents about these tools. In the future, after such IT and RT improvements are made, it is necessary to consider how much the frequ- ency of RT should be increased. In resuscitation training, it has been shown that repeated RT gradually improves trainees’ skills (Wik et al. 2002), but too short time intervals and high frequency of training decrease trainees’ motiva- tion (Woollard et al. 2006). Chamberlain et al. (2001) reported that less than 50% of trainees return to RT even though they had agreed to participate in RT before receiving IT. In the future, IT and RT need to be redesigned to include this background and to describe eyewitness memories in IRs.

Trends in Residents’ Factual Descriptions Six Months 33

CONCLUSION

In this study, I analyzed medical residents’ IRs 6 months after RT to verify whether their IR writing skills were retained. Specifically, I compared IRs after IT, 6 months after IT, after RT, and 6 months after RT to verify the qua- lity of the descriptions. As a result, there were differences in the description rates depending on the nine description categories.

The description rate of patients and hardware was significantly highest 6 months after RT; for these categories, the description rate tended to increase from IT to 6 months after RT. Liveware (individual), post-incident response, and original work procedures also maintained a high description rate in IRs 6 months after RT. For these categories, RT boosted residents’ writing skills, and the effect persisted 6 months later, indicating that writing skills were still retained. Thus, sustained RT may not be required.

For environment, liveware (team member’s action), and safety check pro- cedures, the description rate significantly decreased 6 months after IT, and gradually recovered after RT. However, the description rate after 6 mon- ths of RT was not high. The description rate of contents of conversation was always lower than 30%. These categories were involved in descriptive skills related to eyewitness memory. The sustainability of training effects in these categories was low and further improvement of IT and RT is considered necessary.

ACKNOWLEDGMENT

I would like to thank Editage (www.editage.com) for English language editing.

FUNDING SOURCES This work was supported by the JSPS KAKENHI (Grant Number JP20K14985).

REFERENCES

Battles, J. B., Kaplan, H. S., Van der Schaaf, T. W., and Shea, C. (1998). The attributes of medical event-reporting systems: experience with a prototype medical event- reporting system for transfusion medicine, Archives of Pathology & Laboratory Medicine Volume 122 No. 3 pp. 231–238. Beaubien, J. M. and Baker, D. P. (2002). A review of selected aviation human

factors taxonomies, accident/incident reporting systems and data collection tools, International Journal of Applied Aviation Studies Volume 2 No. 2, pp. 1–42. Benn, J., Koutantji, M., Wallace, L., Spurgeon, P., Rejman, M., Healey, A., and Vincent, C. (2009). Feedback from incident reporting: Information and action to improve patient safety, BMJ Qualilty & Safety Volume 18 No. 1, pp. 11–21.

34 Maeda

Chamberlain, D., Smith, A., Colquhoun, M., Handley, A. J., Kern, K. B., and Woollard, M. (2001). Randomised controlled trials of staged teaching for basic life support. 2. Comparison of CPR performance and skill retention using either staged instruction or conventional training, Resuscitation Volume 50 No. 1, pp. 27–37. Evans, S. M., Berry, J. G., Smith, B. J., Esterman, A., Selim, P., O’Shaughnessy, J., and

DeWit, M. (2006). Attitudes and barriers to incident reporting: A collaborative hospital study, Quality & Safety in Health Care Volume 15 No. 1, pp. 39–43. Fukushima, Y. and Itsukushima, Y. (2018). Postidentification feedback distorts eyewitness memory: A review on current knowledge and future directions, Japanese Psychological Review Volume 61 No. 4, pp. 407–422. Gabbert, F., Hope, L., and Fisher, R. P. (2009). Protecting eyewitness evidence:

Examining the efficacy of a self-administered interview tool, Law and Human Behavior Volume 33 No. 4, pp. 298–307. Holzmueller, C. G., Pronovost, P. J., Dickman, F., Thompson, D. A., Wu, A. W., Lubomski, L. H., Fahey, M., Steinwachs, D. M., Engineer, L., Jaffrey, A., Mor- lock, L. L., and Dorman, T. (2005). Creating the web-based intensive care unit safety reporting system, Journal of the American Medical Informatics Association Volume 12 No. 2, pp. 130–139. Kaplan, H. S., Battles, J. B., Van der Schaff, T. W., Shea, C. E., and Mercer, S. Q.

(1998). Identification and classification of the causes of events in transfusion medicine, Transfusion Volume 38 No. 11–12, pp. 1071–1081. Kawano, R. (2012). Patient safety and quality of medical care. Topics: III. Management of patient safety and quality of medical care: Theory and practice; 2. Human factors in medical care, Nihon Naika Gakkai Zasshi Volume 101 No. 12, pp. 3463–3469. Krouss, M., Alshaikh, J., Croft, L., and Morgan, D. J. (2019). Improving incident

reporting among physician trainees, Journal of Patient Safety Volume 15 No. 4, pp. 308–310. Maeda, Y., Asada, Y., Suzuki, Y., Kawahira, H., and Shimpo, M. (2021a). Training program on how to write an incident report with improved readability: Guide for new medical staff, The Japanese Journal of Quality and Safety in Healthcare Volume 16 No. 4, pp. 479–487. Maeda, Y., Kawahira, H., Asada, Y., Yamamoto, S., and Shimpo, M. (2023). The

effect of refresher training on fact description in medical incident report writing in the Japanese language, Applied Ergonomics Volume 109, p. 103987. Maeda, Y., Suzuki, Y., Asada, Y., Yamamoto, S., Kawahira, H., and Shimpo, M.

(2021b). A study on the tendency of event expressions in free descriptions of medical incident reports, JES Ergonomics Volume 57(suppl), pp. 210–211. Maeda, Y., Suzuki, Y., Asada, Y., Kawahira, H., and Shimpo, M. (2022a). Follow-up simulation training on how to write incident reports’, Proceedings of the Japan Association for Simulation-based Education in Healthcare Professionals, Volume 10, p. 43. Maeda, Y., Suzuki, Y., Asada, Y., Yamamoto, S., Shimpo, M., and Kawahira, H.

(2022b). Training residents in medical incident report writing to improve inci- dent investigation quality and efficiency enables accurate fact gathering, Applied Ergonomics Volume 102, p. 103770. Matsuo, K., and Miura, H. (2017). Effectiveness of the Self-Administered Interview and drawing pictures for eliciting eyewitness memories, Psychiatry, Psychology, and Law Volume 24 No. 5, pp. 643–654.

Trends in Residents’ Factual Descriptions Six Months 35

Mohsin, S. U., Ibrahim, Y., and Levine, D. (2019). Teaching medical students to recognise and report errors, BMJ Open Quality Volume 8 No. 2, p. 000558. Nakamura, N., Yamashita, Y., Tanihara, S., and Maeda, C. (2014). Effectiveness

and sustainability of education about incident reporting at a university hospital in Japan, Healthcare Informatics Research Volume 20 No. 3, pp. 209–215. Pearce, P. F., Ferguson, L. A., George, G. S., and Langford, C. A. (2016). The essential SOAP note in an her age, The Nurse Practitioner Volume 41 No. 2, pp. 29–36. Pfeiffer, Y., Briner, M., Wehner, T., and Manser, T. (2013). Motivational antecedents

of incident reporting: Evidence from a survey of nurses and physicians, Swiss Medical Weekly Volume 143, p. 13881. Runciman, W. B., Sellen, A., Webb, R. K., Williamson, J. A., Currie, M., Morgan, C., Russell, W. J. (1993). Errors, incidents and accidents in anesthetic practice, Anaesthesia and Intensive Care Volume 21, pp. 506–519. Vincent, C., Taylor-Adams, S., and Stanhope, N. (1998). Framework for analysing

risk and safety in clinical medicine, BMJ Volume 316 No. 7138, pp. 1154–1157. Vincent, C. (2009). ‘Feedback from incident reporting: information and action to improve patient safety’, BMJ quality & safety, 18(1), p. 11–21. Wik, L., Myklebust, H., Auestad, B. H., and Steen, P. A. (2002). Retention of

basic life support skills 6 months after training with an automated voice advisory manikin system without instructor involvement, Resuscitation Volume 52 No. 3, pp. 273–279. Wixted, J. T., Mickes, L., and Fisher, R. P. (2018). Rethinking the reliability of eyewitness memory, Perspectives on Psychological Science Volume 13 No. 3, pp. 324–335. Woollard, M., Whitfield, R., Newcombe, R. G., Colquhoun, M., Vetter, N., and

Chamberlain, D. (2006). Optimal refresher training intervals for AED and CPR skills: a randomised controlled trial, Resuscitation Volume 71 No. 2, pp. 237–47.

Cover of the proceedings volume Healthcare and Medical Devices

Explore the complete digital volume

Healthcare and Medical Devices

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-55-1

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →