Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Using High Reliability and 'Lean' Thinking to Drive Excellence in Patient Safety: A Case Study from Radiation Oncology

Radiation Oncology Department – School of Medicine, University of North Carolina, Chapel Hill, NC 27599-7512, USA ↗
Publication year
Digital Object Identifier10.54941/ahfe100525

Abstract

A grand challenge for most radiation oncology clinics is to develop highly reliable systems that deliver value to every patient. We present a case study from the Radiation Oncology Department at the University of North Carolina (UNC) where we recognize that our systems are imperfect despite multiple built-in quality assurance (QA) steps to detect human errors. We took on a journey to organize our systems in such way that we are better able to notice the ‘week signals’ of human errors. We herein highlight previously-reported initiatives and demonstrate concurrent improvements in patient safety culture. In summary, this case study suggests that high reliability and ‘Lean’ thinking initiatives can be successfully implemented to yield measurable improvements in patient safety culture.

INTRODUCTION

Radiation oncology is a modest-sized field with ≈ 3,000 practicing radiation oncologists in the US. Nevertheless, the clinical impact of radiation therapy (RT) is large. Approximately 50% of patients with cancer receive RT, accounting for ≈ 600,000 patients annually in the US alone. Recent technological advances (e.g. medical imaging, computer-based planning systems, and radiation delivery systems) have driven a rapid evolution in clinical practice. We applaud the multiple technology-based initiatives aimed at improving patient safety; e.g. the efforts to promote inter-connectivity between different radiation therapy-related products. We understand the need for a strong focus on these technical factors. However, we believe that technical solutions alone are not going to bring our field to the desired level of reliability and value creation. Based on the available data, a reasonable estimate is that there is an incident during the course of treatment in ≈1-3% of patients, but the vast majority of these are not clinically relevant. This compares unfavorably with high reliable industries such as commercial aviation (≈1 death in 4.7 million passenger flights; IOM, 1999), or other areas of medicine such as anesthesiology (≈1 death in 200,000 procedures; IOM, 1999). These comparisons might not be totally fair since the reporting thresholds are different. If in aviation we were to count faulty take-offs, landings, or unplanned returns to the airport, and in anesthesiology we reported intubation failures or ventilator equipment/tube malfunctions, aviation and anesthesiology might not appear as favorable. The relatively high rate of any type of event within radiation oncology is cause for concern as it suggests inherent shortcomings of our current systems.

Therefore, recognizing the importance for change, since 2009 we have implemented several quality improvement initiatives that are amenable to metrics that quantify patient safety culture (Marks et al., 2011). We herein highlight previously reported initiatives in our clinic at three levels – Organizational, Workplace, and People

– and show how these initiatives have improved the patient safety culture in our department.

CONCEPTUAL MODEL A grand challenge for radiation oncology facilities is to develop highly reliable and efficient systems that deliver value to every patient. Value from the patient’s viewpoint is the timely provision of service that provides maximum therapeutic benefit with the least amount of cost, harm and effort. The opposite of reliability and value is waste. In a world of limited time and resources, any waste within the system serves (at least) as a distraction and (more often) as a hindrance to reliability and value. We define waste broadly, and on three levels, such as:

  • Organizational (e.g. suboptimal teamwork and communication, unclear or conflicting goals, unnecessary meetings, redundant/ambiguous policies, insufficient training of personnel, etc.);
  • Workplace (e.g. suboptimal workload and situational awareness, poor lighting, slow/cumbersome computer systems, cluttered clinic rooms, missing/broken equipment, interruptions, etc.);
  • People (e.g. suboptimal engagement in continuous quality improvement efforts, workarounds, etc.).

Besides leading to lost time, rework, and excessive rechecking, waste also creates anxiety, frustration, and low employee satisfaction. The relentless elimination of waste in any form and shape can enable the delivery of value to patients. We believe that the following concepts borrowed from high reliability and value creation organizations, as guided by the James Reason’s “Swiss Cheese Model” (Reason, 2004), can help guide radiation oncology centers (Figure 1).

Figure 1: Conceptual model. (Adopted from)
Original PDF Figure 1:

Patient harm, depicted at the far right-hand side of the Figure 1, is most-often attributed to the decisions and behaviors of individual people. High reliability and value creation organizations recognize that these active failure pathways are greatly influenced by the upstream factors, or latent failure pathways, at a workplace and organizational level. They are preoccupied with ways latent and active failure pathways can occur in the system. They work hard to detect and correct small emerging latent failures, and also see these as potential clues to additional latent failures elsewhere in the system. They anticipate specific failures that are at risk of occurring and build into their processes initiatives intended to prevent these failures. We embraced this approach to maximize patient safety.

ORGANIZATIONAL LEVEL The successful delivery of RT requires concerted efforts of multiple individuals at all levels in the organization. Human Aspects of Healthcare (2021)

Their activities need to be harmonized in order to develop systems that can prevent serious incidents, and eliminate waste. Below we highlight selected initiatives we spearheaded in our clinic to improve teamwork and communication at the organizational level.

Clinical Peer Review and Daily Huddles Peer review has been shown to facilitate quality assurance of RT planning resulting in improvement in patient safety (Adams et al., 2011; Marks, et al., 2013). We increased the formality of our long-standing pre-RT peer review processes and more systematically review things such as the indications for treatment, image segmentation, proposed dose/fractionation, and tentative beam arrangements. Attendees broadly represent the different components of the department and typically include faculty physicians, dosimetrists, residents, nurses, physicists, therapists, students, and administrators. This broad participation enables a wide array of possible challenges to be identified. Further, this peer review was integrated with a daily departmental “morning huddle” where the day’s schedule, anticipated challenges, patient census are discussed and where participants are encouraged to raise concerns, make announcements, etc. We also continued a second peer-review of patient’s records during their first week of treatment (akin to traditional “chart rounds”). This meeting is usually attended by a subset of the same staff that attends pre-RT peer review. Attention is paid to the final dosimetry, portal films, and dose prescriptions.

In order to assess the utility of the pre-RT peer review and morning huddle, we measured the: a. percent of cases with suggested changes in this meeting,

b. percent of cases with suggested changes during the second “on-treatment” peer review session (as an indication of the utility of the pre-RT review session), c. percent of patients who initiated therapy then needed “an avoidable re-plan.” A high rate of replanning suggests that the pre-RT peer review was suboptimal. This replan rate excluded replans that were anticipated (e.g. field reductions) or that were indicated for medical reasons (e.g. tumor shrinkage or progression).

d. Perceptions of the departmental personnel that attended the meeting regarding the utility of this meeting (via survey), and We noted that >50% of cases had suggested changes during the pre-RT peer review sessions. A far smaller number (8%) of patients have changes suggested at the second “on treatment” review, perhaps reflecting the effectiveness of the pre-RT peer review process (Adams et al., 2011). The fraction of patients who initiated therapy who then needed a “re-plan” (excluding anticipated field reductions or changes indicated for medical reasons) over time decreased significantly (p<0.01) (Chera et al., 2012). Survey responses of departmental personnel who attend the pre-treatment peer review and “morning huddle” were favorable (Table 1).

Table 1: Survey results of pre-treatment peer review (n=27; Source: Adams et al., 2011).
Original PDF Table 1:

7. Helps Clinical Operations Run Smoothly

3 11% 16 59% 8 30%

8. Fosters Communication Within the Department

1 4% 16 59% 10 37%

9. I am satisfied with simulation review. 20 74% 7 26%

WORKPLACE LEVEL High reliability and value creation organizations recognize that the workplace has a profound influence on worker performance. The workplace is broadly defined to be the physical space itself (e.g. desk, telephones, technology etc.), the environment (e.g. temperature and noise level, etc.), as well as the specific processes, workload and situation awareness demands placed on the worker. While this seems obvious, many healthcare workplaces are designed with sub-optimal regard for these concepts. It is particularly important for complex organizations, wherever possible, to clearly define task demands placed on operators and design the workspace to facilitate the anticipated tasks. Below we highlight selected initiatives we spearheaded in our clinic to reduce workload and interruptions.

Workload We observed that high workload and stressors placed on nurses administrating high dose radiation (HDR) procedures was associated with degradation in their procedural performance (Mosaly et al., 2010). We performed a hierarchical task analysis (HTA) to divide the procedures into various tasks, and collected NASA-TLX scores on the identified tasks. A systematic human error reduction and prediction approach (SHERPA) was used to classify potential errors related to high-workload tasks and changes to processes were implemented to reduce workload and stressors (Embrey, 1986). The SHERPA analysis found the following human errors: 1) Information miscommunication (i.e. incorrect vital signs) and 2) inappropriately conducted or missed tasks (i.e., incorrect placement of catheter and rectal tube, etc.). These errors did not result in patient harm but did cause rework and consequentially unnecessary stress and frustration. Major changes were made to standardized procedures and to improve the physical layout of the HDR treatment room. The leadership team worked closely with the nurses over a period of 16 months to change the workflow and treatment room layout. Figure 2 shows the NASA-TLX scores before and after the interventions; p<0.01 (Chera et al., 2012).

Figure 2: Impact of Improvement Initiatives on Workload Levels (Source: Chera et al., 2012) Interruption

Figure available in the original PDF paper.

We observed that radiation therapists working on our linear accelerators were interrupted an average of 4 times per patient treatment. We performed an observational study and learned that interruptions were mostly due to a) patient and staff phone calls to the machine inquiring whether they were “on time”, and b) non-urgent communication between other radiation oncology professionals and the radiation therapist (Mazur et al., 2012). Interventions included a) rerouting of phones, b) placing non-urgent communication between radiation oncology professionals in the electronic medical record system (Mosaiq™, Eleckta, Stockholm, Sweden), and c) institution of public display of the treatment machine status (i.e. on time vs. delayed). The mean number of interruptions was reduced from 4 (range 0-11) to <1 (range 0-3); p < 0.001.

PEOPLE LEVEL Humans are imperfect. Their participation in any endeavor, including the delivery of health care, introduces the possibility of human error stemming from their behaviors and/or decision making processes. On the other hand, our inability to turn all tasks over to robots attests to the unique skills that humans possess. Indeed, for a huge number of societal tasks, particularly those in health care, human participation is required. Human decision making and oversight, and thus ‘safety mindfulness,’ is required to address the needs of patients with complex clinical problems. This certainly applies to patients with cancer where the disease/treatment affects multiple organs, and who have patient-specific concurrent medical conditions, family relationships, and emotional responses to their illness. Below we highlight selected initiatives we spearheaded in our clinic to improve error reporting and problem solving efforts.

Good Catches We implemented a ‘Good Catch’ program to report, analyze and manage events (incidents, adverse events, and near misses). The phrase “Good Catch” (rather than an ‘error’ or ‘near-miss’, etc.) is used to provide a positive annotation and disseminate a “no blame” message for reporting events to the staff. A systematic root-cause analysis is performed to identify: a) where a Good Catch has started/caught in the process; b) number of safety barrier (Quality Assurance (QA) steps) it has crossed; c) the root-cause(s) (or contributing factors) based on the Swiss Cheese Model; and d) the action(s) to be taken for preventing future occurrence and improve the overall process. Employees are encouraged to actively report good catches. Since the start of the program in June 2012, over 450 Good Catches have been reported through December 2013. The results of Good Catch analysis led to process redesign, technological improvements, and new policies and procedures for communication (Mosaly et al., 2013). Some of the key improvements included: 1) improved communication system via electronic medical record (EMR) system (via use of discrete data elements); 2) identified suboptimal safety barriers (quality assurance steps) were modified to be more effective in catching potential events; 3) limited over-ride rights in treatment delivery system to supervisors; 4) implemented additional time-outs, morning-huddles, use of checklists, etc.; and 5) increased awareness and engagement of staff into safety mindfulness.

We also placed a lot of focus on the need to better prepare our employees for quality improvement work using ‘Lean’ thinking and tools. Our front line employees make up the vast majority of the workforce and know their work the best. Therefore, they have the best opportunity to improve it. We operationalized this through Lean training and broad empowerment of employees to engage into Plan-Do-Study-Act (PDSA) cycles for problem solving using an A3 tool. The problem solving tool used by Toyota, termed “A3”, derives its name from the paper size used for the report, which is the metric equivalent to 11”x17” paper. The A3 tool follows the PDSA cycle that the investigator(s) must go through to move from ‘problem faced’ to ‘problem solved’. Overall, 30 A3s have been successfully implemented and at least partially sustained (Taylor et al., 2014; Table 2).

Table 2: Summary of A3 improvements.

Table available in the original PDF paper.

A3 Title Description and Results Installed automatic doors for improved patient transport Automatic Doors

Developed standard protocols for different types of Cyber Knife patients

Cyber Knife Protocols

Standardized operations in recovery room

Recovery Room

Implemented new scheduling phone for simulator

CT Simulator Phone

Improved information flow for patient prescriptions

Quick Rx

Improved process for inpatient to outpatient transition while on treatment

OP-IP Protocols

Developed a standard procedure for consent, creatinine and IVs

Consent and creatinine levels

Improved notification of protocol patients and protocol guidelines

CT Imaging for protocol patients 3Ps Implemented safety barrier to screen for pregnancy, pacemaker, or prior radiation for 100% of patients

Nurse Carts Standardized and implemented Kanban system for restocking nursing cars New Patient Orientation Revamped new patient orientation materials

Clean utility room 5S of clean utility room Pyxis scanner swap Improved location of pyxis and scanner in clinic

Overhead paging Decreased overhead paging by ~70% Problems with Queuing Reduced treatment delays due to unknown patient location by ~50%

Cyber Knife phone Improved communication by installing new phone Charge nurse role Improved utilization of charge nurse role

Dept phone calls Decreased misrouted phone calls by 85% Late RN communication Improved communication of late nurse transition

Improved sterilization safety in utility room

Sterilization for utility room

Improved organization of utility room

Organize utility room

Improved process for inpatient consult requests

IP consult requests

Improve communication between MDs and therapists to reduce omissions

Miscommunication of Sim orders

Implement phase 2 of exam room improvements

Clinic exam rooms 2.0

Improve pre-authorization process to reduce claim denials

Pre-Auth process

Improve safety and efficiency of late treatments

Late Tx patients

Improve skin contours for software

Skin Contours

Increase number of successful deep inspiration breath hold treatments

DIBH process

Improve flow and communication with financial counselor

Financial Counselor

Develop standard process for emergent afterhours treatments

Emergent afterhours treatment

CULTURE OF PATIENT SAFETY The Agency for Healthcare Research and Quality (AHRQ) defines the safety culture of an organization “. . . as the product of individual and group values, attitudes, perceptions, competencies, and patterns of behavior that determine the commitment to, and the style and proficiency of, an organization’s health and safety management” (AHRQ, 2004). AHRQ has an assessment tool that measures the patient safety culture for hospitals, nursing homes, ambulatory outpatient medical offices, and pharmacies. The AHRQ patient safety survey measures 12 dimensions of patient safety culture. The institution routinely administers this survey every 18 months for all outpatient clinics. Survey results from 2009, 2011, and 2013 are compared to examine trends in all dimensions to evaluate the cultural impact associated with the aforementioned initiatives. Figure 3 shows the AHRQ patient safety culture survey results in the department for 2009, 2011 and 2013. Percentages of positive responses in the selected three dimensions (i.e., organizational learning, overall perception of patient safety and quality, and office process and standardization) of patient safety culture appear to have increased from 2009 to 2013 (p<0.01).

Figure 3. Results from AHRQ patient safety culture.
Original PDF Figure 3.

CONCLUSIONS

We have implemented and quantified the impact of several patient safety and safety operational efficiency initiatives in our clinic. Based on our subjective assessments and the quantitative data shown, these initiatives seem to have been, at least in part, successful. Concurrent with these efforts we recognize an apparent improvement in the patient safety culture in our department as measured via an independent AHRQ patient safety culture survey. While cause and effect cannot be established, the timing of the improvements in the patient safety culture survey results does suggest at least some causality.

The presented case study has several limitations. First, the methods used to gather data are imperfect. Surveys, data quantification methods, and subjective assessments of workload may not be ideal means to objectively assess the impact of the initiatives. Second, there were other clinical initiatives and changes to the operation that may have influenced the changing metrics. Third, there is a clear reporting bias. We specifically selected to describe several initiatives that have quantitative data suggesting a positive outcome. Many other initiatives had outcomes that were not readily quantifiable and others had clearly sub-optimal outcomes. Fourth, as the AHRQ surveys are anonymous, it is not know if the same staff participated in the survey over time. Indeed, due to staffing changes, this is almost certainly not the case. Nevertheless, this survey is the best perception-based data available of the department’s overall ‘patient safety culture.’

Our analysis suggests that a multifaceted approach to address patient safety and operations might improve the safety culture in a clinical radiation oncology department. Such effort is often frustrating, time consuming, and requires consistent oversight. Change is hard, and staff are often interested in maintaining existing systems that are familiar and that do not require them to alter their perceptions or actions. Nevertheless, the quantitative data reported suggests that such efforts can be fruitful.

ACKNOWLEDGMENT

We would like to thank all staff that have been involved in our safety and quality activates, in particular the operations team. We thank the faculty in radiation oncology for their patience and participation, and the UNC Health Care System for their financial support.

REFERENCES

Agency for Healthcare Research and Quality (2004). Hospital Survey on Patient Safety Culture, Prepared by Westat, Rockville, MD. Adams R. Marks L. Pawlicki T. Hayman J. Church J. (2011). “The New Radiation Therapy Clinical Practice: The Emerging

Role of Clinical Peer Review for Radiation Therapists and Medical Dosimetrists”, Medical Dosimetry Journal Volume 35 No. 4. Chera, B.S., Mazur, L.M., Jackson, M., Taylor, K., Mosaly, P., Rockwell, J., Adams, R., Chang, S., Deschesne, K., Marks, L.B.

(2013), “Quantification of the Impact of Multifaceted Initiatives Intended to Improve Operational Efficiency and the Safety Culture: A Case Study from an Academic Medical Center Radiation Oncology Department Practical Radiation Oncology”, Practical Radiation Oncology, published online 01 July 2013, in press. Embrey, D. (1986), “SHERPA: A Systematic Human Error Reduction and Prediction Approach”, Proceedings of the International Topical Meeting on Advances in Human Factors in Nuclear Power Systems, Knoxville, Tennessee. Institute of Medicine (1999), “To Err is Human: Building a Safer Health System”, A Report of the Institute of Medicine.

Washington, DC. Marks, L.B., Adams, R.D., Pawlicki, T., Blumberg, AL., Hoopes D., Brundage, M.D., Fraass, B.A. (2013), “Enhancing the Role of Case-oriented Peer Review to Improve Quality and Safety in Radiation Oncology: Executive Summary. Practical Radiation Oncology Volume 3 No. 3. Marks, L.B., Jackson, M., Xie, L., Chang, S.X., Deschesne-Burkhardt, K., Mazur, L.M., Jones, E., Saponaro, P., LaChapelle, D.,

Baynes, D., Adams, R. (2011), “The Challenge of Maximizing Safety in Radiation Oncology. Practical Radiation Oncology Volume 1 No. 1. Mazur, L.M., Mosaly, P., M., Jackson, M., Chang, S.X., Burkhardt, D.K., Adams, R., Jones, E.L., X., J., Rockwell, J., and Marks, L.B. (2012), “Quantitative Assessment of Workload and Stressors in Clinical Radiation Oncology”, International Journal of Radiation Oncology, Biology and Physics Volume 83 No. 5. Mosaly, P., Xu, J., Jackson, M., Mazur, L. (2010), “Assessment of Mental Workload of Nursing Staff in High Dosage Radiation

(HDR) Treatment Room”, In Proceedings of the Institute of Industrial Engineers Annual Conference (IERC) and Expo, Cancun, Mexico. Mosaly, P., Mazur, L.M., Miller, S., Eblan, M., Jones, E. (2013), “Applying Human Factors Analysis and Classification System (HFACS) to the Analysis of ‘Good Catches’ in Radiation Oncology”, American Society for Therapeutic Radiology and Oncology (ASTRO), 55th Annual Conference, Atlanta, GA. Reason, J. (1990), “Human Error”, Cambridge University Press, Cambridge, NY. Reason, J. (2004), “Beyond the Organizational Accident: The Need for ‘Error Wisdom’ on the Frontline”, Quality and Safety in

Health Care Volume 13 No. 1.

Taylor, L.M., Mazur L, Chera, B., and Marks, L. (2014). “Application of “A3 Thinking” to Operational Improvements in Radiation Oncology”, Industrial and Systems Engineering Research Conference, Montreal, Canada, May 2014 (submitted).

Cover of the proceedings volume Advances in Human Aspects of Healthcare

Explore the complete digital volume

Advances in Human Aspects of Healthcare

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-4951-2093-0

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →