Abstract
Background & Objective: Error Board Reporting (EBR) is a comprehensive approach for quickly reporting and analyzing errors that occur during surgical operations. This study assesses the influence of EBR on student error frequency and proficiency during the COVID-19 pandemic, with the goal of enhancing clinical teaching.
Material & Methods: This research used an experimental design that included a pretest and posttest to investigate the impact of the intervention on two distinct groups: the intervention group receiving EBR training and the control group undergoing conventional training. The participants were randomized to either EBR or conventional training. Both groups had pretests and posttests, with logbook results included as part of the final assessment process. Furthermore, a satisfaction survey was conducted after the study to gauge participant satisfaction levels. The data gathered from this study underwent rigorous statistical analysis employing various tests, including the Shapiro-Wilk test for assessing normality, the independent t-test for comparing across groups, the paired t-test for comparing within-groups, and multiple linear regression analysis.
Results: The study found no statistically significant differences in age, gender, semester, or age between the intervention and control groups. Furthermore, these characteristics did not have a significant influence on the final scores. However, posttest results and scores from post-practicum logbooks demonstrated significant differences between the two groups (p < 0.001 and p = 0.002, respectively). EBR training significantly improved surgical procedural mastery skills (p = 0.002) and self-efficacy (p = 0.001). A paired t-test demonstrated a statistically significant difference between the two groups regarding their mean pretest and posttest scores (p < 0.001). EBR decreased student errors 86% of the time, effectively promoted a positive spirit, improved critical thinking, and strengthened critical thinking skills (76%).
Conclusion: Incorporating EBR into educational practices can minimize errors and enhance performance during the COVID-19 pandemic, increasing patients' quality of life.
Introduction
Clinical education is a major element of medical and paramedical schools, educating students for varied tasks in the clinical environment (1). It stresses real-world experience and supports active learning to change learners' knowledge and skills (2). In surgical techniques, errors may have enduring consequences, impacting the quality of care and increasing mortality rates. Medical errors are unfortunate incidents that damage patients during treatment, posing a grave risk to their health and well-being (3, 4). Student mistakes are a serious worry for instructors and students, as they can cause severe problems and raise healthcare expenditures. Educators are attempting to address nursing errors by reporting and recording them (5). Consistent reporting increases patient care and avoids future mistakes. However, many clinical errors remain unreported due to worries about legal repercussions, inappropriate behavior, and fear of incompetence (6, 7). Shifting from the perspective that mistakes should never happen to encouraging open communication among staff is critical for promoting improvement and lowering error rates in healthcare services (8).
Self-efficacy is vital for people to overcome problems and enhance their abilities (9). Nurses' self-efficacy is closely connected to their drive to increase their abilities (10). Mastery experiences, obtained via effective training programs and evaluations, contribute to confidence and improved self-efficacy (11). Appropriate assessment approaches promote acquiring these mastery experiences in the clinical setting (12). Clinical education is primarily teacher-centered, depending significantly on students' memorizing abilities (13, 14). To enhance clinical education, it should increase learning levels and eliminate errors (15). During the COVID-19 pandemic, rising demand for healthcare services needs active engagement from educational institutions (16). Addressing falling service quality and medical errors is critical to sustaining a healthy healthcare system (17). In order to fight and win over these challenges, it is necessary to consider the potential of utilizing Error Board Reporting (EBR) as a strong and effective tool.
EBR is a comprehensive and dynamic approach to painstakingly recording and evaluating errors observed during difficult surgical procedures. The fundamental purpose of EBR revolves around elevating the standards of patient safety while simultaneously enhancing the overall quality of care provided (18). By rigorously discovering the underlying reasons behind these errors, an assortment of targeted corrective actions can be strategically implemented for optimal outcomes (19). Moreover, EBR incorporates within itself a fundamental culture that supports openness and responsibility across surgical teams, thereby generating an environment conducive to perpetual learning and continuous growth. Therefore, in the current study, EBR was employed to measure student competence by self-report throughout the COVID-19 pandemic. This study aims to investigate the impact of "error board reporting" on student performance in the surgical setting during the COVID-19 pandemic. The hypothesis is that error board reporting promotes student competency by facilitating learning from mistakes, improving communication and teamwork skills, and lowering stress and anxiety. The study examined and compared outcomes in knowledge, skills, attitudes, and behaviors related to surgical performance.
Material & Methods
Design and setting(s)
This study used a quasi-experimental design with pretest and posttest measures to investigate the influence of EBR on students' ability to identify and report errors in an operating room setting at Nyshabure University of Medical Sciences. The study especially targeted interns who were enrolled in the three-year Bachelor of Science program for Operating Room, which includes a specialized course on operating room skills. This course has a duration of five weeks and consists of 102 instructional hours. The current research consisted of two separate groups: one receiving an intervention and another serving as the control group for comparison purposes. The training duration for participants lasted three months, equivalent to one semester (Figure 1 illustrates key steps within this process).
Figure 1. Flow chart of the study process
Participants and sampling
A statistical analysis using G-Power software was conducted to achieve 88% statistical power (β = 0.12). The large primary outcome effect of 0.90 was estimated with a 2-sided test at α = 0.05, revealing that a minimum of 25 participants were required for each group. For this study, a convenience sampling method was employed to select 53 third-year operating room students who had completed their clinical clerkship. To be eligible for the study, participants needed to express an interest in research and also complete a course related to operating room procedures. Any participants who did not attend an instructional session or failed to finish the second phase of the study were excluded from the analysis. Furthermore, it was mandatory for all students participating to have had immunization, and those who refused vaccination were excluded. Three students who did not complete their clinical clerkship prior to conducting the research were eliminated from further analysis. The researcher recruited participants in the study and then allocated them to either the intervention (EBR) group or the control group using computerized random number generation software, ensuring a fair and unbiased allocation. A pretest assessed routine procedures, tools, scrubbers, and circulator tasks.
Tools/Instruments
To gather data, a questionnaire consisting of three parts was deployed. The questionnaire consisted of demographic information, procedural knowledge, and satisfaction. The design of the procedural knowledge test drew upon observed errors and points collected from operations conducted during the internship period. Both pretest and posttest questions were drawn from the content covered in the clinical clerkship. A combination of scores obtained in the posttest and logbook entries was applied to measure learning results. A maximum score of 50 points may be reached for each component—posttest and logbook record, respectively. Participants also completed a pretest, where they may earn up to 50 points.
To gather demographic data, a questionnaire was developed by the researcher. The development of the questionnaire includes evaluating research publications and consulting with relevant faculty members to guarantee its validity and relevance.
Satisfaction levels were assessed using a Likert scale consisting of three response options: "neutral," "satisfied," and "very satisfied." Alongside demographic information (four items), the questionnaire contained questions regarding general satisfaction (three items) and satisfaction with specific areas such as error recording methods in learning, error reduction strategies, promoting positive attitudes, and critical thinking skills. Each item on the questionnaire was assigned a weight ranging from 1 to 3, depending on its importance. In this study, respondents could obtain scores between 10 and 40, demonstrating their positivity towards learning about surgical errors through evidence-based research methodologies.
The study adopted a test-retest procedure to comprehensively examine the reliability of the instrument. An extensive analysis revealed a Cronbach's alpha coefficient of 0.83, showing good internal consistency. Additionally, the content validity of the study instruments was rigorously confirmed by 10 prominent faculty members, affirming their appropriateness for use in this research endeavor.
Data collection methods
Preparation of EBR
In developing the EBR, faculty members conducted thorough consultations and received valuable input from student representatives. To analyze requirements, they employed focused group methods (one session) and sought expert opinions in accordance with AMEE guidance No. 91 (20). A group of five people, including the faculty members of the operating room department and the training instructors of operating room techniques, formed and investigated the problems and errors of the students. The most important complaint of the students from the point of view of the professors was the difficulty of memorizing drug information, drug calculations, the wrong drug injection, and scrubbing techniques. After careful deliberation, it was concluded that the EBR would be arranged into two major sections: one dedicated to surgical errors and another focused on learning tips. To guarantee its accessibility and utility, the operating room director displayed the EBR prominently in all operating rooms while making it essential for students to complete this board after each surgery diligently. The category of surgical errors comprised errors pertaining to instruments and errors related to scrubbing techniques and circulator duties.
Intervention Phase
Intervention group
This internship was planned and implemented for each group of trainees (groups of 5-7 people) for a period of 102 hours over 5 weeks and 5 days. In order to reflect on their faults, such as drug knowledge, drug calculations, and wrong drug injection and learning points, student rounds were organized within these operating rooms in five sessions weekly. As part of their daily activities, students carried out duties in operating rooms and then participated in a 30-minute debriefing session conducted in the hospital conference hall. The sessions were led by a clinical professor who used a plus-delta model for performing the debriefings. In this model, the clinical professor, referred to as the debriefer, would ask each student questions regarding successful parts of their performance (plus column), areas that required development, and strategies to enhance future performance (21). These sessions adopted a collaborative approach that involved group discussions alongside corrections and feedback from the instructor.
Students in operating rooms participated in peer review and correction of each other's errors under the guidance of an instructor. Instructors highlighted essentials, eliminated negative scores, and provided blind scoring for error reports. In this scenario, blinding was not originally employed throughout the research. However, to ensure fairness and objectivity, the error reports were made visible to all students without disclosing the individuals responsible for the errors. This allowed for anonymized reporting and corrections based solely on the nature of the errors. The reported error patterns were extensively reviewed and validated scientifically. Emphasis was placed on errors that posed higher risks to patients' lives to encourage increased attention from students in these critical cases. Both groups had the same topics and educational objectives.
Figure 2. Flow chart of activities in two approaches. A) An overview of how the EBR was designed and what activities were performed in this approach. B) Describes the activities that were performed in the control group.
Control group
At the commencement of the clinical practice, a division was formed within the control group. This resulted in five separate groups being formed. The traditional training method was employed for this control group. Each individual subgroup visited an operating room, where they were imparted knowledge about different devices and responsibilities pertaining to scrubbing procedures. Furthermore, all students in this cohort had to diligently maintain a logbook to document their activities upon completing each task or activity. A picture of these activities carried out by the control group is presented in Figure 2, Part B.
On the final day of their clerkship, the trainees administered a posttest to evaluate their degree of satisfaction with both EBR and traditional training methods. This assessment was built on the surgical procedures they had watched throughout their internship and the valuable insights they had obtained during this period.
Data analysis
Statistical analysis of the data was done using the program PRISM 7. Shapiro-Wilk tests, histograms, and Q-Q plots were employed to determine whether the data followed a normal distribution. The results of this analysis indicated that all variables had a normal distribution (Table 1). In addition, independent t-tests and chi-square tests were run to check the homogeneity of the groups and to examine significant relationships between the test results. In addition, multiple linear regression was done to assess the effect of each variable on students' final grades. Statistical significance was determined for variables with a p-value of less than 0.05.
Table 1. Analysis of the normality of the dependent variables
Shapiro-Wilk test |
Pretest
control |
Pretest
intervention |
Posttest
control |
Posttest
intervention |
Logbook
control |
Logbook
intervention |
Final score
(control) |
Final score
(intervention) |
W |
0.9530 |
0.9552 |
0.9502 |
0.9464 |
0.9319 |
0.9696 |
0.9667 |
0.9374 |
P value |
0.2926 |
0.3272 |
0.2540 |
0.2077 |
0.0959 |
0.6353 |
0.5624 |
0.1292 |
Passed normality test
(alpha=0.05)? |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Yes |
Note: The tests were conducted using the Prism software, which allowed for precise and detailed analysis.
Abbreviations: W: shapiro-wilk statistic; p: probability
Result
The chi-square test was performed to examine the age of the students, revealing that the intervention and control groups were homogeneous (P >0.999). The mean ages in both groups were 20.75 ± 1.16 and 20.70 ± 1.16, respectively, demonstrating no significant difference (P = 0.93). Table 2 summarizes the demographic characteristics of all participants included in this study. Furthermore, a multiple regression analysis indicated that neither gender, semester, nor age significantly impacted final scores within either group (P = 0.811 and P = 0.773, respectively).
Table 2. Demographic data of the participants
.
Variable |
Intervention group |
Control group |
N (%) |
Mean ± SD. |
N (%) |
Mean ± SD. |
Age (Years) |
. |
. |
. |
. |
19 |
7 (28%) |
20.04 ± 0.84 |
7 (28%) |
20.12 ± 0.88 |
20 |
11 (44%) |
9 (36%) |
21 |
7 (28%) |
9 (36%) |
Gender |
. |
. |
. |
. |
Female |
15 (60%) |
1.600 ± 0.50 |
13 (52%) |
1.520 ± 0.50 |
Male |
10 (40%) |
12 (48%) |
Semester |
. |
. |
. |
. |
4th |
7 (28%) |
2.00 ± 0.76 |
7 (28%) |
2.080 ± 0.81 |
6th |
11 (44%) |
9 (36%) |
8th |
7 (28%) |
9 (36%) |
Note: The data in the table provides a detailed analysis of the mean comparisons between age, gender, and academic semester within two distinct groups.
Abbreviations: N, number of participants; SD, standard deviation
These data are shown in Table 3 as evidence that demographic characteristics did not influence the results obtained from the intervention. In order to determine the students' competencies before and after the intervention, an independent t-test was done.The mean value and standard deviation (± SD) were computed and shown in Table 4.
Table 3. Analyzes the effects of confounding variables on students' final scores (test: multiple linear regression).
Model |
Variable |
Estimate |
Std. error |
t |
Sig. |
Adjusted R squared |
Intervention group |
β0 |
Intercept |
179.6 |
111.5 |
1.611 |
0.1221 |
-0.09300 |
β1 |
B: Age |
-5.807 |
6.260 |
0.9275 |
0.3642 |
β2 |
C: Semester |
6.293 |
6.885 |
0.9140 |
0.3711 |
β3 |
D: Gender |
1.120 |
2.409 |
0.4651 |
0.6467 |
Control group |
β0 |
Intercept |
115.0 |
79.39 |
1.449 |
0.1622 |
-0.08490
|
β1 |
B: Age |
-2.579 |
4.446 |
0.5800 |
0.5681 |
β2 |
C: Semester |
1.773 |
4.792 |
0.3700 |
0.7151 |
β3 |
D: Gender |
0.7014 |
1.729 |
0.4058 |
0.6890 |
Note: (Y = β0 + β1*B + β2*C + β3*D)
Abbreviations: Std, standard deviation; t, t-tests (test statistic); Sig, significance; R, multiple correlation coefficient
Table 4. Compares the mean ± SD of clerkship scores in the intervention and control groups during the pretest, logbook, and posttest.
Groups |
Pretest |
Posttest |
Logbook |
Final score (posttest + logbook) |
Intervention group (Error Board Reporting) |
12.72 ± 12.337 |
42.28 ± 3.021 |
35.24 ± 3.961 |
77.64 ± 5.559 |
Control group (Conventional training) |
12.60 ± 2.00 |
35.56 ± 3.137 |
32.32 ± 2.096 |
67.88 ± 4.014 |
P value |
0.8462 |
< 0.0001 |
0.0021 |
< 0.0001 |
Note: The asterisk signifies statistical significance. P values less than 0.05 are considered statistically significant.
Abbreviation: p, probability
When comparing the control group with the intervention group using a pretest independent t-test, it was found that there was no significant difference in procedural knowledge or mastery level between them (P = 0.846). However, when looking at posttest results and scores from post-practicum logbooks, significant differences were noticed between the two groups (P < 0.001 and P = 0.002, respectively). Specifically, compared to conventional training with logbooks, error board reporting-based training led to significantly improved surgical procedure mastery skills (P = 0.002). In addition, the mean ± SD final scores (posttest scores plus logbook scores) showed a significant benefit of Error Board Reporting training (77.64 ± 5.56 vs. 67.88 ± 4.01) on students' ability to diagnose surgical errors and self-efficacy (Table 4).
A paired t-test was performed to compare the pretest and posttest results in the intervention and control groups. The paired t-test demonstrated a statistically significant difference between the two groups regarding their mean pretest and posttest scores (P < 0.001).
Self-reports within the intervention group revealed that EBR decreased student errors 86% of the time. In addition, 94% of students were satisfied with using this method. The findings suggest that this method was effective in promoting a positive spirit (72%), improving critical thinking (80%), and strengthening critical thinking skills (76%) (Table 5). According to the student satisfaction questionnaire in the control group, the routine method decreased student errors by 43%. At the same time, 37% of the students judged that the routine method was useless. Moreover, this method also revealed unsatisfactory outcomes in promoting a positive spirit and critical thinking (Table 5).
Table 5. Participants' willingness to use EBRs and traditional approaches in surgical units and interest in using these methods in nursing.
Neutral |
Moderately effective |
Effective |
Satisfaction factors |
Control
N (%) |
Intervention
N (%) |
Control
N (%) |
Intervention
N (%) |
Control
N (%) |
Intervention
N (%) |
6 (25%) |
1 (6%) |
8 (32)% |
2 (8%) |
11(43%) |
22 (86) |
Reduce individual error |
9 (37%) |
0% |
10 (38%) |
1 (6%) |
6 (25%) |
24 (94%) |
Satisfaction with the method |
10 (41%) |
2 (8%) |
5 (20%) |
5 (20%) |
10 (39 %) |
18 (72%) |
Raising the spirit of positivity |
16 (62%) |
1 (2%) |
4 (18%) |
4 (18%) |
5 (20%) |
20 (80%) |
Critical thinking |
10 (41%) |
2 (8%) |
8 (30%) |
4 (16%) |
7 (29%) |
19 (76%) |
Increase the spirit of criticism |
Note: Satisfaction is shown as a percentage.
Discussion
The operating room is commonly acknowledged as a high-risk environment within the hospital setting (22). Healthcare organizations aim to minimize nursing errors that might lead to severe patient damage or injury (23, 24). This concern becomes much more pronounced during pandemics such as the existing COVID-19 crisis. Therefore, efforts to eliminate nursing errors are crucial to enhancing patient care (7, 25). To evaluate students' understanding and practical skills, their knowledge was assessed through posttests and logbooks, respectively. Remarkably, the final scores earned by these students also served as an indicator of the effectiveness of the training method employed. Significantly, multiple regression analysis demonstrated that factors like age and gender did not impact students' final scores, thus attributing improvements solely to the intervention. The EBR-trained group demonstrated an improvement in their learning of surgical tips, as revealed by the mean posttest scores (42.28 ± 3.021 vs. 35.56 ± 3.137). This positive outcome may be attributed to the opportunity for group discussions among students within intervention groups consisting of five to seven students each. Previous studies have provided evidence supporting the notion that acknowledging adverse events, extracting valuable lessons from them, and proactively taking steps to prevent future occurrences may successfully limit errors and enhance patient safety (26).
Logbooks measure students' procedural knowledge, reflecting their competence level (27). The EBR method includes recording errors and learning from others' mistakes and has improved students' performance during practical exercises in the operating room. This error-based approach promotes higher self-efficacy and increased confidence, enhancing their ability for accurate assessment (28). Studies have also indicated that virtual learning environments, group activities, and counseling sessions may promote student autonomy and self-efficacy (27). Combining these methods with logbooks can further enhance students' learning and performance in surgical skills.
The efficacy of the EBR method in increasing clerkship training is evident from the contrasting final mean scores observed between the two groups. This shows that employing this methodology has resulted in a more targeted and informative clerkship program for the intervention group.
The student's self-assessment of their performance in the EBR was regarded as extremely acceptable, achieving a respectable 96%. Research done by Ismail Mohammadnejad et al. investigated the reporting patterns of nursing students regarding errors they had committed. The findings revealed that just 44.6% of these students disclosed their mistakes to their instructors and charge nurses. In contrast, a worrisome percentage of just 17.9% admitted 37 errors throughout their internship period. This study also reported considerable gains in critical thinking skills (72%) and critical thinking ability (80%) (29). To enhance the standard of patient care and facilitate a system where errors can be reported without legal repercussions, educational institutions and healthcare organizations must foster an environment that promotes constructive criticism amongst educators and nurses (30). Research done by Sahebalzamani et al. found that managers and doctors are hesitant to report errors owing to concerns about being perceived as incompetent, primarily due to the prevalent culture of attribution of blame and punitive actions (31, 32). Similarly, Khalili's study indicated that students frequently avoid reporting mistakes because they do not receive sufficient support (33). According to Mardani et al., nurses' responses towards errors are deemed insufficient, with a mean score of 4.27, while legal issues pertaining to these errors obtained a higher mean score of 4.64 (34). The investigation into the occurrence and factors behind nursing errors among operating room students discovered that 57.5% of students avoid reporting mistakes due to concerns about uncovering them and their associated legal implications. Additionally, an overwhelming majority of 82.5% lack adequate understanding regarding error reporting, while 55.6% fear negative repercussions on their evaluation grades if they report such incidents (35). Hence, it is imperative to instill in students the understanding that assigning blame is insignificant in the case of an accident. Instead, emphasis should be placed on comprehending the cause and nature of the incident and identifying any weaknesses or vulnerabilities that contributed to its occurrence (35, 36). Consequently, internships must foster an atmosphere where students feel comfortable reporting their mistakes without fearing repercussions (37).
The primary motivation for doing this research was the pressing requirement for public education. Medical personnel must possess the capability to react appropriately under crucial conditions. Such scenarios, where disease outbreaks disrupt service delivery, present a valuable opportunity to identify suitable training programs and paths for learning (38, 39). Moreover, this study suggested that implementing the EBR technique can enhance students' performance when confronted with critical situations.
This research highlighted the difficulty in ensuring that all students reported their errors despite being assured of anonymity. It was observed that the fear of disclosure still influenced their willingness to report. Moreover, while arrangements were made with the hospital management to facilitate the process, the negative influence of certain hospital staff members hindered students' self-esteem and confidence in presenting their errors effectively. Interestingly, it was noted that some students also reported errors noticed by others, which further exacerbated non-cooperation among staff members. Additionally, logistical issues originating from the COVID-19 pandemic caused coordination problems among relevant departments. However, among these challenges, it is worth noting that applying EBR demonstrated promising results in improving student performance an important positive consequence obtained from this educational intervention. Considering these findings, exploring possibilities for designing an electronic EBR system might produce even more significant insights and establish additional connections within this field of study.
Conclusion
The success of the EBR method in increasing students' daily errors and learning progress has been verified by this study. Furthermore, the logbooks have proven that this teaching method significantly improves students' efficiency in completing clinical tasks. Moreover, applying such a technique fosters student satisfaction with their learning experience, cultivates a positive attitude towards education and critical thinking, and enhances their critical thinking capacity. The relevance of this research is particularly evident when considering its potential applications in pandemic scenarios, where comprehensive training becomes indispensable.
Ethical considerations
The research project underwent a comprehensive evaluation and received approval from the Ethics Committee at the national agency for strategic research in medical education (NASR) (Ethics Code: 983086). All participants provided informed consent, with a guarantee that their personal information would remain secret and only aggregated statistics and data would be released in any publications.
Artificial intelligence utilization for article writing
No artificial intelligence has been used in the preparation and production of the scientific content of the article. The use of artificial intelligence, specifically typest.io, was limited to paraphrasing certain sentences within the manuscript.
Acknowledgment
The authors wish to thank the National Agency For Strategic Research in medical education (NASR) of Iran for financial support and Mashhad University Of Medical Sciences (MUMS) for moral cooperation in conducting the study. We are very grateful to the students and colleagues who helped us with this study.
Conflict of interest statement
The authors have no conflicts of interest.
Author contributions
A.G. and M.M.H. performed the design and supervision of the intervention. A.G. conducted the study intervention and collected data. M.M.H. analyzed the data and designed a concept map. Z.M., S.T.M.H., and M.M.H. were involved in the study's design and the manuscript's revision. The manuscript was read and approved by all authors.
Supporting resources
The National Agency For Strategic Research in Medical Education (NASR) provided funding for the study under grant number 983086.
Data availability statement
Upon a reasonable request, the corresponding author can provide the data set that was analyzed during this study.