Structured Feedback Tools in Surgical Training: Results of the Endo-Workshop of the German Society of Residents in Urology 2024

The acquisition of manual and cognitive surgical skills is a fundamental component of urological residency training [1]. However, it is often underrepresented in daily clinical practice due to economic pressures, working time directives, and different operation techniques [2‒6]. The commonly applied paradigm “see one, do one, teach one” is no longer sufficient to effectively impart core surgical competencies while also meeting ethical and economic demands [1, 7]. Innovative tools such as simulation-based training, video training, and mental training are rarely incorporated into training curricula even if they have already been validated to be efficient [1, 8‒12]. The yearly organized Endo-Workshop of the German Society of Residents in Urology e.V. (GeSRU) represents an established extracurricular opportunity for the development and enhancement of manual and cognitive surgical competencies for urologic residents while maintaining patient safety [13, 14]. Specifically, it includes a 2-day simulation-based training for ureterorenoscopy (URS) and transurethral resection of the bladder (TURB) and transurethral resection of the prostate (TURP).

Likewise, although the quality of the provided feedback can positively impact the surgical performance of residents [15‒17], feedback in daily clinical practice is infrequent, subjective, and performed without standardization [18, 19]. To assess surgical skills, especially within simulation trainings, the validated Global Rating Scale (GRS) is widely used [14, 20, 21]. Recently, Biyani et al. [22] described the novel Global Assessment of Urological Endoscopic Skills (GAUES), specifically developed for endourological procedures including cystoscopy, URS, and TURB or TURP. To the best of our knowledge, GAUES has only been used and tested in British trainees. Thus, its transferability to other training programs remains uncertain. Additionally, it is unknown whether the assessment results obtained by GAUES correlate with those evaluated by GRS. Moreover, to our knowledge, no previous studies have directly compared validated assessment tools like GRS and GAUES when actively used as feedback instruments by experienced tutors in urological training.

In the current prospective study, accompanying GeSRU Endo-Workshop 2024, we addressed these knowledge gaps. Based on previous non-urologic publications, we postulated that structured feedback is considered helpful by residents but lacks in daily clinical practice. Additionally, we assumed that the two validated assessment tools, namely, GRS and GAUES may be implemented as structured feedback tools in endourologic training. Finally, we hypothesized the scores obtained through GRS are positively correlated with those obtained through GAUES.

The 2-day GeSRU Endo-Workshop 2024 was conducted at the Olympus Training Center in Hamburg (Germany) in November 2024. The target group of the Endo-Workshop was urologic residents in postgraduate year 1–4 (PGY 1–4). The workshop consisted of four parts. Specifically, it included a theoretical introduction and a one-hour hands-on simulation-based training for stone removal using URS and TURB and TURP. Here, participants had the opportunity to familiarize themselves with the instruments and to practice URS, TURB, and TURP using realistic box trainer models. Following the hands-on simulation-based training, a manual skills assessment was conducted for each procedure, namely, URS, TURB, and TURP, based on specific urological cases. The instruction, supervision, and assessment of participants were carried out by experienced tutors (consultants) in a 1:2 ratio. The assessment included the validated manual skills assessment questionnaires GRS [21] and GAUES [22]. Assessment time was measured and limited to a maximum of 10 min per procedure and per participant. Immediately after each assessment, all tutors provided structured feedback to participants based on results obtained through GRS and GAUES.

The prospective scientific evaluation of our study accompanying GeSRU Endo-Workshop 2024 consisted of three parts. First, participants completed a baseline online questionnaire prior to the workshop. Second, manual skills assessment was completed during the workshop as described above, relying on GRS, GAUES, and time measuring. Third, an evaluation online questionnaire was sent to all participants after the workshop to evaluate the potential of GRS and GAUES as structured feedback tools. The primary study endpoint represented the comparison of the results obtained through GAUES with those obtained through GRS during the assessment of URS, TURB, and TURP. Secondary study endpoints included the need of structured feedback in daily urologic residency training as well as surgical confidence of participants to independently complete URS, TURB, and TURP.

Three analytical steps were completed. First, baseline characteristics of all participants as well as participant’s experience regarding feedback prior to (baseline questionnaire) and after the GeSRU Endo-Workshop (evaluation questionnaire) were tabulated. Subsequently, assessment results of URS, TURB, and TURP obtained through GRS, GAUES, and time measuring were tabulated. Descriptive statistics included means and standard deviations for continuously coded variables and absolute and relative proportions for categorical variables. Second, Box-Whisker plots were used to display item-specific results for GRS and GAUES. Third, scatter plots were used for graphical depiction of the correlation between the total scores obtained through GRS and those obtained through GAUES for each participant. Finally, spearman correlation was used to quantify the association between the total scores obtained through GAUES and those obtained through GRS. All statistical tests were two sided, with a level of significance set at p < 0.05. R software environment was used for statistical computing and graphics (R version 4.3.2; R Foundation for Statistical Computing, Vienna, Austria) [23].

Overall, 15 participants completed GeSRU Endo-Workshop 2024. Due to time constraints, TURP assessment and evaluation online questionnaire after the workshop were only completed by 14 of 15 participants. Of all 15 participants, 8 (53%) were female and 9 (60%) were in 1st to 2nd year of residency (Table 1). Most participants were hired at a university hospital (9 [60%]), worked in full time (13 [87%]), and did not have any experience with endourologic simulation-based training (11 [73%]). The average number of procedures completed as first surgeon before the workshop was 6 (±9) for URS, 9 (±11) for TURB, and 1 (±2) for TURP. Half of the participants reported having a training curriculum in their institution (7 [47%]) and having annually training interviews with their institutional supervisor (8 [53%]).

Feedback was desired by all 15 participants (100%; Table 1). Conversely, only 11 (73%) participants reported receiving feedback in clinical practice. Of these, the majority stated that received feedback was delivered immediately after the procedure (10 [91%]) and in an unstructured verbal form (10 [91%]). Feedback was given in particular by senior physicians (11 [73%]); 5 (33%) participants also stated that they received feedback from nursing staff (Table 1).

In the manual skills assessment, participants needed on average 10 min (±0.1) for the completion of URS task, 7.4 (±1.7) for TURB task, and 9.8 (±0.4) for TURP task (Table 2). Using GRS for manual skills assessment for URS, TURB, and TURP, participants achieved on average a total score of 26.4 (±4.8), 28.3 (±4.0), and 25.0 (±4.6) out of a maximum of 35 points. Relying on GAUES for manual skills assessment for URS, TURB, and TURP, participants achieved on average a total score of 31.9 (±5.1), 36.9 (±4.6), and 32.8 (±3.5) out of a maximum of 58 points. Single item-specific scores of GRS and GAUES can be found in online supplementary Figures 1 and 2 (for all online suppl. material, see https://doi.org/10.1159/000552084). In Spearman correlation analyses, total scores obtained through GRS and GAUES were positively correlated with each other (URS: rho = 0.85, TURB: rho = 0.5, TURP: rho = 0.54; Fig. 1).

In the evaluation questionnaire sent to participants after the GeSRU Endo-Workshop, in each case, 13 (93%) participants rated feedback based on structured assessment tools in general, based on the GRS, or based on the GAUES to be helpful (Table 3). Furthermore, 11 (79%) participants considered GAUES to be specific enough, while 12 (86%) participants stated this for GAUES. The majority of participants considered the implementation of feedback tools, such as GRS (12 [86%]) and GAUES (11 [79%]), in clinical practice to be useful.

At baseline, in each case, 2 (14%) participants felt confident to independently complete URS, TURB, or TURP (Fig. 2a, c, e). After the GeSRU Endo-Workshop, a total of 11 (79%) participants felt confident to independently complete URS, 8 (57%) TURB, and 4 (31%) TURP (Fig. 2b, d, f). Addressing URS, 12 (86%) participants experienced an increase in surgical confidence, 1 (7%) showed no change in surgical confidence, and 1 (7%) experienced a decrease in surgical confidence. Similarly, 13 (93%) participants experienced an increase in surgical confidence, 1 (7%) showed no change in surgical confidence, and no participant experienced a decrease in surgical confidence to independently complete TURB. Finally, 11 (79%) participants experienced an increase in surgical confidence, 3 (21%) showed no change in surgical confidence, and no participant experienced a decrease in surgical confidence to independently complete TURP.

In the present study, we evaluated the need of structured feedback in urologic residency training and examined the simultaneous implementation of two assessment tools, namely, GRS and GAUES, as feedback tool in endourologic surgical training. Moreover, we tested whether total scores obtained through GRS and GAUES correlate with each other regarding surgical skills assessment of urological residents performing URS, TURB, and TURP task. Finally, we further assessed surgical self-confidence of participants to independently perform URS, TURB and TURP.

Firstly, of 15 participants in the study cohort, 7 (47%) were male and 8 (53%) were female (Table 1). Comparing gender demographics in the present study cohort to the GeSRU Endo-Workshop cohort from 2022, a more balanced gender ratio can be observed [14]. The GeSRU Endo-Workshop 2022 was held in conjunction with the annual congress of the German Society of Urology (DGU). The overrepresentation of male residents may be related to differences in conference attendance, as male residents are more likely to participate in scientific meetings and associated workshops [24‒26]. In contrast to that, the GeSRU Endo-Workshop 2024 was conducted as an individual event which may have led to a more balanced gender ratio in the workshop cohort. Most participants (60%, n = 9) were in 1st to 2nd year of residency and were hired at a university hospital (9 [60%]) and did not have any experience with endourologic simulation-based training (4 [73%]) (Table 1).

Second, all participants desired feedback in surgical education (100%). However, only 73% reported receiving such in clinical practice. When receiving feedback, it was mostly delivered immediately after the procedure and in an unstructured verbal form (91%) (Table 1). Similarly, Scott et al. [18] and Waisanen et al. [19] previously reported that feedback in clinical practice is infrequent, subjective and performed without a standardized form [17]. This observation is worrisome, since the quality of provided feedback can positively impact the surgical performance of residents [15‒17]. A reason for the lack of feedback in clinical practice could be a differing perception of resident and supervisor in this regard [27]. Hutul et al. [27] demonstrated that supervisors perceived their own feedback to be high qualitative and frequent. In contrast, only 30% of residents felt the feedback to be helpful. Moreover, supervisors reported giving feedback nearly three times more often than residents reported receiving it. This observation highlights the urgent need for structured feedback tools to objectify and standardize surgical education. Regarding the optimal timeframe for feedback in surgical education, studies showed that immediate structured feedback can lead to a faster acquisition of basic surgical skills when compared to later feedback [15, 28].

Third, in the present study, most urological residents considered the implementation of GRS (86%) and GAUES (79%) as structured assessment and feedback tools in daily clinical practice to be useful (Table 3). GRS and GAUES offer the potential to regularly and easily assess residents’ surgical skills to evaluate their progress but can also serve as free and easily accessible feedback tools. The implementation of structured feedback tools could provide an opportunity to address the previously described lack of structured feedback in daily clinical practice. This lack is often caused by varying perceptions of the frequency and quality of feedback, missing standardization, and the absence of established and comparable tools. Furthermore, our data contributes to the validation of GAUES as a new and task-specific endourological training tool for assessment and feedback in surgical training.

The GeSRU Endo-Workshop 2024 significantly improved self-reported confidence of residents to independently perform the three assessed endourologic procedures, namely, URS, TURB, and TURB. The increase in confidence was particularly pronounced for URS and TURB, where most participants reported feeling more capable after the workshop. The surgical confidence for TURP increased less, which may reflect the higher complexity of this procedure or lower baseline experience (Fig. 2). These results confirm previous findings that show that simulation-based workshops can improve surgical confidence of urological residents in an unlimited training environment while maintaining patient safety [1, 14, 20, 29]. It should be noted that subjective surgical confidence may support motivation and structural learning, but it cannot be directly equated with enhanced manual skills and surgical performance. Nevertheless, including structured feedback in simulation workshops can be an easily accessible approach to increase the efficacy of mostly rare surgical training opportunities. The fact that no participant reported a decrease in surgical confidence emphasizes the importance of opportunities for extra-clinical education in surgical training programs.

Fourth, concerning our primary endpoint, total scores obtained through GRS and GAUES in URS, TURB, and TURP were positively correlated with each other (Fig. 1). Specifically, very high correlation was observed for URS (rho = 0.85) and moderate to high correlation for TURB (rho = 0.5) and TURP (rho = 0.54). While GRS is a common, nonspecific and widely used assessment tool in surgical education [14, 20, 21], GAUES was recently designed for endourological procedures compromising task-specific and global rating skills items for objective and reliable assessment of basic or intermediate endourological skills during simulation [22]. Our data contributes the validation of GAUES in urological education as it can be adequately used to assess surgical skills in endourological procedures, especially URS.

Besides its strength, limitations must be considered. First, the sample size of 15 participants is relatively small, which limits the generalizability of our findings. Future studies with larger cohorts would help validate our findings and further explore the impact of structured feedback tools on surgical education. Additionally, the reliance on subjective data may introduce response bias, especially regarding surgical confidence and the perceived helpfulness of feedback. Also, the box trainers used in this study can only represent the clinical situation closely, as anatomical variations or complications such as bleeding cannot be simulated. Another limitation of our study is the use of GAUES and GRS only in the simulation setting. Yet, we cannot draw any conclusions about its transferability to the real operating room. To our knowledge, the use of GRS and GAUES, both as assessment instruments and, even more so, as structured feedback tools, remains largely limited to simulation-based training settings. Their systematic implementation in real operating room environments is still uncommon. However, recent evidence suggests that structured feedback may be supported in real-world surgical training through targeted interventions, such as automated feedback reminders [30] or smartphone applications for real-time assessments in urology residency programs [31]. Evaluating the feasibility and educational impact of GRS and GAUES for assessment and feedback in daily clinical practice therefore represents an important next step for future research in surgical education. Additionally, the combination auf GRS and GAUES combined with video debriefing could be analyzed further in urological procedures, as Cizmic et al. [32] already showed promising effects in general surgery.

In conclusion, the GeSRU Endo-Workshop 2024 provided a valuable opportunity for urology residents in Germany to enhance their manual skills, receive structured feedback, and improve their surgical confidence. The positive reception of validated assessment tools such as GRS and GAUES used for direct oral feedback suggests that they may serve as feasible instruments for structured feedback. Their use offers the opportunity to provide regular, concrete, and standardized feedback in surgical education. We cannot rely on unstructured verbal feedback, mentoring, and Halsted’s concept of “see one, do one, teach one” because it limits the ability of residents to critically assess and improve their skills. The standardization of surgical assessment and feedback will be crucial in preparing the next generation of surgeons for independent practice within the disparity of expanding socioeconomic requirements and limited training opportunities while maintaining patient safety.

All in all, the standardization of surgical assessments and feedback using validated tools as GRS and GAUES present a significant potential to improve training opportunities. Optimizing surgical education by constantly evaluating assessment tools and improving feedback will be essential to maintain high-quality patient care in future.

The authors would like to thank Olympus for the joint organization of the GeSRU Endo-Workshop 2024. Furthermore, the authors express their gratitude to the active members of GeSRU for the continuous support in the annual organization of the workshop as well as all tutors who supervised the participants during the GeSRU Endo-Workshop 2024.

In advance of the workshop, the participating physicians provided informed written consent to scientific evaluations as well as to anonymized publication of the results of their learning assessments and survey responses. Because all data were collected anonymously and participation was fully voluntary, an ethics application was not required in accordance with national guidelines. The study has been conducted in accordance with the principles set forth in the Helsinki Declaration. We adhere to best practice guidelines on publication ethics. Written informed consent was obtained from all participants to participate in the study.

Margit Fisch was a member of the journal’s Editorial Board at the time of submission. The remaining authors have no conflicts of interest to declare.

This study was not supported by any sponsor or funder.

Julia C. Kaulfuss and Carolin Siech: conceptualization, data curation, investigation, methodology, validation, writing – original draft, and writing – review and editing; Christian P. Meyer: conceptualization, methodology, validation, and writing – review and editing; Maximilian Reimann, Henning Plage, Benedikt Becker, Andreas Beck, Maha Ullmann, Luis A. Kluth, Dejan K. Filipas, and Timm Schäfer: data curation, investigation, and writing – review and editing; Margit Fisch, Thorsten Schlomm, and Felix K.H. Chun: conceptualization, methodology, and writing – review and editing.

All data generated or analyzed during this study are included in this article and its online supplementary material. Further inquiries can be directed to the corresponding author.