Use of a national continuing medical education meeting to provide simulation-based training in temporary hemodialysis catheter insertion skills: a pre-test post-test study
Canadian Journal of Kidney Health and Disease volume 1, Article number: 25 (2014)
Simulation-based-mastery-learning (SBML) is an effective method to train nephrology fellows to competently insert temporary, non-tunneled hemodialysis catheters (NTHCs). Previous studies of SBML for NTHC-insertion have been conducted at a local level.
Determine if SBML for NTHC-insertion can be effective when provided at a national continuing medical education (CME) meeting. Describe the correlation of demographic factors, prior experience with NTHC-insertion and procedural self-confidence with simulated performance of the procedure.
Pre-test – post-test study.
2014 Canadian Society of Nephrology annual meeting.
Nephrology fellows, internal medicine residents and medical students.
Participants were surveyed regarding demographics, prior NTHC-insertion experience, procedural self-confidence and attitudes regarding the training they received. NTHC-insertion skills were assessed using a 28-item checklist.
Participants underwent a pre-test of their NTHC-insertion skills at the internal jugular site using a realistic patient simulator and ultrasound machine. Participants then had a training session that included a didactic presentation and 2 hours of deliberate practice using the simulator. On the following day, trainees completed a post-test of their NTHC-insertion skills. All participants were required to meet or exceed a minimum passing score (MPS) previously set at 79%. Trainees who did not reach the MPS were required to perform more deliberate practice until the MPS was achieved.
Twenty-two individuals participated in SBML training. None met or exceeded the MPS at baseline with a median checklist score of 20 (IQR, 7.25 to 21). Seventeen of 22 participants (77%) completed post-testing and improved their scores to a median of 27 (IQR, 26 to 28; p < 0.001). All met or exceeded the MPS on their first attempt. There were no significant correlations between demographics, prior experience or procedural self-confidence with pre-test performance.
Small sample-size and self-selection of participants. Costs could limit the long-term feasibility of providing this type of training at a CME conference.
Despite most participants reporting having previously inserted NTHCs in clinical practice, none met the MPS at baseline; this suggests their prior training may have been inadequate.
L’apprentissage assuré par la simulation est une méthode efficace pour former les résidents en néphrologie à insérer un cathéter d’hémodialyse non tunnellisé. Des études précédentes sur l’apprentissage assuré par la simulation pour l’insertion de cathéters non tunnellisés ont été effectuées à l’échelon local.
Déterminer si l’apprentissage assuré par la simulation pour l’insertion de cathéters non tunnellisés peut être efficace lorsque les possibilités sont offertes lors d’une conférence nationale de formation médicale continue (FMC). Décrire la corrélation entre les facteurs démographiques, les expériences antérieures d’insertion de cathéters non tunnellisés, de même que l’assurance personnelle en matière de simulation de la procédure. Évaluer la perception des apprenants face à l’apprentissage assuré par la simulation dans le cadre d’une conférence nationale de FMC.
Réunion annuelle 2014 de la Société canadienne de néphrologie.
Les résidents en néphrologie et en médecine interne et les étudiants en médecine.
On a effectué un sondage auprès des participants au sujet des caractéristiques démographiques, de leurs expériences antérieures d’insertion de cathéters non tunnellisés, de leur assurance personnelle et leur attitude par rapport à la formation reçue. Les compétences en matière d’insertion de cathéters non tunnellisés ont été évaluées selon une liste de contrôle en 28 points.
Les participants ont subi un prétest de leurs compétences en matière d’insertion de cathéters non tunnellisés dans la veine jugulaire interne, à l’aide d’un simulateur de patient et d’une machine à échographie. Les participants ont ensuite suivi une séance de formation qui comprenait une présentation didactique et deux heures d’exercices sur le simulateur. Le jour suivant, ils ont subi un post-test de leurs compétences. Tous les participants devaient atteindre ou dépasser la note minimale de passage précédemment fixée à 79%. Ceux qui n’ont pas atteint cette note ont dû effectuer des exercices supplémentaires jusqu’à l’atteindre.
Vingt-deux personnes ont participé à la formation sur l’insertion de cathéters d’hémodialyse non tunnellisés. Aucun d’entre eux n’a atteint ou dépassé la note minimale de passage en premier lieu, pour une médiane de 20 (ÉI = écart interquartile, entre 7,25 et 21). Dix-sept des 22 participants (77%) ont terminé le post-test en améliorant leur note, pour une médiane de 27 (ÉI, entre 26 et 28; p < 0,0001). Tous ont atteint ou excédé la note de passage lors de leur premier essai. Il n’existe aucune corrélation significative entre les facteurs démographiques, l’expérience antérieure et l’assurance personnelle, d’une part, et les résultats du prétest, d’autre part. Les participants ont confirmé l’apport de la formation, et qu’elle devrait être intégrée à la formation postdoctorale en néphrologie.
Limites de l’étude
Échantillonnage restreint et autosélection des participants. Le rapport coût-efficacité n’a pas été évalué. Les coûts pourraient limiter la faisabilité à long terme de la prestation de ce type de formation au cours de conférences de FMC.
Bien que plusieurs participants aient rapporté posséder de l’expérience antérieure dans l’insertion de cathéters non tunnellisés en pratique clinique, aucun d’entre eux n’a atteint la note minimale de passage en premier lieu; ceci suggère que leur formation antérieure ait été inadéquate. Il est possible d’offrir des possibilités d’apprentissage assuré par la simulation pour l’insertion de cathéters d’hémodialyse non tunnellisés qui soit efficace dans le contexte d’une conférence nationale de formation médicale continue.
What was known before
Simulation-based-mastery-learning (SBML) is an effective method to train nephrology fellows to insert temporary, non-tunneled hemodialysis catheters (NTHCs). Prior studies of SBML for NTHC-insertion have been conducted at a local level.
What this adds
SBML for NTHC-insertion can be effective when provided in the setting of a continuing medical education (CME) conference.
The ability to competently insert a temporary non-tunneled hemodialysis catheter (NTHC) is a requirement of nephrology training in Canada . Nonetheless, evidence suggests that many Canadian nephrology trainees do not feel adequately trained to perform the procedure . This is important because improper NTHC-insertion is associated with mechanical and infectious complications that are serious and potentially avoidable -.
Simulation-based-mastery-learning (SBML) is a rigorous form of competence-based education in which learners complete pre-testing, deliberate skills practice with feedback and post-testing until they meet or exceed a predetermined minimum passing score (MPS) ,. Learners who do not initially achieve the MPS undergo further deliberate practice and are retested until they achieve the MPS. In mastery learning, practice time may vary between learners but educational outcomes are uniform. Use of this model (and a stringent MPS) assures that all learners demonstrate their competency in a simulated environment before performing the procedure during actual clinical care ,. SBML has been shown to be effective for multiple clinical skills, including NTHC-insertion ,; however, most previous studies of SBML procedural-training have occurred at the level of a single training program and no information is available regarding the use of SBML in a continuing medical education (CME) setting.
The Canadian Society of Nephrology (CSN) annual meeting provides a unique opportunity to gather current and future nephrology fellows for specialized training during its pre-conference educational program. As such, we sought to determine if SBML for internal jugular (IJ) NTHC-insertion would be effective in this CME setting. The specific aims of this study were to: 1) evaluate the feasibility and effectiveness of NTHC-insertion SBML sessions at the CSN annual meeting; 2) describe the correlations of demographic factors, prior experience with NTHC-insertion and procedural self-confidence with simulated performance of the procedure; and 3) assess learners’ perceptions of SBML as part of a national CME conference.
The study was a pre-test – post-test (before-after) design , assessing an NTHC-insertion SBML pre-conference course during the 2014 CSN annual meeting in Vancouver. Participants were Canadian medical trainees who registered for the CSN annual meeting pre-conference educational program or precourse. The precourse took place on April 24 and 25, 2014 in a conference room of the same hotel that hosted the CSN annual meeting. The Northwestern University Institutional Review Board (Chicago, IL, USA) and the Ottawa Hospital Research Ethics Board (Ottawa, ON, Canada) approved the study and all participants provided informed consent.
Study participants provided demographic information including age, gender, year of post-graduate medical training; prior NTHC-insertion clinical experience (number of prior IJ NTHC-insertions) and; a rating of their self-confidence to perform NTHC-insertion competently using a scale of 0-100 (0 = not confident, 100 = extremely confident). Subsequently, all participants underwent an NTHC-insertion clinical skills baseline examination (pre-test). Pre-test clinical skills examinations were conducted using a previously described IJ NTHC-insertion checklist . Over approximately 2 hours, all participants together completed a lecture, video presentation and ultrasound training. Subsequently, groups of 10 or less sequentially underwent approximately 2 hours of deliberate NTHC skills practice using the simulator with directed feedback. Practice sessions involved two participants and one faculty member (JHB, EC, CE, RM, JJP, MS) with expertise in teaching and performing NTHC-insertion at each of the 5 practice-and-testing stations. Participants were scheduled to return the following day for a repeat clinical skills examination (post-test). Both pre- and post-testing was organized in 30-minute increments during which 5 learners were tested simultaneously with each learner being evaluated by one trainer at their own practice-and-testing station. Consistent with the mastery model, participants who did not meet or exceed the MPS would then complete additional deliberate practice and retesting until the MPS was achieved. After post-testing, participants completed a course evaluation questionnaire. A random sample of one-third of pre- and post-test clinical skills examinations were video-recorded and rescored to assess inter-rater agreement between instructors.
We conducted the NTHC SBML course using the CentraLineMan™ (Simulab, Seattle, WA, USA) simulator that provides a realistic representation of the anatomy of the right upper torso and head. This simulator has IJ and subclavian (SC) veins and carotid and SC arteries. In addition, the simulator features an arterial pulse, different coloured venous and arterial ‘blood’, appropriate venous and arterial blood pressures and realistic tissues with self-sealing veins and skin into which needles, dilators and guidewires may be inserted. It is compatible with ultrasound so as to produce highly realistic ultrasound images. A portable ultrasound machine (SonoSite Inc., Chicago, IL, USA) was made available at each practice and testing station. In addition, NTHC kits, sterile gowns, sterile gloves, sterile drapes and sterile CVC-insertion trays (Cardinal Health, LLC., McGaw Park, IL, USA) were available in unopened packaging at each practice and testing station.
To ensure standardization of training and grading of checklist items, faculty instructors (EC, CE, JJP, MS, RM) underwent a four-hour ‘train-the-trainers’ session led by an instructor with extensive NTHC SBML experience (JHB) the day before the trainees’ precourse. During this ‘train-the-trainers’ session, instructors reviewed the SBML curriculum and approaches to providing feedback, and practiced scoring simulated NTHC-insertions using the 28-item checklist.
A 28-item skills checklist was used to complete clinical skills evaluations (Table 1). This was adapted from a previously published NTHC-insertion skills checklist . Each skill or action required for safe IJ NTHC-insertion was listed in order, given equal weight and scored dichotomously (done correctly/done incorrectly). The MPS was previously established at 79% by a multidisciplinary panel . If the SC approach was used, the carotid artery was punctured or more than 2 needle passes (punctures of the skin) occurred, the simulation was terminated and the remaining checklist items were marked incorrect.
To assess inter-rater agreement, video-recorded clinical skills evaluations were rescored by one of two authors (JHB, JJP) who were blinded to pre- or post-testing status and the original learners checklist score.
We evaluated differences between pre- and post-test scores using the Wilcoxon signed-rank test to assess the impact of the intervention. Spearman’s rank correlation coefficients were calculated to assess relationships between demographic factors, prior procedural experience and self-reported procedural-confidence with pre-test scores. The difference between pre-test scores of nephrology fellows versus those of the other trainees was assessed using the Wilcoxon rank-sum test. Inter-rater agreement was assessed by evaluating the percentage of items upon which the two raters agreed. A Kappa coefficient was not calculated because the checklist was previously validated and there was complete agreement on several items. We evaluated differences between the original scores of the video-recorded clinical evaluations with the scores obtained through video review using the Wilcoxon rank-sum test. We performed all statistical analyses using IBM SPSS version 22 (Chicago, IL).
Twenty-two participants (11 nephrology fellows, 10 internal medicine residents and 1 medical student) enrolled in the precourse and completed pre-testing and training. Table 2 reports demographics, NTHC-insertion experience and procedural self-confidence. The median pre-test score was 20 (IQR, 7.25 to 21) checklist items correct out of a possible of 28 with no participants attaining the MPS. Seventeen participants participated in post-testing sessions on Day 2. The median post-test number of correct items increased by 7 (p = 0.001) to 27 (IQR, 26 to 28) with all participants achieving the MPS on their first attempt. Figure 1 depicts participants’ pre- and post-test clinical skills examination performance. The percentage of participants who performed each of the 28 checklist items correctly is detailed in Table 1.
The 5 participants who did not return for post-testing were nephrology fellows who each reported performing at least 20 prior IJ NTHC-insertions in clinical practice. Their median pre-test score of 19 (IQR, 3.5 to 20) was similar to overall group performance. Overall, no significant difference in pre-test scores were observed for nephrology fellows (n = 11) compared with trainees who had less medical training (n = 11): the fellows’ median score was 14 (IQR, 5 to 21) compared to 19 (IQR, 3.5 to 20) for the residents and student (p = 0.44).
Spearman’s rank correlation coefficients demonstrated no significant correlations between pre-test scores and age (r = -0.19; p = 0.40), gender (r = -0.42; p = 0.06), years of post-graduate medical training (r = -0.31; p = 0.89), clinical experience (r = 0.90; p = 0.69) or procedural self-confidence (r = 0.17; p = 0.45).
For the evaluations video-recorded in order to assess inter-rater agreement (n = 13), raters agreed on 331/364 (91%) of the rescored checklist items. The difference between the median of scores obtained by initial grading (22 (IQR, 20.5 to 27.5)) and the median of scores obtained from video review (23 (IQR, 18.5 to 27)) was not significant (p = 0.88).
Of the 17 participants who completed a post-training survey, 14 ‘strongly agreed’ (5/5 on the Likert scale) and 3 ‘agreed’ (4/5 on the Likert scale) that SBML-training improved their ability to competently insert NTHCs. As well, 15 ‘strongly agreed’ and 2 ‘agreed’ that this type of training should be a required component of nephrology fellowship training.
This study shows that SBML can be used to train current and future nephrology fellows in NTHC insertion at a national CME meeting. Consistent with the findings of earlier research , this program significantly improved trainee NTHC-insertion skills. No participant met or exceeded the MPS at baseline, however all met or exceeded the MPS on their first attempt after the educational intervention. Participants strongly endorsed that SBML improved their NTHC-insertion skills and that training of this type should be a required component of nephrology fellowship.
We believe that one of the strengths of this training intervention was the maintenance of a high level of realism during pre-testing, post-testing and deliberate-practice sessions. Anecdotally, many participants commented that they had received ‘partial’ simulation training for NTHC-insertion in the past, typically involving the use of a patient-simulator and ultrasound machine to repeatedly insert the finder needle into the IJ vein. Forcing participants to complete the entire procedure, using all necessary supplies in their original packaging, allowed them to encounter and overcome difficulties they will face in clinical practice but not during less realistic simulations: for example, the challenge of maintaining sterile technique while preparing the cover for the ultrasound probe only becomes evident during highly realistic simulation.
In addition to failing to achieve the MPS at pre-test, most trainees committed potentially serious errors. Notably, over-advancing the guidewire (beyond 20 cm) can provoke serious ventricular arrhythmias, particularly in patients with acute kidney injury . This is a concerning finding as nearly all participants indicated having inserted at least 3 NTHCs in clinical practice before completing the simulation-based intervention. This confirms the findings of a recent survey in which many Canadian nephrology trainees reported inserting NTHCs despite feeling inadequately trained to do so . Taken together, results of this study and others ,, suggest that providing SBML-training for NTHC insertion to current and incoming nephrology fellows at the annual CSN meeting could improve patient-safety. We suggest that training be mandatory because procedural self-confidence and prior experience inserting NTHCs did not correlate with proficiency.
SBML has many potential clinical benefits for Canadian nephrology training programs. SBML for CVC insertion reduces the number of needle passes per insertion ,, arterial punctures , line malpositions , insertion failures  and central line-associated bloodstream infections (CLABSI) . Nonetheless, some experts have suggested that it may be prohibitively time-consuming and labour-intensive to require training, of any type, for NTHC-insertion, across all nephrology training programs . Our findings suggest that a national CME meeting can be an effective venue to train a relatively large group of trainees from multiple centres, thus allowing training resources to be pooled. While there is evidence to suggest that substantial savings might result from reduced rates of CLABSIs ,, additional costs associated with conducting this type of training at a conference include accommodation expenses, flights, shipping costs, and costs related to trainers’ travel time. These additional costs may be mitigated somewhat if a large number of potential trainers and trainees plan to attend a conference for reasons other the training course alone. Companies may also be more willing to donate supplies for training in the context of a national meeting. As such, the longer-term feasibility of providing this type of training to nephrology fellows from across Canada annually would depend upon the results of a formal evaluation of its cost-effectiveness which was beyond the scope of this study. In the case of this study, all supplies and trainers’ time was donated leaving only travel expenses. This limits the generalizability of our findings with respect to providing similar programs at future conferences and in other settings. Training of all Canadian nephrology fellows at a CME meeting might not be necessary as some trainees may be able to receive similar training in a more cost-effective manner at their local training sites. SMBL-training for NTHC-insertion could be particularly helpful for filling in gaps in this training on a national level, particularly for nephrology fellows from centres where SMBL-training for NTHC-insertion may not be offered. One particular advantage of SBML training is that is enables standardization of procedure competence across many types of learners (e.g. nephrology fellows, nephrologists, internists) working in different settings.
This study has several other important limitations. The small sample size and self-selection of participants limit the generalizability of our findings. It is unknown if those who participated are more or less likely to be proficient at NTHC-insertion than their colleagues who did not elect to participate. We did not assess skill retention although this has been reported elsewhere . Additional long-term studies are required to better define the duration for which procedural skills are retained following SBML, particularly when these skills are not used frequently in clinical practice. Use of a CME setting may allow for this important study at future CSN meetings. Finally, 5 of the 22 (23%) participants did not return for scheduled post-tests. The need for participants to devote a relatively large block of time to NTHC-insertion training while other conference events were taking place may be responsible for this finding. Because pretest scores and self-confidence were similar between those who dropped out and those who completed the program, we strongly believe SBML training should be mandatory.
NTHC-insertion SBML during a national CME conference is an effective method to train current and future nephrology fellows. The long-term feasibility of providing this type of training at a national conference on a recurring basis would depend upon a formal evaluation of its cost effectiveness. Most participants reported having significant experience previously inserting NTHCs. However, pretest performance on a clinical skills examination was poor. SBML should be considered for incoming nephrology fellows prior to performing actual NTHC-insertions on patients.
Non-tunneled temporary hemodialysis catheter
Minimum passing score
Continuing medical education
Canadian Society of Nephrology
Central line-associated bloodstream infection
Objectives of Training in the Subspecialties of Adult and Pediatric Nephrology Version 1.0. 2012.
Clark EG, Schachter ME, Palumbo A, Knoll G, Edwards C: Temporary hemodialysis catheter placement by nephrology fellows: implications for nephrology training. Am J Kidney Dis 2013,62(3):474–480. PubMed PMID: 23684144 10.1053/j.ajkd.2013.02.380
Vital signs: central line-associated blood stream infections--United States, 2001, 2008, and 2009 MMWR Morb Mortal Wkly Rep 2011,60(8):243–248. PubMed PMID: 21368740
Clark EG, Barsuk JH: Temporary hemodialysis catheters: recent advances. Kidney Int 2014. Advance online publication, 7 May 2014; PMID: 24805107.
Vats HS: Complications of catheters: tunneled and nontunneled. Adv Chronic Kidney Dis 2012,19(3):188–194. PubMed PMID: 22578679 10.1053/j.ackd.2012.04.004
Block JH: Mastery Learning: Theory and Practice. Holt, Rinehart and Winston, New York, NY; 1971.
McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB: Medical education featuring mastery learning with deliberate practice can lead to better health for individuals and populations. Acad Med 2011,86(11):e8-e9. PubMed PMID: 22030671 10.1097/ACM.0b013e3182308d37
Ahya SN, Barsuk JH, Cohen ER, Tuazon J, McGaghie WC, Wayne DB: Clinical performance and skill retention after simulation-based education for nephrology fellows. Semin Dial 2012,25(4):470–473. PubMed PMID: 22309946, Epub 2012/02/09. eng 10.1111/j.1525-139X.2011.01018.x
Barsuk JH, Ahya SN, Cohen ER, McGaghie WC, Wayne DB: Mastery learning of temporary hemodialysis catheter insertion by nephrology fellows using simulation technology and deliberate practice. Am J Kidney Dis 2009,54(1):70–76. PubMed PMID: 19376620, Epub 2009/04/21. eng 10.1053/j.ajkd.2008.12.041
Shadish WR, Cook TD, Campbell DT: Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin, Boston, MA; 2002.
Fiaccadori E, Gonzi G, Zambrelli P, Tortorella G: Cardiac arrhythmias during central venous catheter procedures in acute renal failure: a prospective study. J Am Soc Nephrol 1996,7(7):1079–1084. PubMed PMID: 8829125
Barsuk JH, McGaghie WC, Cohen ER, Balachandran JS, Wayne DB: Use of simulation-based mastery learning to improve the quality of central venous catheter placement in a medical intensive care unit. J Hosp Med 2009,4(7):397–403. PubMed PMID: 19753568 10.1002/jhm.468
Barsuk JH, McGaghie WC, Cohen ER, O’Leary KJ, Wayne DB: Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med 2009,37(10):2697–2701. PubMed PMID: 19885989 10.1097/CCM.0b013e3181a57bc1
Barsuk JH, Cohen ER, Feinglass J, McGaghie WC, Wayne DB: Use of simulation-based education to reduce catheter-related bloodstream infections. Arch Intern Med 2009,169(15):1420–1423. PubMed PMID: 19667306 10.1001/archinternmed.2009.215
Negoianu D, Berns JS: Should nephrology training programs continue to train fellows in the placement of temporary hemodialysis catheters? Semin Dial 2014,27(3):245–247. PubMed PMID: 24666034 10.1111/sdi.12207
Cohen ER, Feinglass J, Barsuk JH, Barnard C, O’Donnell A, McGaghie WC, Wayne DB: Cost savings from reduced catheter-related bloodstream infection after simulation-based education for residents in a medical intensive care unit. Simul Healthc 2010,5(2):98–102. PubMed PMID: 20389233 10.1097/SIH.0b013e3181bc8304
Oliver SW, Thomson PC: Training can be cost-effective in reducing morbidity associated with temporary hemodialysis catheter insertion. Am J Kidney Dis 2014,63(2):346. PubMed PMID: 24461681 10.1053/j.ajkd.2013.10.060
The authors thank the medical trainees who participated in this program for their dedication to education and high quality patient care.
This study was facilitated through the donation of supplies and use of equipment, provided at no charge, by: Simulab, Seattle, WA, USA; Sonosite Inc., Chicago, IL, USA; Cardinal Health, LLC., McGaw Park, IL, USA.
JHB and DBW’s institution has received educational grants from ‘Medical Error Reduction and Certification, Inc.’, Seattle, WA, USA.
EC conceived of the study, participated in its design, co-ordination and data collection, and drafted the manuscript. DBW conceived of the study and participated in its design, and helped to draft the manuscript. CE, RM, JJP, MS and SH participated in the co-ordination of the study and data collection, and helped draft the manuscript. JHB conceived of the study, participated in its design, co-ordination and data collection, and helped to draft the manuscript. All authors read and approved the final manuscript.
About this article
Cite this article
Clark, E.G., Paparello, J.J., Wayne, D.B. et al. Use of a national continuing medical education meeting to provide simulation-based training in temporary hemodialysis catheter insertion skills: a pre-test post-test study. Can J Kidney Health Dis 1, 25 (2014). https://doi.org/10.1186/s40697-014-0025-6