Article Text
PDF
Abstract
The Accreditation Council for Graduate Medical Education has shifted to competency-based medical education. This educational framework requires the description of educational outcomes based on the knowledge, skills and behaviors expected of competent trainees. It also requires an assessment program to provide formative feedback to trainees as they progress to competency in each outcome. Critical to the success of a curriculum is its practical implementation. This article describes the development of model curricula for anesthesiology residency training in regional anesthesia and acute pain medicine (core and advanced) using a competency-based framework. We further describe how the curricula were distributed through a shared web-based platform and mobile application.
- education
- pain management
- internship and residency
- regional anesthesia
- acute pain
Statistics from Altmetric.com
Introduction
The last 10 years have seen a dramatic increase in interest in regional anesthesiology and acute pain medicine due to the advent of ultrasound-guided regional anesthesia procedures and the impact of the opioid crisis. Regional anesthesia and acute pain medicine (RAAPM) are now considered essential knowledge and skills for anesthesiologists. The majority of anesthesiology education, including RAAPM education, is delivered through mentored clinical care, coupled with didactic instruction. Many anesthesiology residency training programs have created dedicated resident rotations for regional anesthesia or regional anesthesia rotations combined with acute pain medicine.1
The Accreditation Council for Graduate Medical Education (ACGME) has adopted a competency-based education framework. This framework was used by the ACGME residency review committees to create competency milestones.2 3 The key features of competency-based medical education are specific definitions of learning objectives and assessment to document achievement of competencies.4 These objectives must be translated into a detailed curriculum map that supports deployment by residency programs.5–9
This article summarizes the development of comprehensive competency-based RAAPM curricula for anesthesiology residency training that includes methods to assess the achievement of competency. In addition to the development of these curricula, we also describe the crowd-sourced development of peer-reviewed educational resources to assist programs with implementing the educational experiences called for in the curricula to help learners achieve competency. Finally, we describe the practical implementation of the curricula through the Anesthesia Toolbox, a shared multi-institutional online educational platform and mobile app to support anesthesiology residency training.10
Methods
Curriculum development
To address the absence of organized RAAPM curricula in residency training, in 2014, we convened a panel of experts in RAAPM. The panel was comprised of 17 RAAPM fellowship program directors from 16 different academic anesthesia departments and was chaired by one of the authors (GW) with expertise in curriculum development. The first goal of the committee was to develop a curriculum map. A curriculum map includes5–9:
Summary educational goals.
Specific learning objectives with expected educational outcomes.
Defined educational and training experiences to help learners achieve competency for each learning objective.
Teaching strategies and resources for the educators.
Logical sequencing of the educational and training experiences that are scaffolded on each other.
Methods to determine the achievement of competency for each learning objective.
The committee reviewed the available literature11–16 and information available from the ACGME (competencies and milestones),17 Anesthesiology Residency Review Committee program requirements and American Board of Anesthesiology content outline18 for topics to include in the curriculum. The work by Smith et al 12 published in 2009 served as the foundation for the initial draft of the curriculum. Additional material was added based on the 2010 publication by the American Society of Regional Anesthesia and Pain Medicine and the European Society of Regional Anesthesia and Pain Therapy on recommendations for education and training in ultrasound-guided regional anesthesia.15 Although the 2011 guidelines for fellowship training by the American Society of Regional Anesthesiology and Acute Pain Medicine Fellowship Directors Group16 was directed at fellowship training in RAAPM, it gave insight into topics to be included in the curriculum. Based on this material, a subcommittee of RAAPM fellowship directors and education experts created an initial list of goals and objectives for dedicated core and advanced rotations in regional anesthesia. Using an iterative design process, several in-person and virtual meetings were held with the full committee of 17 RAAPM fellowship program directors and experts until consensus was reached on the goals and learning objectives to include in the curricula. The subcommittee then translated the goals and objectives into educational resources and experiences designed to achieve the goals and objectives. This included specifying the format for the resources to best help learners achieve competency in each learning objective. For example, if the learning objective was to identify and manage local anesthetic systemic toxicity, a simulation might be called for to supplement more traditional didactics like a lecture. The subcommittee also strived to provide different resource formats to support a variety of learning styles, to use newer modes of education delivery (eg, podcasts), and to promote active learning and engagement with the material (eg, problem-based learning discussions (PBLDs), simulation). Next, the subcommittee defined the methods to assess the achievement of competency for each learning objective. The final step in the process was to develop a logical schedule for the educational experiences.
Due to a large number of procedural skills in the RAAPM curricula, key procedural skills were required to have three training elements as educational resources:
A self-guided online module.
A hands-on skills training session script for a simulated environment with a standardized patient to teach patient positioning, equipment setup and sonography for the procedure.
A checklist assessment for documentation of competency in procedure setup and sonography on a standardized patient.
Finally, one of the authors (GW) with expertize in the ACGME anesthesiology milestones mapped each learning objective to the ACGME subcompetencies and milestones. The completed curriculum was then reviewed by the full committee. The curriculum was also distributed to the Regional Anesthesiology and Acute Pain Medicine Fellowship Directors Group for comment. Revisions were made by the committee until consensus was reached on the final curriculum and map.
Results
The final materials produced from the curriculum development process include (1) summary goals and objectives and a description of the scope of the curriculum, (2) a curriculum map which pairs each objective with specific learning experiences to assist learners in achieving competency in the objectives, assessment methods for competency achievement, and the ACGME milestones relevant to the learning objective and educational experiences and (3) A week-by-week schedule designed to organize and scaffold the educational experiences and supporting materials for use during resident rotations.
The RAAPM educational materials were organized into two curricula: a core and an advanced curriculum. The core curriculum is an intensive introductory course intended to prepare trainees for regional anesthesia or acute pain. The expected audience for the core curricula is anesthesia residents during their first dedicated rotation for regional anesthesia. The advanced curriculum is for residents who already have education and experience in the fundamentals of regional anesthesia. To illustrate the structure of the curricula, representative excerpts from the core regional anesthesia goals and objectives, curriculum map and schedule are presented in boxes 1 and 2 and table 1. The full curricula are included as online supplementary material 1 and 2.
Supplemental material
Supplemental material
Regional core curriculum summary of curricular goals and objectives
1. Gain competency in the basic principles of ultrasound-guided regional anesthesia procedures
Describe ultrasound physics and how images and artifacts are generated.
Describe how to use ultrasound machine controls to optimize an ultrasound image.
Setup for a regional anesthesia procedure to optimize procedure performance.
Properly handle the ultrasound probe to obtain an optimal ultrasound image.
Use ultrasound-guidance to efficiently guide a needle to a target with minimal corrections in probe position while maintaining optimal needle visualization.
Describe common pitfalls of ultrasound-guided regional anesthesia and novice behavior.
2. Gain competency in the management of patients presenting for a surgical procedure that can benefit from regional anesthesia for surgical anesthesia and/or postoperative analgesia
Describe local anesthetic pharmacology. Compare and contrast the properties of commonly used local anesthetics.
Describe the indications for regional anesthesia and apply these principles to select appropriate candidates for regional anesthesia with indirect supervision.
Describe the contraindications for regional anesthesia and apply these principles to patient management with conditional independence.
Describe the complications of regional anesthesia including the mechanisms, presentation and management of local anesthetic systemic toxicity (spinal hematoma and peripheral nerve injury are covered in the advanced curriculum).
Identify and manage patients with local anesthetic toxicity with direct supervision.
3. Gain competency in the performance of common regional anesthesia procedures (thoracic epidural analgesia, anesthesia/analgesia of the upper extremity, anesthesia/analgesia of the lower extremity)
Compare and contrast the indications and contraindications of the common regional anesthesia procedures for the upper extremity and lower extremity.
Select the appropriate regional anesthesia procedure based on the surgical procedure, the need for surgical or postoperative analgesia and patient comorbidities.
Describe the relevant anatomy for the common regional anesthesia procedures of the upper and lower extremity.
Obtain appropriate sonographic images and identify the sonoanatomy for the common regional anesthesia procedures of the upper extremity and lower extremity.
Perform common regional anesthesia procedures (thoracic epidural analgesia, anesthesia/analgesia of the upper extremity, anesthesia/analgesia of the lower extremity, in particular, the knee, ankle and foot) with indirect supervision including:
Describe the expectations of a well-performed regional anesthesia procedure from patient workup to performance of the block, to documentation of the procedure and patient outcomes.
Perform a patient assessment and work-up.
Obtain informed consent for a regional anesthesia procedure.
Demonstrate appropriate preparation for the procedure—patient and equipment setup.
Display adequate communication with the patient, staff and supervisors.
Demonstrate aseptic technique.
Demonstrate knowledge of the procedure and anatomy/sonoanatomy.
Demonstrate smooth procedure flow.
Provide management of sedation during the procedure.
Provide appropriate documentation of the regional anesthesia procedure.
Assess the outcomes of the procedure (specific dermatomes or nerves missed, effectiveness of the procedure for anesthesia or analgesia).
Monitor patients for complications.
Appropriately select and perform rescue analgesia in a patient with a failed primary regional anesthesia procedure.
Incorporate feedback, outcomes and literature for performance improvement and learning plans.
4. Gain competency in multimodal analgesia for acute postoperative pain
Compare and contrast different medication options for multimodal analgesia including benefits, indications, contraindications and complications.
Select the appropriate medication regimen for multi-modal analgesia for acute postoperative pain with indirect supervision.
Week 1 schedule from the regional core curriculum
● Self-guided
Videos/online modules
Anatomy and sonoanatomy of the brachial plexus above the clavicle.
Practical considerations for interscalene block.
What does a good block look like.*
Interscalene block and the stoplight.
Self-assessment/quizzes
Week 1 Quiz.
Suggested materials
Introduction to needle guides for US-guided peripheral nerve block: online module.
Week 1 reading list.
How to use the American Society of Regional Anesthesia (ASRA) Coags App: online module.
● Faculty materials
Lectures/problem-based learning discussions
Basic upper extremity anatomy and blocks: lecture
Basic lower extremity blocks: lecture.
Hands-on skills training sessions
Needle guidance with ultrasound.
Sonoanatomy of the brachial plexus above the clavicle.
Assessment tools
Nerve block preparation and setup: objective structured clinical exam.
Interscalene block: checklist.
Supraclavicular block: checklist.
Needle skills under ultrasound: checklist.
Regional anesthesia global assessment rating tool.
Entrustable professional activities/procedural skills rating application.
Suggested materials
Complications of peripheral nerve blockade: lecture.
Each weekly schedule is divided into self-guided learning activities for the trainee and facultyresources to assist faculty in conducting educational sessions. The schedule also containssuggested activities that can be utilized if time permits.
*A short video describing peripheralnerve block from start to finish including pre-procedure assessment and development of a plan,patient and equipment positioning, monitoring, communication, sedation, aseptic technique,documentation of the procedure, and follow-up to assess outcomes of the procedure.
Sample extract from the RAAPM core curriculum map
Curricula revision
The initial core and advanced curricula were developed in 2014–2015. Over the subsequent 5 years, the curricula underwent significant and ongoing revisions based on feedback provided by trainees and faculty as well as new insights gained by review of published literature (eg, updates to guidelines, new techniques or therapies, or updates to the evidence-base supporting existing techniques). In addition to informal feedback, trainees could post online comments about the curriculum or content that was then automatically routed to one of the developers of the curricula (GW). The major feedback from users was that the amount of material in the curriculum was more than they had time to consume. This feedback resulted in the organization of materials into two categories—required and suggested. The new suggested category was intended to reduce this time burden while still providing learners a source of reference material and guidance on which additional materials might warrant further review if additional time was available. Further curricula revisions were made based on the publication of the ACGME program requirements and milestones for RAAPM fellowship training in 2017 and 2018.19 20 These publications were directed at fellowship training but were reviewed for insights that might be incorporated into residency training in RAAPM (eg, the diagnosis of peripheral nerve injury or knowledge of intravenous regional anesthesia).
One of the most important revisions to the curriculum was the addition of core and advanced curricula for rotations in acute pain medicine in 2018–2019. Our committee experience was that although almost all programs had acute pain medicine rotations, there was room for improvement in the quality of existing acute pain medicine curricula and didactics. The same process used for the development of the regional anesthesia curricula was used to create the acute pain medicine curricula that emphasize the management of acute pain in an inpatient setting (eg, postoperative management of continuous blocks, multimodal analgesics and opioids). The full acute pain medicine curricula are included as online supplementary material 3 and 4.
Supplemental material
Supplemental material
Content development
The curricula describe the educational experiences designed to help trainees develop competency in each learning objective. A variety of different types of resources were proposed in the curriculum map for self-guided learning and to assist faculty teaching RAAPM:
Lectures—slide sets.
PBLDs—50 min didactic sessions.
Mini PBLDs—15 min of in-room teaching in a PBLD-style format.
Online learning modules—self-guided interactive electronic learning modules.
Podcasts—15–20 min audio presentations.
Case management guides—standardized guidelines for case management.
Hands-on skills training guides—scripts for faculty to conduct skills training sessions with manikins, partial task trainers, low fidelity simulations or standardized patients.
Simulations—detailed scripts for faculty to conduct high or low fidelity simulations, or objective structured clinical exams (OSCE) for both training and competency assessment.
Quizzes—each week includes a 10-question quiz with fully annotated answers and most content includes a short set of ‘check for understanding’ questions.
Reading lists—landmark papers or key current literature.
To develop the resources called for in the curricula, the committee used crowdsourcing of national experts in RAAPM. Faculty from the nearly 70 anesthesiology departments were solicited to collaboratively develop and peer-review educational material. Each participating department was provided with the curricula and asked to solicit faculty to participate as authors. Many faculty members were reluctant to produce content types other than lecture slide sets. To overcome this barrier, all authors were provided with instructions and templates to assist them with content development. Authors were also provided access to a stock photography library, a medical artist and a computer programmer (for the development of online modules). Similar to a journal publication process, all submitted content was peer-reviewed by two faculty with expertize in the content area. The Editor in Chief of the Anesthesia Toolbox was responsible for selecting reviewers with the assistance of subspecialty steering committees and section editors. The authors were provided written comments from the reviewers. The authors were required to submit a response to the reviewer"s comments along with any revisions to content. Once the content was revised, accepted by the reviewers and approved by a section editor—it was published. The peer-review process was intended to ensure that published education material was of high quality and to provide faculty contributing content support for promotion.
The crowdsourcing effort has resulted in the publication of 32 online modules, 21 PBLDs, 32 lecture slide sets, 9 hands-on session guides, 5 podcasts and 13 simulation or OSCE scripts. The material was created by 154 authors/reviewers with the majority of the material relating to regional anesthesia. The acute pain curricula were developed in 2018–2019 and many of the resources called for in the curricula are still under development.
Implementation
From the beginning, the committee endeavored to create curricula and educational resources, as well as a mechanism to distribute the material. The goal was to create a website for trainees and faculty to access curricula and resources and to organize the curricula into ‘clickable’ study guides with links that would make it easy to access the resources relevant to different rotations. Because of variability in local training requirements and organization of resident rotations, the online portal was designed to allow individual programs to upload custom content that would be accessible only by their users and to customize the study guides to best suit local training requirements. The online portal was called the Anesthesia Toolbox and was originally centered around the regional anesthesia study guides and resources.10 The Anesthesia Toolbox consists of a website and mobile app to view the study guides and published educational resources. Since its original implementation, the Anesthesia Toolbox technology platform and the user interface has changed to provide better service and functionality. The original implementation was a simple content management system. Subsequent iterations of the system added some features of a learning management system (ability to track learners), a mobile app, a 3,000-question quiz bank and features designed to create a learning community (eg, the ability to share material, comment, mark material as helpful, blog, etc).
The Anesthesia Toolbox has grown from the original six member institutions to almost 70 as of January 2020. The project is run as a non-profit collaborative administrated out of Oregon Health and Science University. Participating anesthesiology residency programs pay a membership fee to support the content development, review process and technology platform.
Competency assessment
While the ACGME describes the competencies and milestones for trainees, it does not provide methods or tools to assess competency. The developed curricula use mixed methods of competency assessment including quiz questions, PBLDs, procedural skills checklists, OSCEs and simulation. Structured assessment using simulation or standardized patients has the advantage of increasing the objectivity of the assessment.21 Formative assessment is accomplished via several methods. All self-guided learning materials, such as podcasts or online modules, include three to five quiz questions at the end to serve as a ‘check for understanding’ of the key learning objectives of the material. All quiz questions include a detailed explanation of the answer to make each question a teachable moment. All explanations include links to reference articles or other Anesthesia Toolbox resources to assist learners that want additional information on the concepts addressed in the quiz question. In addition to quiz questions associated with content, the curricula contain weekly 10-question quizzes: seven of the quiz questions address the topics covered in the curricula that week, two questions address topics in the prior week, and one question is a ‘grab bag’ question. These quizzes serve as a formative assessment and reinforce the weekly content.
Regional anesthesia necessitates competency in a large number of procedures, and competency of actual patient care was assessed with direct observation using a structured direct observation assessment tool. The original curricula included a 7-item direct observation tool for procedures that was a simplification of a 22-item tool developed by Cheung et al.22 Each of the following domains was assessed on a 9-point scale: preoperative patient assessment, preparation for the procedure, communication, asepsis, procedure flow, knowledge of the procedure and anatomy, and level of supervision required. Despite simplification of the tool, faculty compliance with using the tool in the setting of a busy clinical practice during a multi-institutional pilot was low. Reasons for low compliance were multifactorial but included the time needed to document the assessment and that not all faculty routinely provide structured feedback. Despite low compliance with using the tool, some faculty commented that the 7-item tool was beneficial for identifying specific areas for improvement that could be discussed when providing feedback to the trainee.
To address low faculty compliance with documenting and providing feedback, the Anesthesia Toolbox is currently piloting and validating a mobile app-based assessment tool that was designed to only ask the faculty to provide four elements: procedure name, complexity of the procedure, level of ‘trust’ the faculty has in the ability of the trainee to perform the procedure independently and any comments (specifically why they did not feel the trainee could not be entrusted to perform the activity independently, if applicable) (figure 1). The app uses a competency assessment framework termed Entrustable Professional Activities (EPAs). An EPA represents a unit of professional practice or work that integrates multiple competencies.23 24 The use of ‘trust’ is a familiar concept of supervising physicians and lends itself well as a conceptual framework for workplace-based competency assessment.25 Entrustment as a measure of competency is assessed along five levels that reflect how much supervision the trainee required during the performance of the EPA or procedural skill:
Screenshot of the mobile application for faculty to assess procedural competency. Faculty first Pick the procedure out of a list and then initiate an assessment. They fill out three fields—complexity of the procedure, level of trust they have with the trainee for the procedure and comments.
Supervisor did the activity
Direct supervision
Reactive supervision with frequent consultation
Available if needed
Ready for independent practice.26 27
Using a Delphi process to reach consensus, a national committee of education experts, some of whom worked on this curriculum project, defined EPAs for anesthesiology residency training. One of the EPAs addresses the perioperative care of a patient managed with regional anesthesia including perioperative assessment, anesthetic plan, performance of the regional anesthesia procedure, conduct of the anesthetic or sedation, transfer of care and postoperative management. A separate EPA was developed for acute pain management of an inpatient (outside of the operating room). In addition to EPAs, the app is used to assess competency for specific procedural skills (eg, popliteal sciatic peripheral nerve block, popliteal sciatic nerve block using a catheter technique, interscalene peripheral nerve block, etc). The goal of the application is to allow faculty to file assessments with the minimum number of clicks in under 1 min and is currently being pilot tested at four different institutions. The addition of the competency assessment system provided a mechanism to allow residency programs to implement competency assessment tools published in the Anesthesia Toolbox, including procedural skills assessment, EPAs, OSCEs, performance during simulations and milestone assessments.
Discussion
This article summarizes the development of comprehensive competency-based RAAPM curricula for anesthesia residency training that includes methods and tools to assess the achievement of competency. The curricula cover core and advanced 4-week rotations for both regional anesthesiology and acute pain medicine. In addition to the development of the curricula, we also describe the crowd-sourced development of peer-reviewed educational resources required to fulfill the curricula. Furthermore, we describe the implementation of the curricula through an online educational platform to support anesthesiology residency training. Finally, we discuss the competency assessment tools integrated into the curricula and our efforts to improve the utilization of these competency assessment tools.
In addition to the development of curricula for resident rotations on regional anesthesiology and acute pain medicine, the committee is using the same methodology to develop a curriculum for RAAPM fellows. This curriculum includes many of the same topics but in greater depth, introduces more advanced concepts, and expects fellows to achieve greater levels of competency. The fellow curriculum also includes goals for gaining competency in patient safety, quality improvement, literature review and research methods, as well as in creating and managing an RAAPM service.
Despite the success of this project, we have encountered numerous challenges along the way. Feedback from trainees consistently indicated that curricula designed by faculty contain more elements than the trainees can absorb or have time to fully use.28 An effective strategy for addressing these concerns has been the categorization of content into required elements containing the most essential materials and suggested elements that trainees can access if they have additional time. An additional related source of consistent feedback was the limited non-clinical time that trainees have for learning outside the clinical environment. Trainees found it difficult to sit through a 30 min video because of time constraints, no matter the educational value. Based on this feedback, we have since attempted to limit the duration of podcasts, videos and online modules to a maximum of 15–20 min. When they could not be completed in that time frame, the material was divided into sub-sections. We have also added information on the expected time commitment in the introduction of each learning activity to make clear the feasibility of completing the curricula components.
Another challenge was related to the development of the content called for in the curricula. As mentioned earlier, faculty were often reluctant to volunteer for content types with which they had limited experience (eg, online modules, podcasts, OSCEs, and even PBLDs). To overcome this barrier, faculty were often solicited to author topics without identification of the required content format. After volunteering to author the content, the faculty member was provided with extensive information and support which was almost always successful in overcoming faculty reluctance to tackle a new content type. On the other hand, many of the faculty were excited about attempting to engage in a new method of developing content. In addition to the instructions and other author support, the first draft submitted by authors underwent an instructional design review by two editors experienced in instructional design (GW ad CMS). The authors were provided written feedback on how to improve the instructional design of the submitted content. Review of the actual content was addressed during subsequent peer review. This process provided significant opportunities for faculty development, with many of the contributing faculty commenting to the editors and peer reviewers their appreciation for the process and for the opportunity to create different types of content.
From a program implementation viewpoint, we faced some recurring issues or obstacles that required consideration. The first was the ‘not invented here’ syndrome. Some faculty expressed that the materials produced in their institution were of very high quality, and they did not need to access outside resources. This concern was addressed by implementing a rigorous peer-review process and implementing an editorial board that maintained a very high standard for content quality. Another concern was that some programs were reluctant to implement a system that contained prefabricated materials for faculty because the process of content development was an important aspect of faculty development. Access to premade materials might make faculty less likely to undertake content development on their own. However, the counter-argument to this was, ‘Why reinvent the wheel?’ Faculty commonly encounter difficulty securing adequate non-clinical time. The large amount of content development required for this project provides ample opportunities for faculty to develop or peer review content. Likewise, to ensure the ongoing relevance of content, the 3-year cycle for updating content provides additional authorship opportunities.
A major concern for the Anesthesia Toolbox planners was the heterogeneity of different anesthesiology training programs. Despite commonality, individual programs have different resident rotation structures, case mixes, faculty preferences. For example, some programs have a dedicated out-of-operating room regional anesthesia team while others do not. In some programs, the regional anesthesia and acute pain service teams are combined in a single service. Some programs have PGY-2s rotating on a regional anesthesia service, while others have only PGY-3s or 4 s. Additionally, local practice differences result in some programs having an abundance of certain blocks and a paucity of others (eg, paravertebral vs erector spinae plane blocks). These challenges were addressed by creating curricula that could be modified by individual programs. The described curricula were ‘model’ curricula with links to content that could be individually tailored to suit the training needs of each program by altering the sequence of content, moving content from required to suggested or vice versa, deleting links to content, or even uploading custom content. The published educational resources were intended to serve as building blocks that could be organized in a way to best suit the specific needs of individual programs. These individualized curricula and custom content were local to the program that created them and not visible by other programs.
Heterogeneity in clinical experience is another variable that has made planning a structured curriculum challenging. While the developed curricula provide a week-by-week schedule that follows a logical progression and is scaffolded to build on prior presented concepts, clinical exposure is unpredictable and often does not align with this schedule. For example, it would make sense to focus on one of the more common and perhaps simpler blocks to perform during the first week of training followed by more advanced topics or blocks in subsequent weeks. However, a trainee might encounter a more advanced block during their first week on a rotation. Thus, the educational experiences in the schedule may not align with what is happening on the clinical service. In response to these challenges, faculty and trainees may use the educational resources out of order. Select programs may have teams comprised of trainees of different levels (eg, PGY-2, PGY-4, and fellow) which allows them the flexibility to assign block cases to the trainee whose curriculum and competency are most aligned with the clinical case. Moreover, flexibility in the curricula schedules is necessary to adapt to the inherent clinical variability.
Another challenge for the project was whether or not to assign content to learners and track their progress through a curriculum. This capability was introduced in the second iteration of our technology platform. It was anticipated that educators would find this capability very useful and initially it was well received. It soon became apparent, however, that the associated workload on the part of faculty or rotation directors to assign learners content or curricula and to monitor their progress was often prohibitive. Even though the system sent automated reminders to trainees about assignments and presented faculty information on trainee completion of assignments, this feature was eventually discarded as it was used by too few faculty to justify the additional cost associated with the implementation of the technology. Faculty and program directors can still assign quizzes to learners and track their progress, but it is no longer possible to track if a learner has watched a video or listened to a podcast other than by assigning them the quiz associated with the content.
A final challenge is the relative underfunding of collaborative graduate medical education projects. The participation of national or international societies in projects like these would be helpful.29 Despite the lack of funding to support advanced technology, users expect the experience to be equivalent to commercial websites they use every day such as Facebook, Google, and Amazon. We have found that our users are technologically sophisticated and thus have high expectations for the electronic platform’s usability. The project’s original focus was ensuring the publication of high-quality content, but this alone was not sufficient and likely impeded early use by trainees. As a result, usability testing and design are now more highly prioritized.
Any website for residency training is not only competing against the ease of use of commercial websites but is also competing against the vast amount of content freely and readily available on the internet via modern search engines. A trainee will ask, ‘why not just Google it?’ Unfortunately, much of the readily available content on the internet is not peer-reviewed, updated frequently, or evidence based and thus is inappropriate for professional medical education.30 Trainees may not be able to distinguish between online content that is suitable for their education and those that may be inadequate. Instead, training programs may recommend trainees use curated, peer-reviewed sources of content like the Anesthesia Toolbox. In the absence of strong recommendations from program directors and faculty, trainees will likely continue to access other sources of content due to their ready availability through sources like YouTube.31 32 As mentioned above, the participation of national and international societies in creating online curated peer-reviewed resources for resident education could also address this problem.
A limitation of these curricula is they were not developed using a formalized process to reach consensus among experts (eg, a Delphi process). However, the curricula were developed using a large number of experts in RAAPM and education. The curricula have also undergone iterative improvements over 5 years based on informal feedback from the trainees and faculty that have been using the curricula.
Conclusions
In conclusion, we summarize the process to develop a comprehensive competency-based RAAPM curricula for anesthesia residency training. These curricula were distributed through an online educational platform to support anesthesiology residency training. This multiyear project has overcome multiple hurdles and is now used by almost 70 anesthesiology residency programs and is continually incorporating feedback to make additional improvements to the content, platform and competency assessment tools.
References
Footnotes
Twitter @rmaniker, @nelkassabany
Contributors GW: This author helped develop the curricula, write and review this paper. RBM: This author helped develop the curricula, write and review this paper. CMS: This author helped develop the curricula, write and review this paper. RI: This author helped develop the curricula, write and review this paper. NIL: This author helped develop the curricula, write and review this paper. ATM: This author helped develop the curricula, write and review this paper. J-LH: This author helped develop the curricula and review this paper. NME: This author helped develop the curricula and review this paper. KG: This author helped develop the curricula and review this paper. PK: This author helped develop the curricula and review this paper. KV: This author helped develop the curricula and review this paper. TT: This author helped develop the curricula and review this paper. KS: This author helped develop the curricula and review this paper. AM: This author helped develop the curricula and review this paper. MH: This author helped develop the curricula and review this paper. JS: This author helped develop the curricula and review this paper. EHW: This author helped develop the curricula and review this paper. MB: This author helped develop the curricula, write and review this paper.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.