Article Text
PDF
Abstract
Background/importance There is heterogeneity among the outcomes used in regional anesthesia research.
Objective We aimed to produce a core outcome set for regional anesthesia research.
Methods We conducted a systematic review and Delphi study to develop this core outcome set. A systematic review of the literature from January 2015 to December 2019 was undertaken to generate a long list of potential outcomes to be included in the core outcome set. For each outcome found, the parameters such as the measurement scale, timing and definitions, were compiled. Regional anesthesia experts were then recruited to participate in a three-round electronic modified Delphi process with incremental thresholds to generate a core outcome set. Once the core outcomes were decided, a final Delphi survey and video conference vote was used to reach a consensus on the outcome parameters.
Results Two hundred and six papers were generated following the systematic review, producing a long list of 224 unique outcomes. Twenty-one international regional anesthesia experts participated in the study. Ten core outcomes were selected after three Delphi survey rounds with 13 outcome parameters reaching consensus after a final Delphi survey and video conference.
Conclusions We present the first core outcome set for regional anesthesia derived by international expert consensus. These are proposed not to limit the outcomes examined in future studies, but rather to serve as a minimum core set. If adopted, this may increase the relevance of outcomes being studied, reduce selective reporting bias and increase the availability and suitability of data for meta-analysis in this area.
- REGIONAL ANESTHESIA
- OUTCOMES
- analgesia
Statistics from Altmetric.com
Introduction
Peripheral regional anesthesia, where nerves or clusters of nerves outside the central neuraxia are targeted that supply particular areas of the body, reduces acute pain, opioid consumption and opioid-related side effects. While the use of ultrasound has led to the description of multiple novel techniques1 there remains little consensus on what outcomes are of most importance when these new, or indeed any, blocks are evaluated. Even when an outcome is widely reported, the parameters by which it is recorded such as measurement scale, timings and definitions often vary among studies. Inconsistent outcome and outcome parameter selection limits the robust comparison of data between trials and therefore limits the comparison of techniques.2 While core and extended outcome sets for perioperative trials have recently been published,3 none have yet been agreed on specifically for regional anesthesia.
In 2010 it was noted that the top five most accessed and cited Cochrane reviews in the previous year all reported problems related to outcome selection and heterogeneity in outcome measurement.4 In response to this growing problem, the Core Outcome Measures in Effectiveness Trials (COMET) initiative5 was launched by the Medical Research Council to coordinate and provide a framework to develop core outcome sets (COS) for particular areas of research by systematic review and consensus building.
In this COMET framework, a systematic review is performed to gather potential outcomes followed by a series of Delphi surveys to select out the core outcomes from this long list. This technique, initially developed by Norman Dalkey and Olaf Hemler in the 1960’s,6 promotes anonymity and non-confrontational information sharing of opinions between participants to aid in ‘the gradual formation of considered opinion’. The original Delphi concept was published in the defense industry but has now spread to other areas, including anesthesia research such as ventilation strategy7 and hip fracture care.8
We aimed to develop the first COS for peripheral regional anesthesia trials to represent a minimum standardized set of outcomes to be reported in future regional anesthesia research. These are not intented to limit the range of outcomes to be examined in the future. If adopted, these recommendations may result in more relevant outcomes being measured, reduce selective reporting bias and increase the data available for direct comparison between trials.9
Methods
The methodology of this study adhered to the COMET handbook.5 A steering group was formed consisting of four authors (JH, TA, SW, MC) and the study was registered prospectively on the COMET initiative database, which records planned and completed projects. The study consisted of three components:
Systematic review of the literature to generate a long list of potential outcomes for the COS.
Selection of the outcomes for the COS, determined by a three-round electronic Delphi survey.
Selection of the parameters for each of the core outcomes, such as their time points, methodologies, definitions and measurement scales, determined by a single electronic Delphi survey and consensus meeting.
Systematic review
Data sources and searches
A database search of MEDLINE (via Ovid platform), Embase, Cochrane Library and Web of Knowledge (via Ovid platform) was undertaken for the period of January 1, 2015, to December 31, 2019, following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis.10 The search was limited to the following journals: Anesthesia, Anesthesiology, Anesthesia and Analgesia, British Journal of Anesthesia, European Journal of Anaesthesiology and Regional Anesthesia and Pain Medicine. These journals were selected by the steering group based on their impact factor and regular publication of regional anesthesia articles. A combination of keywords and Medical Subject Headings terms were used to search for English language randomized controlled trials, systematic review and meta-analysis articles studying peripheral regional anesthesia techniques. These were combined with Boolean operators to formulate a final search strategy (online supplemental appendix 1).
Supplemental material
Study selection
Randomized controlled trials, systematic reviews and meta-analyses reported in English were included. Studies involving volunteers, patients younger than 18 years of age, central neuraxial blockade, cadaver studies, cranial nerve blocks or pharmacokinetic studies were excluded. Two authors (JH and TA) used Rayyan systematic review software11 to identify studies meeting these criteria independently. Once completed, screening decisions were unblinded and conflicting decisions re-examined (JH and TA) before agreeing on the included articles. If a conflicting decision could not be resolved, a third author (MC) arbitrated. Full-text papers of selected studies were examined by both reviewers for eligibility before the final decision regarding inclusion.
Data extraction
Study characteristics and both primary and secondary outcomes from all included papers were extracted to a database by two authors (JH and TA). In addition, outcome parameters for each outcome including time points, methodologies, definitions and measurement scales were recorded (online supplemental appendix 2).
Data synthesis
Outcomes (eg, nerve injury) were then categorized by the chronology in the operative pathway at which they were measured (eg, postoperative) and domain (eg, adverse events) (online supplemental appendix 3). After removal of duplicates, a long list of outcomes was generated for consideration by the participants in a Delphi consensus process.
Analysis
After outcome selection, we analyzed the prevalence of the outcomes chosen in the randomized controlled trials included by the systematic review.
Selection of core outcomes
Invited participants
The steering committee invited 25 regional anesthesia experts, comprizing the Regional Anaethesia UK (RA-UK) Board and invited speakers to the planned RA-UK 2020 conference. This convenience sample panel included clinical and academic experts in regional anesthesia from North America, Europe and the UK.
Delphi electronic surveys
Participants were sent the long list of outcomes prior to the first survey and were invited to suggest further outcomes to be considered. Three electronic Delphi surveys were performed remotely to establish progressive consensus on a COS.
Outcomes were presented in a Microsoft Excel (Microsoft, Redmond, USA) spreadsheet format for each Delphi survey. Outcomes were ordered first by chronology during the patient journey (eg, preoperative, intraoperative, postoperative) and then by domain (eg, adverse events). Outcomes exploring similar concepts were therefore adjacent. Where necessary, explanatory material and/or hyperlinks were provided in the spreadsheet to allow participants to access further information on the outcomes. No information was provided to the experts detailing how frequently an outcome was reported in the literature.
Participants were asked to rate the importance of each outcome, considering its validity, reliability, feasibility and contribution to patient-centered care. Participants were asked to rate the outcomes on a Likert 9-point scale as follows:
1–3: limited importance.
4–6: important but not critical.
7–9: critically important.
In each round participants were encouraged to comment on each outcome. These comments were shared among the participants anonymously in subsequent rounds to help build consensus.
Consensus thresholds were prespecified by the steering committee to determine whether an outcome would be eliminated, progress to the second or third rounds, or would be selected as a core outcome (after the third round). There is no accepted standard for consensus thresholds in COS development, with most setting a threshold of greater than 70% of stakeholders rating an outcome as critically important.5 A slightly higher threshold of 75% has been favored by some authors12–14 to increase rigor in the final step and this was agreed by our steering committee for the third round. Lower thresholds were selected for the preceding rounds to allow outcomes to be appraised with the benefit of comments and voting from all participants. The prespecified thresholds were as follows:
To progress to the second round an outcome required <15% of participants scoring it 1–3 and >50% of participants scoring it 7–9 in the first round.
To progress to the third round an outcome required <15% of participants scoring it 1–3 and >70% of participants scoring it 7–9 in the second round.
To be selected as a core outcome after the third round required <15% of participants scoring it 1–3 and >75% of participants scoring it 7–9 in the third round.
After each round, the steering committee analyzed the outcomes’ scores and eliminated outcomes failing to meet the prespecified thresholds. In the second and third rounds, the results of the previous round were available to the participants. Prior to re-scoring the outcomes, participants were able to review the following for each outcome: mean score in the previous round; the proportion of 1–3 (‘limited importance’) and 7–9 (‘critically important’) scores in the previous round; the participant’s own individual scores from the preceding round; and anonymized comments regarding that outcome from the previous round.
After the COS had been agreed on, the prevalence of the selected core outcomes in the contemporary literature was analyzed. The randomized controlled trials extracted by systematic review were examined and a count was made for each time one of the agreed core outcomes had been reported in that literature.
Selection of the parameters for each of the core outcomes
Delphi electronic survey
Participants were then asked to score parameters for each potential outcome. Parameters were grouped into categories such as the measurement scale, timing, and definitions used. For example, for the potential core outcome ‘sensory testing’ of a regional anesthesia technique, multiple outcome parameters were considered; 11 measurement scales, 6 stimuli and 6 time points (online supplemental appendix 4).
Participants rated the outcome parameters on a Likert 9-point scale based on their importance. The steering committee set a threshold to decide which should be eliminated and which outcome parameters should be selected or shortlisted for further consideration at the video conference. To be selected, an outcome parameter had to be rated as 1–3 (‘limited importance’) by <15% of participants and rated as 7–9 by >50% of participants. Where only one outcome parameter met these thresholds, this was automatically accepted as the recommendation without further need for discussion at the video conference. Where more than one outcome parameter met the thresholds a vote in the final video conference was undertaken to decide which should be recommended. Where no outcome parameter had met these thresholds, none was recommended as a standard for that outcome.
Consensus meeting
A video conference consensus meeting was held on October 20, 2021. Participants who were unable to attend this video conference call were offered the chance to submit their comments in advance, which were shared with the participants on the video conference call.
Where multiple parameters for an outcome had reached the Delphi survey consensus thresholds, they were brought forward to the video conference. Each outcome parameter was presented, with the parameter with the highest mean score from the survey being shown first. A 4 min discussion was held, concluding with a vote on the options held in real time. The outcome parameter given the most votes was chosen as the agreed parameter for that core outcome.
Results
Systematic review results
Extraction and synthesis
A total of 206 publications were included (figure 1): 158 randomized controlled trials, 31 meta-analyses, 15 systematic reviews and 2 dose-finding studies (online supplemental appendix 4). In total, 871 outcomes were extracted from these publications. Following removal of duplicate outcomes, a long list of 224 unique outcomes remained for consideration in the Delphi consensus process to produce a COS.
Preferred Reporting Items for Systematic Reviews and Meta-Analyses flowchart of selection of studies in this systematic review.
Selection of core outcomes
Of the 25 experts invited, 21 agreed to participate. No outcomes were added to those gathered during the systematic review by the participants prior to the first round. All 21 participants completed each of the three-Delphi electronic survey rounds, resulting in 10 core outcomes being selected for the COS (figure 2). A summary of the voting results for all the outcomes through each round is found in online supplemental appendix 5. The 10 outcomes chosen, along with the prevalence of these outcomes in the randomized controlled trials generated by the systematic review are displayed in table 1.
Supplemental material
The outcomes included in the core outcome sets and their prevalence in the 158 randomized controlled trials included in the systematic review
Summary of the progressive exclusion of outcomes by the participants to form the core outcome sets. An outcome was eliminated if more than 15% of the experts scored it 1–3. In addition, the outcome had to meet a predetermined % threshold of scores in the 7–9 range as above.
Selection of the parameters for each of the core outcomes
Delphi electronic survey
Outcome parameters, such as the time points, methodologies, definitions and measurement scales, for the 16 remaining potential core outcomes in Round 3 were scored by all 21 participants.
Table 2 shows the number of outcome parameters considered, by category, for each of the 10 core outcomes that met the threshold for inclusion in the COS in Round 3 (online supplemental appendix 6).
Supplemental material
Outcome parameter categories, voting and elimination during the Round 3 survey
Three outcome parameters (table 3) surpassed the thresholds from the survey without alternatives to be considered. These outcome parameters were therefore accepted as the preferred outcome parameter in the COS.
Outcome parameters recommended following the Round 3 survey
No outcome parameter met the prespecified thresholds for the stimulus category of ‘sensory testing of the block’ and so this category was removed from the process and not taken forward to the consensus meeting. Ten outcome parameters had more than one alternative surpass the thresholds from the survey and were therefore discussed and voted on in the conference meeting.
Consensus meeting
Seventeen participants contributed to the consensus meeting. Where more than one outcome parameter exceeded the thresholds from the survey, they were taken forward to be voted on in the video conference. Voting options and results are summarized in online supplemental appendix 7 and table 4.
Supplemental material
The core outcome set, together with their recommended time points, methodologies, definitions and measurement scales as determined by Delphi electronic surveys and the consensus meeting
Discussion
The selection of outcomes when designing randomized controlled trials is crucial in order to compare the efficacy and safety of interventions. The UK Medical Research Council-funded COMET initiative5 provides a framework for developing consensus-driven COS. A COS, when adopted, aims to increase the relevance of outcomes being measured and the quality of data available for meta-analysis, while reducing selective reporting bias.
Previous COS have been shown to be successful in achieving this. An example of this is COMET-RA,15 a COS for rheumatoid arthritis published in 1994. Subsequent literature review demonstrated uptake of the COS in 60%–70% of trials by 200916 and over 80% by 2017,17 with a further review in 201918 showing sustained uptake at this level. Quality indicators in regional anesthesia have been evaluated and reviewed recently,19 20 with calls to create consensus on what defines success in regional anesthesia.21 To our knowledge, this is the first COS in this area. Delphi-based methodology has been used elsewhere in recent regional anesthesia publications examining the nomenclature of regional anesthesia techniques22 and sonographic structures to be identified for basic blocks (the ‘Plan A blocks’).23
Our systematic review highlighted the large number of different outcomes currently being reported in regional anesthesia research. Furthermore, there was significant variability in the time points, methodologies, definitions and measurement scales of these outcomes. Online supplemental appendix 4 illustrates the range of outcome parameters currently being used for sensory testing with 11 different measurement scales, 6 different stimulation methods and a variety of 6 possible time points; making fair meta-analysis of this data challenging. Through this COMET initiative involving 21 international experts, we achieved consensus on 10 core outcomes, across the preoperative, intraoperative, and postoperative periods. It is important to emphasize that a COS is not intended to diminish the contributions of researchers using alternative outcomes or methodologies. Rather, they represent a suggested minimum core set, with any other relevant outcome or parameter used in addition.
For the preoperative period, cold sensory testing at 30 min after block performance was the only agreed outcome for the COS. Motor testing was narrowly eliminated in round three of voting. Many peripheral regional anesthesia techniques aim to provide only sensory blockade and it is therefore a suitable core outcome. However, only 28% of randomized controlled trials in the systematic review reported this. The final meeting discussion considered whether cold testing should occur at 30 min after a block or every 5 min after a block. Concerns were raised by some participants about the burden of testing every 5 min and its relevance beyond trials specifically testing speed of onset; hence the participants opted for 30 min only.
Intraoperatively, unexpected additional interventions such as unplanned general anesthesia and intraoperative opioid use were deemed to be core outcomes. Both are surrogates for the effectiveness of techniques but could have multifactorial causes. Some participants felt that unplanned general anesthesia was perhaps a more specific measure, but ultimately both were deemed to be core outcomes.
Postoperative core outcomes of pain scores, opioid consumption and first pain timing reflect the most prevalent measures recorded in the literature, reported in 75%, 58% and 8% of the randomized controlled trials, respectively. A move to more functional testing was discussed, but it was agreed this was hard to define as an outcome suitable for all regional anesthesia techniques. The timing of pain scores (12 and 24 hours) and opioid consumption (recovery and 24 hours) gained consensus. Concerns were raised in the final meeting about the practicality of recording a pain score at 12 hours, which may occur late in the night. Furthermore, many of the participants expressed a preference for a time point around 6 hours, when most techniques would be expected to be effective. However, these time points had been eliminated in voting from the previous round, when all participants were involved, perhaps due to split voting between multiple different time point options close to 6 hours. Therefore the 12-hour option for pain score ultimately prevailed.
Local anesthetic systemic toxicity (LAST) and nerve injury were the only two adverse events to be included as core outcomes. The signs and symptoms of LAST can be divided into major signs such as reduced conscious level, severe agitation, seizure, bradycardia, conduction block or arrhythmia; and minor symptoms such as perioral tingling and tinnitus. Due to the difficulty in ascribing some minor features of LAST to toxicity, the participants agreed not to tightly define this; rather any major or minor feature deemed to be due to toxicity should be recorded. Participants considered nerve conduction studies and MRI to be impractical in some centers and so defined nerve injury as persistent features at 6 months. The increasing spotlight on quality of recovery,24 and the potential role of regional anesthesia in influencing this, is reflected by the inclusion of the Quality-of-Recovery-1525 questionnaire at 24 hours by the participants.
Limitations
Our study reflects the opinions of a group of anesthetists with a special interest in regional anesthesia without input from allied specialties or, more importantly, from patients. This could be a focus for further work. The systematic review sourced outcomes from work published in major anesthesia journals housed in North America and Europe without outcomes from neuroscience and pain journals being included, over a limited 5-year period, which may have limited the full range of options to be considered for the COS. Although an international group of participants was invited, there was a weighting towards UK-based experts. The pandemic forced a shift in our planned methodology of a face-to-face meeting, and by doing so limited the availability of experts to 17 of 21 clinicians for the final voting process. Particularly in the first round, the list of items to score was long and unrandomized. This may have led to fatigue and less attention being paid to later elements of the survey. Our approach to pre-determined exclusion thresholds meant that some outcome parameters (eg, pain score time points) were excluded in early rounds, when numerous similar options were available in the voting spreadsheets. Providing voting options in early rounds which encompassed a range of time points might have led to these being available and changed the agreed outcome parameters. Due to time limitations, a 4-min period was set for discussion of each outcome parameter votes during the conference meeting. This did not allow time for all experts to contribute to discussions before each vote and, although everyone contributed at some point in the meeting, opinion and voting was perhaps influenced by those who voiced opinions.
Conclusion
Our systematic review revealed a spectrum of 224 unique outcomes being recorded in regional anesthesia research, with additional variations in the measurement scales, time points, definitions, and methodologies used. This study represents the first COS for regional anesthesia in the literature. Further work is required to examine the wider multidisciplinary team and most importantly the patient perspective of important outcomes in this field. Adoption of these outcomes as a COS may improve the relevance of outcomes being measured, reduce selective reporting bias and improve the availability of data for meta-analyses.
Ethics statements
Patient consent for publication
Ethics approval
Not applicable.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @ajrmacfarlane, @elboghdadly, @ashwani_doc, @rosie_hogg, @anilhormis, @EMARIANOMD, @PeterMerjavy, @amit_pawa, @LloydTurbitt, @womackjonathan
Collaborators Collaborators listed as main authors above.
Contributors Study concept, design and conduct: JH, TA, SW, MC. Systematic Review: JH, TA. Data collection: all authors. Manuscript preparation: JH, TA. Manuscript editing: JH, TA, SW, AJRM, KE-B, MC. Manuscript review and approval: all authors. Guarantor: JH.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests EA received grants from the Swiss Academy for Anaesthesia Research, Lausanne, Switzerland; B. Braun Medical AG, Sempach, Switzerland; and the Swiss National Science Foundation to support his clinical research. EA has also received an honorarium from B. Braun Medical AG Switzerland, Sintetica Ltd UK and MSD AG Switzerland. BF declares honoraria from B. Braun, Sintetica, Brain Therapeutics and Hospitality Medovate. ERM chairs the American Society of Anaesthesiologists Committee on Regional Anesthesia and Acute Pain Medicine. He is a former director of the American Society of Regional Anesthesia and Pain Medicine (ASRA). TA, AG, NH, DFJ, AJRM, AP, MPS, LT, SW, JW are members of the Board of Regional Anaesthesia UK (RA-UK). KE-B declares honoraria from GE Healthcare, Ambu, Fisher and Paykel, Edwards Lifesciences. AJRM declares honoraria and/or research funding from Intelligent Ultrasound. PM is a board member of European Diploma in Regional Anaesthesia (EDRA) and declares honoraria from B. Braun Medical and Medovate. AP declares honoraria from GE Healthcare, Butterfly Net, Sintetica UK and Pacira. KR was the President of the British Association of Day Surgery.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.