Article Text

Download PDFPDF
Is there a correlation between Altmetric Attention Scores and citation count in Regional Anesthesia and Pain Medicine journal articles?
  1. Taras Grosh1,
  2. James Kim1,
  3. Veena Graff1,
  4. Edward R Mariano2,3 and
  5. Nabil M Elkassabany1
  1. 1Anesthesiology & Critical Care, University of Pennsylvania, Philadelphia, Pennsylvania, USA
  2. 2Anesthesiology and Perioperative Care Service, VA Palo Alto Health Care System, Palo Alto, California, USA
  3. 3Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Stanford, California, USA
  1. Correspondence to Dr Nabil M Elkassabany, Anesthesiology & Critical Care, University of Pennsylvania, Philadelphia, PA 19104, USA; nabil.elkassabany{at}uphs.upenn.edu

Statistics from Altmetric.com

Introduction

An article’s scientific impact is traditionally measured by the number of citations it receives in peer-reviewed journal. There is an inherent lag between publication and citation, which is estimated to be 2 years.1 Digitalization of journals and social networks facilitate rapid dissemination of new publications with highly tweeted articles being 11 times more likely to be cited later.2 Many journals, including Regional Anesthesia and Pain Medicine (RAPM), have social media editors dedicated to this effort (https://rapm.bmj.com/pages/editorial-board/). The Altmetric Attention Score (AAS) is an alternative bibliometric based on a proprietary algorithm3 that aims to capture an article’s immediate reach and estimate potential impact much earlier than traditional citations. AAS searches over 15 different platforms including social media, blogs, Mendeley, mainstream media and policy documents.4

Although AAS has not been shown to have strong correlation to citations in the general anesthesiology-related literature, RAPM conferences are associated with higher impressions than other subspecialties.5 Therefore, we designed this study to test the hypothesis that RAPM articles with the highest citation counts would be associated with high AAS.

Methods

This project was deemed exempt from the institutional review board based on the use of publicly available data. The AAS and number of citations were obtained from Dimensions, Digital Science’s digital research database, available through the Altmetric website (www.altmetric.com) for the top 100 cited articles in RAPM between June 1, 2017 and June 1, 2019. For each of the top 100 cited articles, the AAS was recorded and plotted. The website was accessed on July 1, 2021. This time period was selected to allow for the 2-year lag time between publication and citation of a given article.1 The number of studies included in the analysis was chosen based on a previous study that examined the correlation between the two study variables.3 Dimensions research database extracts citations for an article either from existing databases (such as CrossRef, PubMed Central or Open Citations data) or from the full text of manuscripts provided by the content publisher, and reports the number of citations per article. Stata software package, V.15, was used for statistical analysis. A Pearson correlation test was used to assess the relationship between the study variables.

Results

Overall, 325 articles were published in RAPM between June 1, 2017 and June 1, 2019; we included the top 100 cited articles in Dimensions. Bibliometric scores are reported in median (IQR). Scores were 23 (17–34) for the number of citations and 11 (5–22) for the AAS. The Pearson correlation coefficient was 0.3 (p=0.002). The correlation between the two variables is depicted in figure 1. A subset analysis was performed with the outliers removed. The resulting Pearson correlation coefficient was 0.4 (p<0.001).

Figure 1

Scatter plot for number of citation and Altmetric Scores for the top 100 cited articles in Regional Anesthesia and Pain Medicine (RAPM) between June 1, 2017 and June 1, 2019. Correlation between Altmetric Attention Score and citation count in RAPM. Each dot represent a manuscript. The line of best fit is a linear regression line based on least squares technique.

Discussion

There was a weak, but statistically significant, positive correlation between the number of citations and AAS for the top 100 cited articles in RAPM in the study period that persisted even when outliers were removed. Similar results have been shown in the general anesthesiology-related literature3 and other health science fields from various databases.6 One explanation may be that higher AAS represents the ‘hype factor’ of an article rather than the quality of its contents. Traditional citation-based metrics and readership counts may be better measures of research quality.7 Alternatively, we speculate that the weak correlation may reflect the lack of active management of AAS monitoring and/or the limitation of our study to only 2 years post publication. For example, a high AAS signifies social or mainstream media attention, but this may not result in citations if the full-text article is blocked by a paywall. When an article’s AAS rapidly escalates after publication, a journal may consider making it open access for a fixed period or permanently, issuing a press release, and/or diversifying modes of social media promotion (eg, podcast, infographic, posts on multiple platforms). In conclusion, despite failing to find a strong correlation between AAS and citations for RAPM articles, this result may represent an opportunity to improve the monitoring of alternative bibliometrics and develop an intentional strategy for the journal.

Ethics statements

Patient consent for publication

Ethics approval

Not applicable.

References

Footnotes

  • Twitter @dr_tgro, @James_kim_md, @veenagraffmd, @EMARIANOMD, @nelkassabany

  • Contributors TG did the data collection, organized the data, and contributed to the study design, writing the manuscript. He edited the manuscript and approved it. JK and VG contributed to the study design, writing the manuscript. He edited the manuscript and approved it. ERM and NME contributed to the study design, data analysis, writing the manuscript. He edited the manuscript and approved it.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.