Skip to main content

Bibliometric indicators of impact: Article-level metrics

Article-level metrics

Citation-based and altmetric measures can show  impact of individual research publications.  They can help answer the following questions about individual publications:  

  • how many times was an article cited?  
  • how does it compare to other similar papers? (above/below expected count for its discipline, age or publishing outlet)
  • is it in the top 0.1%, 1% or 5%, etc. in its discipline? (measure of excellence)
  • is it gaining citations at an unusually rapid rate? (measure of early impact)
  • how it it tracking in social media? (possible indication of later citation performance) 
  • what is the geographic distribution of citing papers? (international impact)
  • what is the disciplinary distribution of citing papers? (impact outside own discipline)
  • what is its impact outside scholarly community? (social impact)

Sources of article-level metrics

Web of Science (WoS) is a citation index that provides citation counts to articles in nearly 13,000 peer-reviewed research journals across all disciplines, through its component indices: Science Citation Index (SCI), Social Sciences Citation Index (SSCI) and Art&Humanities Citation Index (A&HCI). University of Pittsburgh subscription allows for searching their content from publications going back to 1945 for SCI and 1992 for SSCI and A&HCI.  The newest addition is Emerging Sources Citation Index (ESCI) with content available form 2015, expanding WoS coverage by additional 2,500 titles of regional importance and from emerging disciplines.

SCOPUS is another good source of citation data (as well as altmetrics data).  It is published by Elsevier and claims to index: 21,000 peer-reviewed journals, including 3,800 full open titles, over 360 trade publications, articles-in-press from over 5,000 international publishers. More than 120,000 books with 10,000 added each year, over 520 book series with subject areas focusing on Social Sciences and Arts & Humanities (from 2003), but also including STM subjects (from 2005).  Book types include monographs, edited volumes, major reference works and graduate level text books. 6.8 million conference papers from over 83,000 worldwide events, including:High energy physics, computer science conferences and workshops, Society meetings including the IEEE, American Chemical Society (ACS), Association for Computing Machinery (ACM), Society of Petroleum Engineers (SPE), The Minerals, Metals & Materials Society (TMS), American Geophysical Union (AGU), European Society of Cardiology (ESC), International Society for Chemotherapy (ISC), American Society for Information Security (ASIS), Japan Society of Mechanical Engineers (JSME), and others. 27 million patents from five patent offices: US Patent & Trademark Office, European Patent Office, Japan Patent Office, World Intellectual Property Organization and UK Intellectual Property Office.

Google Scholar is a good source of citations.  It is especially useful for authors and publications in disciplines less well covered by the commercial services (listed above), but it can inflate citation counts because it indexes varied content, including college reading lists, library bibliographies, etc.  For authors with Google Scholar profiles h-index values are also available.  An author search in Google Scholar will produce a list of publications with citations as well as link to Google Scholar Profile (if available).   

Citation-based article-level indicators

Citation Counts:

Measures of Excellence: In Web of Science platform, you can see article-level metrics in the "Citation Network" panel (see the screenshot on the left).  You note that this article has been cited 40 times (citation rate).  Additionally, WoS provides information showing how the citation rate (in this case, 40) looks in relation to similar publications (benchmark).  If the publication places in the top 1% of similar publications (defined by subject area and age), it is given a Highly Cited status (an indicator of excellence).  For exceptionally highly cited papers published within last two years, a Hot Paper status is awarded (a good indicator of early impact).  

In SCOPUS you will see, in the Metrics window, number of citations received to date, together with percentile position in relation to similar publications (subject area and publication date).  Field-weighted citation impact shows how the citation rate compares to the average (baseline) for the similar publication. In the example on the left and below, shows publication with 66 citations which places it in the 99th percentile of the distribution of all similar publications and shows its field-weighted impact as being 108 times above the average for similar publications.    

Benchmarks: If your publication does not have Highly Cited or Hot Paper badge, you can still benchmark its performance against similar documents.  Pitt's subscription to Web of Science, provides access to Essential Science Indicators (ESI) allowing you to understand baselines and citation distributions across across 22 subject categories for papers published in the last ten years.  The screenshots below show ESI baseline and percentile tables (updated for July 2015).  These tables are updated six times a year.  Access, up to date tables here.  Also, you can view a short tutorial on how to read and interpret the baselines tables here

While SCOPUS does not publish baselines or percentiles tables, each publication indexed in SCOPUS shows citation count, citation benchmarking and field-weighted citation impact.

Characteristics of Citing Articles: Often interesting information about the impact of a publication can be gleaned by analyzing characteristics of citing papers.  In Web of Science, from the bibliographic detail page, you can follow the "Times Cited" link to generate a list of citing documents.  Then,  run "Analyze Results" function for a wealth of information about publications citing your paper.  You can understand geographic impact (countries, institutions and individuals citing your research), disciplinary impact (disciplinary mapping of papers citing your research) or range and impact of citing journals.  View a short tutorial below on how to use "Analyze Results" function in Web of Science.

In SCOPUS, from article bibliographic detail page, follow the link to "view all citing documents".  From there,  "Analyze Search Results" function will provide you distributions of citing: authors, institutions, countries, subject areas and source titles.  See a short tutorial on how to use this function in SCOPUS 

   

 

Non-citation based article-level indicators

Non-citation based indicators of impact (atlmetrics) can be grouped in a number of ways.  We can analyse the usage of the scholarly content measured by views and downloads (form a publisher website or a scholarly networking site) and captures (into a reference management tool).  Other attention can be measured by counting blog mentions or tweets or attention in press.  Below see the screenshot from SCOPUS which tracks non-citation based measures of impact at article-level.

For more information and sources of altmetric data see our Altmetrics Research Guide