Category: Scholarly Metrics

"Same Question, Different World: Replicating an Open Access Research Impact Study"

Julie Arendt, Bettina Peacemaker, and Hillary Miller have published "Same Question, Different World: Replicating an Open Access Research Impact Study" in College & Research Libraries.

Here's an excerpt:

To examine changes in the open access landscape over time, this study partially replicated Kristin Antelman's 2004 study of open access citation advantage. Results indicated open access articles still have a citation advantage. For three of the four disciplines examined, the most common sites hosting freely available articles were independent sites, such as academic social networks or article sharing sites. For the same three disciplines, more than 70% of the open access copies were publishers' PDFs. The major difference from Antelman's is the increase in the number of freely available articles that appear to be in violation of publisher policies.

Research Data Curation Bibliography, Version 9 | Digital Curation and Digital Preservation Works | Open Access Works | Digital Scholarship | Digital Scholarship Sitemap

"From Closed to Open Access: A Case Study of Flipped Journals"

Fakhri Momeni et al. have self-archived "From Closed to Open Access: A Case Study of Flipped Journals."

Here's an excerpt:

The aim of this paper is to show changes in the number of published articles and citations after the flipping of a journal. We analysed a set of 171 journals in the Web of Science (WoS) which flipped to open access. In addition to comparing the number of articles, average relative citation (ARC) and normalized impact factor (IF) are applied, respectively, as bibliometric indicators at the article and journal level, to trace the transformation of flipped journals covered. Our results show that flipping mostly has had positive effects on journal's IF. But it has had no obvious citation advantage for the articles. We also see a decline in the number of published articles after flipping. We can conclude that flipping to open access can improve the performance of journals, despite decreasing the tendency of authors to submit their articles and no better citation advantages for articles.

Research Data Curation Bibliography, Version 9 | Digital Curation and Digital Preservation Works | Open Access Works | Digital Scholarship | Digital Scholarship Sitemap

"Bringing Citations and Usage Metrics Together to Make Data Count"

Helena Cousijn et al. have published "Bringing Citations and Usage Metrics Together to Make Data Count" in Data Science Journal.

Here's an excerpt:

Over the last years, many organizations have been working on infrastructure to facilitate sharing and reuse of research data. This means that researchers now have ways of making their data available, but not necessarily incentives to do so. Several Research Data Alliance (RDA) working groups have been working on ways to start measuring activities around research data to provide input for new Data Level Metrics (DLMs). These DLMs are a critical step towards providing researchers with credit for their work. In this paper, we describe the outcomes of the work of the Scholarly Link Exchange (Scholix) working group and the Data Usage Metrics working group. The Scholix working group developed a framework that allows organizations to expose and discover links between articles and datasets, thereby providing an indication of data citations. The Data Usage Metrics group works on a standard for the measurement and display of Data Usage Metrics. Here we explain how publishers and data repositories can contribute to and benefit from these initiatives. Together, these contributions feed into several hubs that enable data repositories to start displaying DLMs. Once these DLMs are available, researchers are in a better position to make their data count and be rewarded for their work.

Research Data Curation Bibliography, Version 9 | Digital Curation and Digital Preservation Works | Open Access Works | Digital Scholarship | Digital Scholarship Sitemap

"The Relationship between Usage and Citations in an Open Access Mega Journal"

Barbara McGillivray and Mathias Astell have self-archived "The Relationship between Usage and Citations in an Open Access Mega Journal."

Here's an excerpt:

How does usage of an article relate to the number of citations it accrues? Does the timeframe in which an article is used (and how much that article is used) have an effect on when and how much that article is cited? What role does an article's subject area play in the relationship between usage and citations? This paper aims to answer these questions through an observational study of usage and citation data collected about a multidisciplinary, open access mega journal, Scientific Reports. We find that while the direct correlation between usage and citations is only moderate at best, the relationship between how early and how much an article is used and how early it is cited is much clearer. What is more, we find that when an article is cited earlier it is also cited more often, leading to the assertion that if an article is more highly accessed early on, it is more likely to be cited earlier and more often.

Research Data Curation Bibliography, Version 9 | Digital Curation and Digital Preservation Works | Open Access Works | Digital Scholarship | Digital Scholarship Sitemap

"BISG Releases Draft White Paper on Open Access Ebook Usage"

The Book Industry Study Group has released Building a Trusted Framework for Coordinating OA Monograph Usage Data.

Here's an excerpt from the announcement:

This funded project is designed to identify the challenges in understanding the usage of OA ebooks, suggest some opportunities for resolving them, and create a framework for future action through community consultation. It focuses on the challenges of identifying and aggregating relevant information from different platforms, analyzing what has been gathered in ways that respect user privacy, and communicating relevant information about usage to stakeholders.

Academic Library as Scholarly Publisher Bibliography | Digital Curation and Digital Preservation Works | Open Access Works | Digital Scholarship | Digital Scholarship Sitemap

"How Can Bibliometric and Altmetric Suppliers Improve? Messages from the End-User Community"

Elizabeth Gadd and Ian Rowlands have published "How Can Bibliometric and Altmetric Suppliers Improve? Messages from the End-User Community" in Insights.

Here's an excerpt:

This article reports on a 2018 survey of bibliometric and altmetric practitioners—'Three things you want your metrics supplier to know'—that was undertaken to better understand the practitioners' usage of existing tools and services and to invite them to suggest ways in which they would like to see these improve. In total, 149 suggestions were made by 42 respondents, mainly UK librarians. Responses could be categorized into four main themes: A) Improve and share your data; B) Be more responsible; C) Improve your tools; D) Improve your indicators. The findings of the survey are discussed and sample comments shared. Based on these findings, and expanding on the four themes, the article makes a number of practical recommendations to metrics suppliers for ways in which their services could better serve the need of the community for robust and responsible bibliometric and altmetric evaluation.

Academic Library as Scholarly Publisher Bibliography | Digital Curation and Digital Preservation Works | Open Access Works | Digital Scholarship | Digital Scholarship Sitemap

"Over-Optimization of Academic Publishing Metrics: Observing Goodhart’s Law in Action"

Michael Fire and Carlos Guestrin have self-archived "Over-Optimization of Academic Publishing Metrics: Observing Goodhart's Law in Action."

Here's an excerpt:

In this study, we analyzed over 120 million papers to examine how the academic publishing world has evolved over the last century. Our study shows that the validity of citation-based measures is being compromised and their usefulness is lessening. In particular, the number of publications has ceased to be a good metric as a result of longer author lists, shorter papers, and surging publication numbers. Citation-based metrics, such citation number and h-index, are likewise affected by the flood of papers, self-citations, and lengthy reference lists. Measures such as a journal's impact factor have also ceased to be good metrics due to the soaring numbers of papers that are published in top journals, particularly from the same pool of authors.

Academic Library as Scholarly Publisher Bibliography | Digital Curation and Digital Preservation Works | Open Access Works | Digital Scholarship | Digital Scholarship Sitemap

"The Open Access Citation Advantage: Does It Exist and What Does It Mean for Libraries?"

Colby Lil Lewis has published "The Open Access Citation Advantage: Does It Exist and What Does It Mean for Libraries?" in Information Technology and Libraries.

Here's an excerpt:

The last literature review of research on the existence of an Open Access Citation Advantage (OACA) was published in 2011 by Philip M. Davis and William H. Walters. This paper reexamines the conclusions reached by Davis and Walters by providing a critical review of OACA literature that has been published 2011, and explores how increases in OA publication trends could serve as a leveraging tool for libraries against the high costs of journal subscriptions.

Academic Library as Scholarly Publisher Bibliography | Digital Curation and Digital Preservation Works | Open Access Works | Digital Scholarship | Digital Scholarship Sitemap

"Will Open Access Increase Journal CiteScores? An Empirical Investigation over Multiple Disciplines"

Yang Li et al. have published "Will Open Access Increase Journal CiteScores? An Empirical Investigation over Multiple Disciplines" in PLoS ONE.

Here's an excerpt:

This paper empirically studies the effect of Open Access on journal CiteScores. We have found that the general effect is positive but not uniform across different types of journals. In particular, we investigate two types of heterogeneous treatment effect: (1) the differential treatment effect among journals grouped by academic field, publisher, and tier; and (2) differential treatment effects of Open Access as a function of propensity to be treated. The results are robust to a number of sensitivity checks and falsification tests. Our findings shed new light on Open Access effect on journals and can help stakeholders of journals in the decision of adopting the Open Access policy.

Academic Library as Scholarly Publisher Bibliography | Digital Curation and Digital Preservation Works | Open Access Works | Digital Scholarship | Digital Scholarship Sitemap

"Google Scholar, Web of Science, and Scopus: A Systematic Comparison of Citations in 252 Subject Categories"

Alberto Martín-Martín have self-archived "Google Scholar, Web of Science, and Scopus: A Systematic Comparison of Citations in 252 Subject Categories."

Here's an excerpt:

Despite citation counts from Google Scholar (GS), Web of Science (WoS), and Scopus being widely consulted by researchers and sometimes used in research evaluations, there is no recent or systematic evidence about the differences between them. In response, this paper investigates 2,448,055 citations to 2,299 English-language highly-cited documents from 252 GS subject categories published in 2006, comparing GS, the WoS Core Collection, and Scopus. GS consistently found the largest percentage of citations across all areas (93%-96%), far ahead of Scopus (35%-77%) and WoS (27%-73%). GS found nearly all the WoS (95%) and Scopus (92%) citations. Most citations found only by GS were from non-journal sources (48%-65%), including theses, books, conference papers, and unpublished materials. Many were non-English (19%-38%).. . . The results suggest that in all areas GS citation data is essentially a superset of WoS and Scopus, with substantial extra coverage.

Academic Library as Scholarly Publisher Bibliography | Digital Curation and Digital Preservation Works | Open Access Works | Digital Scholarship | Digital Scholarship Sitemap

"Can Microsoft Academic Help to Assess the Citation Impact of Academic Books?"

Kayvan Kousha and Mike Thelwall have self-archived "Can Microsoft Academic Help to Assess the Citation Impact of Academic Books?"

Here's an excerpt:

Despite recent evidence that Microsoft Academic is an extensive source of citation counts for journal articles, it is not known if the same is true for academic books. This paper fills this gap by comparing citations to 16,463 books from 2013-2016 in the Book Citation Index (BKCI) against automatically extracted citations from Microsoft Academic and Google Books in 17 fields. About 60% of the BKCI books had records in Microsoft Academic, varying by year and field. Citation counts from Microsoft Academic were 1.5 to 3.6 times higher than from BKCI in nine subject areas across all years for books indexed by both. Microsoft Academic found more citations than BKCI because it indexes more scholarly publications and combines citations to different editions and chapters. In contrast, BKCI only found more citations than Microsoft Academic for books in three fields from 2013-2014. Microsoft Academic also found more citations than Google Books in six fields for all years. Thus, Microsoft Academic may be a useful source for the impact assessment of books when comprehensive coverage is not essential.

Academic Library as Scholarly Publisher Bibliography | Digital Curation and Digital Preservation Works | Open Access Works | Digital Scholarship | Digital Scholarship Sitemap

Scholarly Metrics Recommendations for Research Libraries: Deciphering the Trees in the Forest

LIBER has released Scholarly Metrics Recommendations For Research Libraries: Deciphering the Trees in the Forest.

Here's an excerpt from the announcement:

The report sets out recommendations on how research libraries and information infrastructures can deal with scholarly metrics, and how to get started with the development of services to support this. The recommendations are grouped into four important types of activities relating to metrics:

  1. Discovery and Discoverability
  2. Showcasing Achievements
  3. Service Development
  4. Research Assessment

Research Data Curation Bibliography, Version 9 | Digital Curation and Digital Preservation Works | Open Access Works | Digital Scholarship | Digital Scholarship Sitemap