In October, my institution was granted access to the Beta version of ScopusAI. I have tested it using a concept connected to my PhD dissertation in physics, an "electromagnon". In this post, I want to share my experience and use it to illustrate the many dimensions the design and assessment of such tools need to consider. . . .
[The author provides an extensive description and analysis of the performed tests as well as their broader implications.]
And if AI is only as good as its underlying data, let’s not forget who owns the scholarly data and regulates access to it. Big scholarly publishers have long been using content as a resource to capitalize on. AI tools amplify existing imbalances in access to scholarly text: if a publisher owns the exclusive right to a text, they can train their own AI on it and make this content unavailable to competing AI projects, profiting from the copyright yet again. Currently, most AI research assistants are grounded with abstracts, but the real value is contained in the full text of articles, and accessing them remains very difficult.
This study examines the tendency to cite older work across 20 fields of study over 43 years (1980–2023). . . . Our analysis, based on a dataset of approximately 240 million papers, reveals a broader scientific trend: many fields have markedly declined in citing older works (e.g., psychology, computer science). . . . Our results suggest that citing more recent works is not directly driven by the growth in publication rates. . . even when controlling for an increase in the volume of papers. Our findings raise questions about the scientific community’s engagement with past literature, particularly for NLP, and the potential consequences of neglecting older but relevant research.
This study explores the potential of ChatGPT, a large language model, in scientometrics by assessing its ability to predict citation counts, Mendeley readers, and social media engagement. In this study, 2222 abstracts from PLOS ONE articles published during the initial months of 2022 were analyzed using ChatGPT-4, which used a set of 60 criteria to assess each abstract. Using a principal component analysis, three components were identified: Quality and Reliability, Accessibility and Understandability, and Novelty and Engagement. The Accessibility and Understandability of the abstracts correlated with higher Mendeley readership, while Novelty and Engagement and Accessibility and Understandability were linked to citation counts (Dimensions, Scopus, Google Scholar) and social media attention. Quality and Reliability showed minimal correlation with citation and altmetrics outcomes. Finally, it was found that the predictive correlations of ChatGPT-based assessments surpassed traditional readability metrics. The findings highlight the potential of large language models in scientometrics and possibly pave the way for AI-assisted peer review.
Journal editors have a large amount of power to advance open science in their respective fields by incentivising and mandating open policies and practices at their journals. The Data PASS Journal Editors Discussion Interface (JEDI, an online community for social science journal editors: www.dpjedi.org) has collated several resources on embedding open science in journal editing (www.dpjedi.org/resources). However, it can be overwhelming as an editor new to open science practices to know where to start. For this reason, we created a guide for journal editors on how to get started with open science. The guide outlines steps that editors can take to implement open policies and practices within their journal, and goes through the what, why, how, and worries of each policy and practice. This manuscript introduces and summarizes the guide (full guide: https://doi.org/10.31219/osf.io/hstcx).
Monitoring systems are essential for tracking the progress in open access (OA) and particularly the goal of transitioning from paywalled to OA publications in many European countries. In this work, we express our opinion about the challenges faced by monitoring dashboards in providing a complete view of the OA status, ensuring accuracy in measuring OA production and achieving efficiency in the entire process. We analyze the characteristics of various monitoring systems from European countries, including the sources of data, formats, visualization methods, update frequencies, granularity and types of access recorded. We conclude by underlining the importance of monitoring systems in showcasing policy implementation, aiding decision-making, ensuring compliance and measuring impact in the pursuit of a more open scholarly landscape.
Diverse efforts are underway to reform the journal peer review system. Combined with growing interest in Open Science practices, Open Peer Review (OPR) has become of central concern to the scholarly community. However, what OPR is understood to encompass and how effective some of its elements are in meeting the expectations of diverse communities, are uncertain. This scoping review updates previous efforts to summarize research on OPR to May 2022. Following the PRISMA methodological framework, it addresses the question: "What evidence has been reported in the scientific literature from 2017 to May 2022 regarding uptake, attitudes, and efficacy of two key aspects of OPR (Open Identities and Open Reports)?" The review identifies, analyses and synthesizes 52 studies matching inclusion criteria, finding that OPR is growing, but still far from common practice. Our findings indicate positive attitudes towards Open Reports and more sceptical approaches to Open Identities. Changes in reviewer behaviour seem limited and no evidence for lower acceptance rates of review invitations or slower turnaround times is reported in those studies examining those issues. Concerns about power dynamics and potential backfiring on critical reviews are in need of further experimentation. We conclude with an overview of evidence gaps and suggestions for future research. Also, we discuss implications for policy and practice, both in the scholarly communications community and the research evaluation community more broadly.Diverse efforts are underway to reform the journal peer review system. Combined with growing interest in Open Science practices, Open Peer Review (OPR) has become of central concern to the scholarly community. However, what OPR is understood to encompass and how effective some of its elements are in meeting the expectations of diverse communities, are uncertain. This scoping review updates previous efforts to summarize research on OPR to May 2022. Following the PRISMA methodological framework, it addresses the question: "What evidence has been reported in the scientific literature from 2017 to May 2022 regarding uptake, attitudes, and efficacy of two key aspects of OPR (Open Identities and Open Reports)?" The review identifies, analyses and synthesizes 52 studies matching inclusion criteria, finding that OPR is growing, but still far from common practice. Our findings indicate positive attitudes towards Open Reports and more sceptical approaches to Open Identities. Changes in reviewer behaviour seem limited and no evidence for lower acceptance rates of review invitations or slower turnaround times is reported in those studies examining those issues. Concerns about power dynamics and potential backfiring on critical reviews are in need of further experimentation. We conclude with an overview of evidence gaps and suggestions for future research. Also, we discuss implications for policy and practice, both in the scholarly communications community and the research evaluation community more broadly.
The article presents the methodology used by the Colombia Consortium to negotiate the first transformative agreements (TAs) in Latin America. These TAs are a strategy to manage costs associated with Article Processing Charges (APCs), facilitate the transition to Open Access (OA) and increase the visibility of Colombian publications.
We posed selected questions to Stevan Harnad thirty years after his "subversive proposal" to self-archive online scholarly articles in university-hosted or disciplinary repositories to make them openly available and thus maximize research impact. A combination of factors including unfounded scepticism concerning open access, and bureaucratic access to the few institutional repositories launched by universities chiefly drove this outcome. The conclusions of the study may further inform educational efforts on scholarly communication in the digital era aimed at undergraduate students and researchers alike.
The book begins with the history of digital developments and their influence on the founding of international policies toward open scholarship. The concept of making research more freely available to the broader community, in practice, will require changes across every part of the system: government agencies, funders, university administrators, publishers, libraries, researchers and IT developers. To this end, the book sheds light on the urgent need for partnership and collaboration between diverse stakeholders to address multi-level barriers to both the policy and practical implementation of open scholarship. It also highlights the specific challenges confronted by the humanities which often makes their presentation in accessible open formats more costly and complex. Finally, the authors illustrate some promising international examples and ways forward for their implementation. The book ends by asking the reader to view their role as a researcher, university administrator, or member of government or philanthropic funding body, through new lenses. It highlights how, in our digital era, the frontiers through which knowledge is being advanced and shared can reshape the landscape for academic research to have the greatest impact for society.
There are many items in Wikidata representing scholarly articles. However, these items have been created mostly by volunteer Wikidata editors and not systematically by journal publishers or editors, which can lead to gaps and inconsistencies in the datasets. This article presents findings from a survey investigating practices of library and information studies (LIS) journals in Wikidata item creation. Believing that a significant number of LIS journal editors would be aware of Wikidata and some would be creating Wikidata items for their publications, the authors sent a survey asking 138 English-language LIS journal editors if they created Wikidata items for materials published in their journal and follow-up questions. With a response rate of 41 percent, respondents overwhelmingly indicated that they did not create Wikidata items for materials published in their journal and were completely unaware of or only somewhat familiar with Wikidata. Respondents indicated that more familiarity with Wikidata and its benefits for scholarly journals as well as institutional support for the creation of Wikidata items could lead to greater participation; however, a campaign of education about Wikidata, documentation of benefits, and support for creation would be a necessary first step. The article presents and discusses the results of the survey, but the conclusions that can be drawn are minimal; therefore, the authors also discuss the benefits of creating Wikidata items for LIS journals as a first step in this educational campaign for editors and publishers.
The Institutional Publishing Landscape Report is built on 685 survey responses from institutional publishers and publishing service providers across the European Research Area. The findings illustrate the state of institutional publishing in Europe and show that a large portion of these organisations are operating with a diamond OA model. The report also discusses how institutional publishers are run and sustained, what activities they are involved in, and which services are outsourced. While the surveyed group is not necessarily representative of all institutional publishers and service providers in Europe, the findings broadly demonstrate the current operations of institutional publishers, their challenges, and the opportunities for supporting them in the future.
The results showed that compared to non-OA, Gold OA is advantageous in reducing the retraction time of flawed articles, but does not demonstrate a significant advantage in reducing citations after retraction. This indicates that Gold OA may help expedite the detection and retraction of flawed articles, ultimately promoting the practice of responsible research.
It is unclear whether regional rather than global or discipline-focussed preprint platforms as an innovation in the communication of science are removing any of the barriers faced by researchers in the scientific periphery or whether they are increasing access to and visibility of science from the periphery. In response, this paper focusses on the uptake, visibility and academic impact of regional preprint publishing platforms in two peripheral regions (Africa and Latin America) to gain insights into the use and possible impact of regional preprint servers.
The publication of the survey results marks a significant milestone for DIAMAS and for Diamond Open Access. There is now a clear and intelligible picture of the European landscape of institutional publishing activities, with clear pathways to strengthen and support their operations. Our findings show how institutional publishers work, the scale of and nature of their operations, the ways finances and funding are managed, how open science practices are managed, and the nature of their challenges.
Artificial-intelligence tools in research like ChatGPT are playing an increasingly transformative role in revolutionizing scientific publishing and re-shaping its economic background. They can help academics to tackle such issues as limited space in academic journals, accessibility of knowledge, delayed dissemination, or the exponential growth of academic output. Moreover, AI tools could potentially change scientific communication and academic publishing market as we know them. They can help to promote Open Access (OA) in the form of preprints, dethrone the entrenched journals and publishers, as well as introduce novel approaches to the assessment of research output. It is also imperative that they should do just that, once and for all.
We are pleased to announce a new Open Education strategy for 2024-2026, Librarians as Agents of Change. We will support Higher Education policymakers, librarians, ambassadors and facilitators of OE in Europe to implement the UNESCO OER Recommendations using a targeted and action-oriented approach. With this strategy, we aim to make the many connections between Open Science policy and Open Education ever clearer to both policymakers and academic institutions.
Today, the scholarly publishing sector is undergoing its second digital transformation. The first digital transformation saw a massive shift from paper to digital, but otherwise publishing retained many of the structures, workflows, incentives, and outputs that characterized the print era. A variety of shared infrastructure was developed to serve the needs of this first digital transformation. In this current second digital transformation, many of the structures, workflows, incentives, and outputs that characterized the print era are being revamped in favor of new approaches that bring tremendous opportunities, and also non-trivial risks, to scholarly communication. The second digital transformation requires shared infrastructure that is fit for purpose. It is our objective with this paper to examine the needs for shared infrastructure that will support this second digital transformation.
Now, after years of little conclusive evidence to support these assertions, researchers report that open-access papers have a greater reach than paywalled ones in two key ways: They attract more total citations, and those citations come from scholars in a wider range of locations, institutions, and fields of research. The study also reports a "citation diversity advantage" for a controversial type of open-access article, those deposited in "green" public repositories.
A high-profile group of funders, academic publishers and research organizations has launched an effort to tackle one of the thorniest problems in scientific integrity: paper mills, businesses that churn out fake or poor-quality journal papers and sell authorships. In a statement released on 19 January, the group outlines how it will address the problem through measures such as closely studying paper mills, including their regional and topic specialties, and improving author-verification methods.
This work reveals an alarming preservation deficit. Only 0.96% of Crossref members (n = 204) can be confirmed to digitally preserve over 75% of their content in three or more of the archives that we studied. (Note that when, in this article, we write "preserved," we mean "that we were able to confirm as preserved," as per the specified limitations of this study.) A slightly larger proportion, i.e., 8.5% (n = 1,797), preserved over 50% of their content in two or more archives. However, many members, i.e., 57.7% (n = 12,257), only met the threshold of having 25% of their material in a single archive. Most worryingly, 32.9% (n = 6,982) of Crossref members seem not to have any adequate digital preservation in place, which is against the recommendations of the Digital Preservation Coalition.
Article processing charges that authors and research institutions pay to make articles open access are increasing. If manuscript submission is price elastic, then rising charges will cause a significant reduction in submissions, leading to decreased revenues under constant acceptance rates. Therefore, the elasticity of manuscript submission to article processing charge is one of the determinants of publishers’ charges. However, several studies that investigated the determinants of article processing charges did not consider this elasticity. This study investigated the determinants of submissions, including the elasticity to article processing charge, by formulating the number of manuscript submissions to fully open access journals published by Hindawi and Elsevier in 2022. Moreover, this study formulated manuscript submissions using both list prices and charges paid to Elsevier that OpenAPC collected to compare the results. The estimation results reveal that the two publishers increase their revenues by raising the article processing charges due to the inelasticity. Moreover, these conclusions do not depend on the data set used, although the number of observations sourced from OpenAPC is small.
Here, based on a large dataset of computer science publications, we study trends in the use of early preprint publications and revisions on ArXiv and the use of X (formerly Twitter) for promotion of such papers in the last 10 years. We find that early submission to ArXiv and promotion on X have soared in recent years. Estimating the effect that the use of each of these modern affordances has on the number of citations of scientific publications, we find that in the first 5 years from an initial publication peer-reviewed conference papers submitted early to ArXiv gain on average 21.1±17.4 more citations, revised on ArXiv gain 18.4±17.6 more citations, and promoted on X gain 44.4±8 more citations. Our results show that promoting one’s work on ArXiv or X has a large impact on the number of citations, as well as the number of influential citations computed by Semantic Scholar, and thereby on the career of researchers. We discuss the far-reaching implications of these findings for future scientific publishing systems and measures of scientific impact.
Scopus AI is based on Scopus’ trusted content from over 27,000 academic journals, from more than 7,000 publishers worldwide, with over 1.8 billion citations, and includes over 17 million author profiles. Scopus content is vetted by an independent board of world-renowned scientists and librarians who represent the major scientific disciplines.
Since the alpha launch in August 2023, thousands of researchers across the world have tested Scopus AI. Their feedback has reinforced that, as generative AI evolves, researchers want trustworthy, cited research that is relevant and highly personalized to their needs.
Feedback from the research community has led to Scopus AI offering the following powerful features:
- Expanded and Enhanced Summaries that provide researchers with fast overviews of key topics that they can dig deeper into, sometimes even highlighting gaps in literature. . . .
- Foundational and Influential Papers that enable researchers to rapidly pinpoint seminal works, navigating academic progress and impact with precision and ease.
- Academic Expert Search identifies leading experts in their fields and provides explanations of their expertise relevant to the user’s query, helping save time.
- Enhanced breadth of research, covering ten years of Scopus content to support well-rounded perspective on topics of interest, and improved design to enhance the user experience.
ChatGPT may effectively distinguish between predatory and legitimate journals, with accuracy rates of 92.5% and 71%, respectively. The potential utility of large-scale language models in exposing predatory publications is worthy of further consideration.
Research data and software are widely accepted as an outcome of scientific work. However, in comparison to text-based publications, there is not yet an established process to assess and evaluate quality of research data and research software publications. This paper presents an attempt to fill this gap. Initiated by the Working Group Open Science of the Helmholtz Association the Task Group Helmholtz Quality Indicators for Data and Software Publications currently develops a quality indicator for research data and research software publications to be used within the Association. This report summarizes the vision of the group of what all contributes to such an indicator. The proposed approach relies on generic well-established concepts for quality criteria, such as the FAIR Principles and the COBIT Maturity Model. It does — on purpose — not limit itself to technical implementation possibilities to avoid using an existing metric for a new purpose. The intention of this paper is to share the current state for further discussion with all stakeholders, particularly with other groups also working on similar metrics but also with entities that use the metrics.