Resource Library

This book critically analyses the value of citation data, altmetrics, and artificial intelligence to support the research evaluation of articles, scholars, departments, universities, countries, and funders. It introduces and discusses indicators that can support research evaluation and analyses their strengths and weaknesses as well as the generic strengths and weaknesses of the use of indicators…

This paper reviewed existing literature and international best practices and established consensus by Delphi survey on 10 key measures for assessing research quality and impact that align with DORA values. These measures were chosen from a pool of 50 compiled measures that emphasize diverse and holistic evaluations of research performance beyond traditional metrics like journal…

This internal report for the University of Waikato investigates researchers’ perspectives on metrics and their influence on research impact, highlighting a multifaceted view of the role these metrics play in the academic landscape. While researchers generally acknowledged the usefulness of metrics, they were concerned about the over-reliance on these metrics, which can lead to a…

This paper explores the relationship between open access (OA) publishing and citation diversity, offering insights into how accessibility shapes scholarly communication. Regarding responsible research assessment, open scholarship is largely important, helping to evaluate work on its own merits when it is openly accessible. The authors determined that freely available scholarly publications attract citations from a…

This report from the White House Office of Science and Technology Policy (OSTP) discusses financing mechanisms for open access publishing of federally funded research. Overall, the report underscores the importance of financing mechanisms to support open access publishing of federally funded research, emphasizing the need for continued efforts to ensure broad access to scientific knowledge…

Reviewers play a critical role in upholding research integrity within the expert review and funding process at the Canadian Cancer Society (CCS). The expert review encompasses a diverse group ranging from researchers, clinicians, patients, survivors, caregivers, decision-makers, and knowledge users, collectively offering a multifaceted evaluation of research grant applications. In alignment with DORA, CCS advocates…

There is a growing recognition that current evaluation metrics often fail to capture the breadth and depth of research impact. In hopes of addressing the reform needed in the research world the Global Young Academy, the InterAcademy Partnership, and the International Science Council have collaborated in conducting a global assessment of research evaluation perspectives. Their…

Progress towards more responsible research assessment requires implementing and supporting more equitable and transparent standards of practice in the scientific community. Some journals have been considering these changes in how they operate, one of which is PLOS Biology. This article discusses PLOS Biology’s commitment to increasing code sharing and publishing peer review files in 2024…

The Luxembourg National Research Fund (FNR) introduced a standardized narrative-style CV format in 2021 as a requirement for applicants to all of FNR’s funding schemes. This new format was introduced to streamline and simplify the evaluation process, broaden the types of contributions considered during review, and to align with DORA’s principles that research should be…

The International Development Research Centre (IDRC) in Canada developed a tool called Research Quality Plus (RQ+) to assess and improve quality of research . The three tenets of RQ+ approach are: identify contexts of research, quality is multi-dimensional, and use of rubrics for systematic and empirical appraisal. RQ+ Approach has five major applications: Priority Setting:…

Preprints are an essential tool for accelerating scientific discovery as they provide early and free access to new findings. Since the COVID-19 pandemic, the usage of preprints have surged on a worldwide level and its availability allows discussion and assessment to be carried over to a wider community. However public participation is very low and…

The Research Quality Committee at Neuroscience Research Australia (NeuRA) has developed a new tool to assess and improve scientific research quality and reproducibility: Quality Output Checklist and Content Assessment (QuOCCA) The QuOCCA was designed after extensive research and feedback from external experts and researchers. The QuOCCA was specifically created to assess a broad spectrum of…

Ruth Schmidt and colleagues explain how systems thinking can advance responsible research assessment. In an infographic format, they identify external forces that reinforce the use of the Journal Impact (JIF) as a surrogate measure for research quality in academic career advancement, despite its well documented deficiencies as a tool for research assessment. By demonstrating schematically…

The Open Reviewers Toolkit was created by PREreview in 2021 to help guide the responsible evaluation of research manuscripts. The toolkit offers guidance to help reviewers assess their biases during the review process, a framework for how to read and evaluate a manuscript, and a rubric for self-assessment to improve the review. The toolkit features…

This is the 2020 final report from the European Open Science Policy Platform (OSPP) that provides an update on the adoption of their Open Science recommendations (OSPP-REC, 2017) and their transformation on PCIs (Practical Commitments for Implementation). The OSPP is an advisory group established by the Directorate-General for Research and Innovation of the European Commission…

Top Factor is a tool developed by the Center of Open Science to rate journals’ policies adherence to the Transparency and Openness Promotion (TOP) Guidelines and guide the implementation of more transparent and reproducible research. Published in 2015, the TOP Guidelines provide eight modular standards of journal publications, with multiple levels of increasing stringency within…

In this piece, Alicia Kowaltowski, Ariel Silber, and Marcus Oliveira outline challenges faced by the scientific research community resulting from the current framework of academic assessment. Challenges outlined by the authors include: Increased pressure on researchers to produce knowledge Increased pressure on researchers to publish and increased frequency of retraction of scientific articles due to…

Editorial decisions influence research assessment. This article provides 10 concrete recommendations that journals and publishers can take to help improve how research is assessed.