A DORAat10 Local Event Report

In May 2023, DORA celebrated it’s 10th Anniversary with two plenary sessions and a decentralized weeklong program of local events organized by community members from around the world. Event organizers were given the option to write brief reports on their events that summarize key takeaways and recommendations.

By SOMMa Open Science Working Group

• Eva Mendez, representing CoARA and Pilar Paneque, director of the ANECA participated in the event
• The round table discussion “The reform of science evaluation and its impact on attracting and retaining talent” was led by several experts in the field

Representatives of the 65 entities that form part of the Alliance of Excellence of Centres Severo Ochoa Centres and María de Maeztu Units (SOMMa) met in Bilbao in an event organised by the Basque Center for Applied Mathematics (BCAM) and Catalan Institute of Nanoscience and Nanotechnology (ICN2) to analyse the latest trends in the evaluation of scientific research.

To celebrate the occasion of the 10th anniversary of the San Francisco Declaration on Research Assessment (DORA), the SOMMa Alliance brought its community of open science professionals together with experts on new policies in research evaluation to discuss their impact on the research activity and the promotion of research careers.

SOMMa actively participated in the process of drafting the first National Open Science Strategy (ENCA) for the period between 2023 and 2027, drawn up by the Ministry of Science and Innovation and the Ministry of Universities.

During the meeting, the SOMMa open science working group and its members assessed the possibility of SOMMa becoming a member of the Coalition for Advancing Research Assessment (CoARA).

Eva Méndez, representing the International Coalition for the Advancement of Research Assessment (CoARA), presented on the agreement, which provides a basis for promoting the necessary reforms to ensure that research assessment is based on qualitative criteria.

Pilar Paneque, director of the National Agency for Quality Assessment and Accreditation (ANECA) shared with the audience the role of quality agencies in the reform of research assessment.

This was followed by the round table “The reform of science evaluation and its impact on attracting and retaining talent” with the participation of Eva Méndez, representing CoARA, Ismael Ràfols from the Centre for Science and Technology Studies (CWTS) Leiden University, Pilar Rico, head of the Open Access Unit FECYT and Fernando Orejas, expert in the area of information technologies. During the round table, the Open Science group considered that the following commitments would apply to SOMMa:

• Recognise the diversity of contributions to and careers in, research in accordance with the needs and nature of the research
• Base research assessment primarily on qualitative evaluation for which peer review is central, supported by responsible use of quantitative indicators
• Abandon inappropriate uses in research assessment of journal- and publication-based metrics, in particular, inappropriate uses of Journal Impact Factor (JIF) and h-index
• Commit resources to reform research assessment as is needed to achieve the organisational changes committed to
• Raise awareness of research assessment reform and provide transparent communication, guidance, and training on assessment criteria and processes as well as their use
• Exchange practices and experiences to enable mutual learning within and beyond the Coalition
• Communicate progress made on adherence to the Principles and implementation of the Commitments

In conclusion of the event, the working group recommended that SOMMa join CoARA. This will be the starting point for a new 5-year journey.

The post DORA’s 10th Anniversary in Bilbao: the starting point of a new journey appeared first on DORA.

A DORAat10 Local Event Report

In May 2023, DORA celebrated it’s 10th Anniversary with two plenary sessions and a decentralized weeklong program of local events organized by community members from around the world. Event organizers were given the option to write brief reports on their events that summarize key takeaways and recommendations.
Scroll down for the English translation of this report: Potentialities of Open Access Repositories for inclusive, diverse and equitable research assessment in LAC

By Ana Luna Gonzalez

El panel “Potencialidades de los Repositorios de Acceso Abierto para la evaluación inclusiva, diversa y equitativa de la investigación en ALC” se centró en las formas en que los repositorios de acceso abierto pueden contribuir con procesos de evaluación de la investigación más inclusivos, diversos y equitativos en Latinoamérica y el Caribe. 

En el marco del 10 Aniversario de DORA y como parte de la Reunión Anual del COAR 2023, organizada conjuntamente por el Consejo Nacional de Rectores (CONARE) y LA Referencia, el evento fue apoyado por el CLACSO-FOLEC. Con la participación de especialistas de distintas instituciones involucradas en la temática, se presentaron los avances y desafíos pendientes de los repositorios de acceso abierto y su potencial contribución con una evaluación más responsable en la región. La moderación estuvo a cargo de Kathleen Shearer, directora ejecutiva de COAR. 

En primer lugar, Andrea Marin Campos (Universidad de Costa Rica) disertó sobre el rol de los repositorios en la evaluación de la investigación y sus alcances. Realizó una historización de los repositorios en Costa Rica e identificó dos objetivos principales: el primero, relacionado al Acceso Abierto y los beneficios para el desarrollo de la ciencia y la tecnología en el país y el segundo, vinculado a robustecer la evaluación de la investigación en diversas escalas.  A su vez, planteó distintos interrogantes: ¿cómo conocer los usos que hacen los diversos actores de la sociedad de la producción científica alojada en los repositorios? ¿Cómo evaluar la calidad de otros formatos de producción científica que nos son artículos? ¿Cómo implementar la evaluación de diversos contenidos y formatos de producción científica académica?

Por su parte, Laura Rovelli (CLACSO FOLEC) destacó que desde CLACSO-FOLEC existe un consenso creciente acerca de la necesidad de incorporar nuevas prácticas de evaluación que incentiven el acceso abierto en revistas diamante y en repositorios, pues no excluyen autores por razones económicas, y permiten concentrar la evaluación de pares “más en la calidad de la investigación que en la revista donde se publica”, siguiendo uno de los principios de la pionera Declaración de DORA sobre la Evaluación de la Investigación, en esta semana en la que estamos celebrando su 10° aniversario.

Marin Dacos (Francia) planteó algunos desafíos pendientes para los repositorios. En primer lugar, señaló la necesidad de visibilizar la diversidad de formatos de la producción científica de manera que sea clara para diferentes actores sociales, no solo para la academia. Asimismo, llamó a fortalecer los estándares de calidad de la evaluación para editores y construirlos para los repositorios. Además, sugirió que hay que aprovechar el momento político en el que actores de fuerte peso se encuentran interesados en la implementación de la Ciencia Abierta. 

Por su parte, Rodolfo Barrere (RICYT/Observatorio CTS OEI) exploró las complejidades en torno a la construcción de indicadores de evaluación de la producción científica que sean más representativos de la producción existente en la región y presentó un conjunto de desafíos para poder fortalecer la producción existente en los repositorios. 

Por último, Lautaro Matas (COAR Notify, Executive and Technical Director, La Referencia) identificó algunos problemas existentes de los repositorios en relación con la cobertura, la duplicación y la calidad de los metadatos. Además, presentó el COAR Notify Project, en el se asiste a los socios a adoptar un modelo común e interoperable de apoyo a las revisiones y aprobaciones de recursos distribuidos en repositorios, preprints y archivos.

La grabación de las intervenciones se encuentra disponibles aquí, en español e inglés.

Potentialities of Open Access Repositories for inclusive, diverse and equitable research assessment in LAC

By Ana Luna Gonzalez

The panel “Potentialities of Open Access Repositories for inclusive, diverse and equitable research assessment in LAC” focused on the ways in which open access repositories can contribute to more inclusive, diverse and equitable research assessment processes in Latin America and the Caribbean (LAC).

In the framework of DORA’s 10th Anniversary and as part of the COAR 2023 Annual Meeting, jointly organised by the National Council of Rectors (CONARE) and LA Referencia, the event was supported by CLACSO-FOLEC. Specialists from different institutions involved in the subject presented the advances and pending challenges of open access repositories and their potential contribution to a more responsible research assessment in the region. Kathleen Shearer, executive director of COAR, moderated the event.

First, Andrea Marin Campos (University of Costa Rica) spoke about the role of repositories in research assessment and their scope. She gave an overview of repositories in Costa Rica and identified two main objectives: the first, related to Open Access and the benefits for the development of science and technology in the country, and the second linked to strengthening the research assessment at different scales.  At the same time, she raised several questions: how to know the usage by different actors of society of the scientific production hosted in repositories? How to evaluate the quality of other formats of scientific production that are not articles? How to implement the evaluation of different contents and formats of academic scientific production?

Laura Rovelli (CLACSO-FOLEC) stressed that there is a growing consensus at CLACSO-FOLEC on the need to incorporate new evaluation practices that encourage open access in diamond journals and repositories, as they do not exclude authors for economic reasons and allow peer review to focus “more on the quality of the research than on the journal where it is published.” This follows one of the principles of the pioneering DORA Declaration on Research Assessment, in this week in which we are celebrating its 10th anniversary.

Marin Dacos (France) raised some pending challenges for repositories. Firstly, he pointed out the need to make the diversity of formats of scientific production visible so that they are clear for different social actors, not only for academia. He also called for strengthening the quality standards of evaluation for publishers and building them for repositories. In addition, he suggested taking advantage of the political moment in which powerful actors are interested in the implementation of Open Science.

Rodolfo Barrere (RICYT/Observatorio CTS OEI) explored the complexities surrounding the construction of scientific production evaluation indicators that are more representative of the existing production in the region and presented a set of challenges to strengthen the existing production in repositories.

Finally, Lautaro Matas (COAR Notify, Executive and Technical Director, La Referencia) identified some existing problems of repositories in relation to coverage, duplication and metadata quality. In addition, he presented the COAR Notify Project, which assists partners in adopting a common, interoperable model for supporting reviews and approvals of distributed resources in repositories, preprints and archives.

Recordings of the speeches are available here, in English and Spanish.

The post Panel “Potencialidades de los Repositorios de Acceso Abierto para la evaluación inclusiva, diversa y equitativa de la investigación en ALC” appeared first on DORA.

A DORAat10 Local Event Report

In May 2023, DORA celebrated it’s 10th Anniversary with two plenary sessions and a decentralized weeklong program of local events organized by community members from around the world. Event organizers were given the option to write brief reports on their events that summarize key takeaways and recommendations.

By Holly Limbert

This year, here at the University of Derby, we joined the global research community in celebration of the 10^th birthday of the Declaration on Research Assessment (DORA). It was important for us to mark this important event to demonstrate our continued commitment to the development of and approaches to responsible research assessment and Open Research more broadly. Our Repository and Open Access Librarian, Holly Limbert of the Research Liaison Team in the Library invited Professor Stephen Curry, Chair of the DORA Steering Committee, and Professor Cameron Neylon, co-lead of the Curtin Open Knowledge Initiative to deliver a talk which considered the what, why and how of research assessment and associated practices in the academy, how far we have come as a community, and where we might be heading in the future. Given both Stephen and Cameron’s commitment to questioning current and established practices when it comes to research evaluation, assessment and opening access to knowledge, the university was incredibly fortunate to secure two leading experts and inspirational speakers!

In his talk, ‘Ten years of DORA – evaluating what matters!‘ Professor Curry discussed the origins of the declaration and the organisation behind it. Whilst he acknowledged that change is evident across the research landscape, he stressed that DORA is still very much required to drive forward reform in research evaluation and assessment. A particularly pertinent aspect of Stephen’s talk addressed the pressures which researchers face relating to career advancement, publishing expectations and real-world impact in society. One example used was from a Ted Talk by Thomas Insel, a leading neuroscientist whose research focuses on mental health. In his talk, Insel makes explicit reference to the number of papers and money spent during his years at the National Institute of Mental Health (NIMH) versus how much impact was felt by those suffering from mental health problems during this time (Insel, 2013). This speaks volumes about the importance of questioning established practices and processes in the academy and advocating for change! Stephen also advocated for the Narrative C.V. which aims to highlight a wide range of skills and experiences, considering the vast contributions that researchers make to the research ecosystem and not just written publication. 

Professor Neylon’s talk ‘The problem of evaluation: DORA the explorer at 10 and our paths into the future’ centered on the role of evaluation in research across departments and geographies and some of the major challenges we face collectively in terms of how research evaluation is even thought of and considered, particularly in relation to why the academy evaluates research at all. One of the key takeaways from Cameron’s talk centered on rethinking what “excellence” means in research and how this is interpreted and understood. It was fascinating to hear that when presented with the question of ‘What is excellence?’ researchers and institutions find it is exceedingly difficult to provide a concrete definition. As co-lead of the Curtain Open Knowledge Initiative, which seeks to ‘…change the stories that universities tell about themselves, placing open knowledge at the heart of that narrative’ (COKI, 2023), Cameron and colleagues at Curtain University Australia, are passionate about using Open data sources to help universities become more open, transparent and accountable in their activities relating to scholarly communications and equality, diversity and inclusion practices.

After the talks took place, the floor was open to questions and a lively discussion ensued. Questions were raised about priority steps institutions can take to address certain established and potentially harmful ‘norms’ and how these can be phased out particularly when it comes to definitions and understandings of quality and prestige. The future research assessment exercise in the United Kingdom was also a topic of discussion, particularly around how metrics and understandings of quality might play a role in the future. There was also some debate regarding citations and readership and what both phenomena may indicate in different contexts and disciplines. A follow up question which Stephen and Cameron were both happy to respond to after the event, related to how research assessments can cater for the humanities especially as much of what is practiced is very heavily focused on STEM subject areas.

Overall, the event was an enormous success with over 160 registrants from across the globe and over 70 attendees on the day! This demonstrates the will and need for researchers and those in research support roles to transform the ways in which we assess and evaluate research. We need to challenge the systems in place which put emphasis on certain incentives and rewards in the academy. Whilst DORA has done much to change the way we think, feel and act when it comes to the value we give to certain measures of quality and what matters, there is still work to be done! The only way to achieve an open, inclusive, and equal culture of research is to continue to collaborate and help to establish new norms which celebrate and recognise the varied and wide-ranging contributions made by researchers the world over.

The post Evaluating what matters with DORA the explorer appeared first on DORA.

A DORAat10 Local Event Report

In May 2023, DORA celebrated it’s 10th Anniversary with two plenary sessions and a decentralized weeklong program of local events organized by community members from around the world. Event organizers were given the option to write brief reports on their events that summarize key takeaways and recommendations.

By Rebecca Hill, Simon Hettrick and David Moher

How do we assess research today and what has changed since the DORA declaration? And what can open research practices contribute to future research assessment systems? To explore these questions in more depth and to celebrate the 10th anniversaries of both DORA and the open research publisher F1000, F1000 hosted a webinar in May 2023, asking ‘What can open research values bring to research assessment reform?’

The panel discussion included David Moher (University of Ottawa/Hong Kong Principles); Simon Hettrick (Software Sustainability Institute/the Hidden Ref/University of Southampton); and Becky Hill (F1000).

What has changed since the DORA declaration?

David: For most researchers, career advancement is based on the currency of the day. Evidence indicates the global currency is usually the number of publications generated in a finite period. This reward scheme typically benefits certain disciplines and ‘public facing’ researchers. It excludes the many talented personnel working to bring the research to fruition. Along with the number of publications, assessors are often interested in the impact factor of the journal the research report is published in, as well as the total dollar figure associated with any awards. Clearly this tends to favor expensive research, often conducted in biomedicine. This reward ecosystem has not materially changed in 50 years.

Re-imagining incentives focused on more inclusive, transparent, and open ways of working are important. The 2020 Hong Kong principles are one such effort combining many aspects of community and research integrity. The principles intend to guide organisations’ research assessment, promotion and tenure practices to focus on incentivizing and rewarding researchers who incorporate open scholarship practices into their research. This aligns with the 2022 US Office of Science and Technology Policy recommendation on data sharing and public access to research. Implementing these frameworks as part of a researcher’s assessment make sense and is a movement towards using evidence as part of the assessment process.

Alongside what DORA is doing, it will be important to watch the development of the Coalition for Advancing Research Assessment (CoARA), an initiative committed to reimaging research evaluation and how we incentivize and reward researchers for the future.

Becky: The last five years have seen the introduction of major policy imperatives aimed at improving how we do and deliver research. These are often underpinned by a drive to more open and collaborative ways of working. The publication of the 2021 UNESCO recommendation on open science was a major landmark for attitudes to research practices and behaviors, endorsed by 193 nations. It highlights ‘Open Scientific Knowledge’ as integral to an effective research system –publishers can play a pivotal role in realising this.

What can publishers do to support responsible research assessment?

Becky: As David notes, the focus on publications as the currency for researchers to demonstrate their value in research assessment systems is widely acknowledged as problematic. DORA sets out clear guidance for publishers to help improve how research is evaluated, which include practicing more responsible use of research metrics, but also highlights the integral role that publishers can play in bringing greater transparency to the research dissemination process.

Publishers are enablers of open scientific knowledge by a combination of services they can provide, be that providing open access to research outputs, facilitating data sharing, creating routes for the publication of a diversity of research outputs, and ensuring research integrity. Publishers can also provide greater visibility of the myriad of specific and valuable contributions to research output through, for example, adoption of the Contributor Role Taxonomy (CRediT), or through providing more transparent and open peer review, bringing the often unseen role of reviewers to the fore.

What are great examples of innovation to help shape research assessment reform?

Simon: Research relies on the efforts of many non-traditional academic roles, such as technicians, data stewards, and research software engineers and the value of such roles is largely ‘hidden’ in traditional research assessment systems. The UK’s Research Excellence Framework (REF) is the national assessment framework for universities to present the impact of their research, the outcome of which determines the distribution of billions of pounds of funding. Despite a broad framework encouraging universities to present the impact of research across everything from musical compositions to software, universities place almost all their focus on published outputs. In the last REF, only 2.4% of outputs presented for assessment were not related to publications.

A focus on publications is problematic because research articles alone rarely describe all of the techniques, methods and technologies involved, nor are all the people who contributed to the research named in publications. The Hidden REF, a community-led initiative, was founded on the principle that if we do not recognise and provide incentives for everyone involved in the conduct of research, we will limit our ability to conduct research now and in the future. The Hidden REF started with a UK national competition in 2021, which recognised the vital work of a number of “Hidden Roles” without which, much research would be impossible. Partially due to lobbying by the Hidden REF, the next UK research assessment exercise will now recognise all roles – not just traditional academic ones.

What needs to happen now?

To drive reform across the research process, in both research culture and adoption of open practices we need to incentivise, recognise, and reward the behaviors we want to see – whether that’s sharing data openly or celebrating ‘hidden roles’ in research.

Each stakeholder has a role to play to drive change and adoption, but collaboration and partnerships across the research system are essential to deliver the promise of open research and reform in research evaluation. The question remains as to who needs to drive forward this change and how to ensure all parts of the system are in alignment – but DORA has certainly laid important foundations.

Want to learn more about what open research practices can contribute to research assessment reform? Watch the recording of the webinar.

The post What can open research values bring to research assessment reform? appeared first on DORA.

A DORAat10 Local Event Report

In May 2023, DORA celebrated it’s 10th Anniversary with two plenary sessions and a decentralized weeklong program of local events organized by community members from around the world. Event organizers were given the option to write brief reports on their events that summarize key takeaways and recommendations.

By Liam Bullingham and Nicola Wylie

Recent discussion on DORA mostly concerns the 2,800 organisations (universities, funders or publishers) which have signed, not the 20,000 individuals. See positions like, ‘Why have Elsevier committed to Leiden but not DORA?’, ‘We as a funder expect you to assess research following the DORA principles’, and ‘how should universities that sign be held to account?’

It’s understandable – organisations have much more agency to bring change and carry greater responsibility than individuals. But this means the community voice is lost, and DORA was started and grew from its individual signatories.

‘Why I Signed DORA’

We encouraged panellists and attendees to sideline their employer’s position and focus on their own morals, principles, or beliefs.

Our researcher panel drew from different disciplines and career stages. It was comprised of:

• Dr Rob Farrow, Senior Research Fellow in the Institute of Educational Technology, Open University, UK

…and all from Edge Hill University, UK:

• Dr Linamaría Pintor-Escobar, recent PhD award in Biology
• Dr Costas Gabrielatos, Reader (Corpus Linguistics & English Language)
• Dr Michel Belyk, Senior Lecturer in Psychology
• Dr Rebecca Pythian, Reader in Policing

‘What does DORA mean to you?’

Rob questioned the binary nature of signing, and whether we sign up to everything in the Declaration or just have a particular focus in mind when we do. Michel noted he hadn’t signed DORA until a few days ago, but was surprised to see how embedded its arguments already are in the wider community – many DORA principles are commonly held across psychologists in 2023. But signing helps us ‘put our money where our mouth is’. Costas recently signed too, simply because he didn’t know individuals could do so. But like Michel, many of these views are already held by him. He criticised obsessions with ‘top’ journals, or whether titles are indexed in Scopus. Poor or questionable research can be indexed too! Rebecca works in policing research. For her, many key journals are community-run and focus on local agendas, so can be overshadowed by big international journal titles or brands. DORA provides a framework to challenge this and move towards equal respect or recognition across disciplines. Signing DORA as an individual puts weight behind influencing wider change. Linamaría considered the range of different outputs you can have as a researcher, and the need for these to be recognised equally alongside journal articles. This is sometimes called ‘bibliodiversity’ and variety in scientific outputs is highlighted in DORA. Space for this diversity is really important for her as an early career researcher (ECR) as she hopes to gain recognition for the wide range of research she is doing.

‘What do you think DORA has achieved?’

Rob mentioned we have a better REF because of DORA. He noted that the way REF assesses ‘quality’ in research outputs deliberately de-couples the venue of a journal article from any assumptions about its quality. Rebecca is reassured by the positive influence on research funders. Funders now consider a wide range of factors when assessing research quality, with less emphasis on publication lists. Instead, more inclusive methods such as Narrative CVs are being drawn on. This is something that Linamaría is pleased to see too – she believes it will empower ECRs.

Audience feedback

Edge Hill PhD researcher Elizabeth Devine ran polls for us during the session, to help us understand where DORA stands in 2023. We share results and comments of the polls below.

During this poll, two people echoed Costas’ remarks that they didn’t know they could personally sign. Another supports DORA personally without signing because their university hasn’t supported DORA. Someone else never thought to sign because they aren’t an academic. In response, Liam offered his opinion about why research supporters are a part of scholarly communications and should feel they can sign too.

In an attempt to identify areas for action, we also asked participants what could be done to influence organisations to follow the DORA principles more. Some themes emerging from these responses included:

• funders can influence practices by including training or requirements
• accountability to DORA responsibilities and awards/recognition
• prestige attached to high impact journals
• improve the agency of individual DORA signatories
• using assessment methods in a wide-ranging, better-informed way over singular metrics
• improve researcher training
• better publisher practices and repository infrastructure

We continued by asking participants about opportunities and challenges in research assessment, with response themes emerging around:

• Responsible metrics and subjectivity weighed against the need for ‘simple’ measures
• Support from research leaders
• Re-use of research as impact
• Open research and bibliodiversity
• Opportunities from generative artificial intelligence
• Collaboration between organisations
• Fairness for ECRs, different disciplines, research in languages other than English

Prompted by the audience, we discussed CoARA and noted that unlike with DORA, individuals can’t sign as individuals, which is a challenge as new initiatives flow from the DORA legacy.

A view from Lancaster

Unable to join us for the event, a senior researcher from Lancaster University offered:

“As a woman in science, even a well-cited one, I have been told by more than one senior (male) colleague that I publish some ‘rather good work’ in some ‘lesser’ journals. I have been asked to consider the impact factors of journals by more than one (male) Head of Department. And there remains a bias on Journal Impact Factor (JIF) in the internal evaluation of publications to be selected for REF, despite the feedback from panel members that they do not take JIF into account (and I believe them).

I truly do believe that we, as scientists, should interrogate the evidence – the published work – and make our own independent judgements on its quality or academic impact. And if we wish to make a broader societal impact, we must be supported to publish non-REFable book chapters and to publish in those journals with lower JIF, but which are more likely to be read by users of research. I would hope that Lancaster’s emphasis on engagement makes us well positioned to be leaders on issues such as those raised by DORA.”

The post Why I signed DORA appeared first on DORA.

A DORAat10 Local Event Report

In May 2023, DORA celebrated it’s 10th Anniversary with two plenary sessions and a decentralized weeklong program of local events organized by community members from around the world. Event organizers were given the option to write brief reports on their events that summarize key takeaways and recommendations.

By Stephanie Warner, PhD, Manager, Knowledge Engagement, University of Calgary with support from representatives of NSERC, SSHRC, CIHR, Genome Canada and CFI

To remove barriers to research funding, hiring, tenure and promotion that many researchers in Canada continue to experience, sustained and collaborative effort is needed. The Declaration on Research Assessment aims for thoughtful inclusion of broader types of research outputs and societal impact in assessment of researchers for funding, tenure, promotion, merit and hiring.

The DORA movement in Canada is growing, with 47 organizations having signed the commitment to more effective and robust approaches to research assessment. The five major Government of Canada research funders (Social Sciences and Humanities Research Council/SSHRC, Canadian Institutes of Health Research/CIHR, Natural Sciences and Engineering Research Council/NSERC, Genome Canada and Canada Foundation for Innovation/CFI) all signed DORA in 2019. Although many institutions in Canada are adopting principles related to responsible research assessment and research impact, only seven Canadian postsecondary institutions have signed DORA.

The University of Calgary was the first university in Canada to sign DORA in 2021, and is committed to advancing conversations, practices and policies that reflect the diversity and priorities of our research community.

The Canadian Conference on Research Administration

The Canadian Association of Research Administrators annual conference brings together funders and postsecondary research administrators – the individuals who support researchers to navigate funding, ethics, legal contracts, equity, diversity, inclusion and accessibility (EDIA) in research, community engagement and research impact – to discuss updates, expectations, and experiences. This meeting is one of the few existing venues within the research ecosystem for deep conversations about research excellence and culture change.

DORA leaders from the University of Calgary, Université de Montreal, NSERC, SSHRC, CIHR, CFI and Genome Canada co-developed a half-day workshop to raise awareness of what DORA is and what it means for assessing research excellence, provide updates on the implementation of DORA into organizational and funding practices in Canada, and enable opportunities for discussion and idea generation among all participants.

The workshop

The session room was full on May 14, 2023, with 49 participants. Their interest and depth of engagement showed that the time is right for deeper engagement between higher education and funders. Through polling, we learned that around 80% of participants were already familiar with DORA, and 66% said their institutions were aware of DORA (only half of those institutions had signed DORA). When asked if their organization is currently working toward implementing DORA and responsible research assessment practices, 40% said yes while 52% were unsure.

The session opened with an overview of DORA from its Acting Director Haley Hazlett. Vincent Larivière, Université de Montréal, spoke on the evolution of research assessment and the responsible use of metrics, followed by lightning updates from the University of Calgary, CIHR, NSERC, SSHRC, and Genome Canada (view slides).

The second half of the workshop allowed attendees to rotate in groups through four discussion questions:

1. How do you feel the recommendations of DORA will change the definition of research excellence?
2. What support do you need from funding agencies to make this change?
3. What support do you need from your institution to make this change?
4. What are your suggestions for overcoming barriers?

While each table had areas of focus that emerged, there were also a number of overarching themes and take-aways.

Equity, Diversity, Inclusion and Accessibility

Overall, attendees pointed to DORA opening a broader set of possibilities under the umbrella of research excellence and understanding of different value systems such as Indigenous and community-first values. Emphasis on the differences in disciplinary norms and ways of working, as well as values emphasized in different communities, points to the relative ease of change in some fields. Examples of holistic and more qualitative impacts from these fields may help to highlight and visualize the positive impact of DORA-aligned assessment practices. Overall, attendees felt that DORA would lead to a more diverse pool of people deemed successful.

Who is responsible for leading the movement?

Everyone is looking for a leader – be it institutional leaders/executives, research councils, or other organizations – to set the expectations firmly and clearly in a way that is easy for others to follow. We were surprised by the frequency with which attendees mentioned the lack of leadership-level commitment to DORA and feel that this requires further discussion. For DORA to take shape in a complex multi-dimensional system, change and commitment is required across all levels.

What can shift the ecosystem to a new paradigm?

Awareness, education and resources arose as key levers for change. Many funders are updating guidance for merit review and reviewer guidelines. To supplement this, participants suggested that funders provide short, easy-to-read resources with clear guidance for both applicants and reviewers (for example, DO NOT include Journal Impact Factor or h-index in applications; reviewers SHOULD NOT consider these if included). Proper training and education of review committees will be crucial to support integration of these policies into practice.

Institutions could dedicate more time and personnel to raising awareness and look to leadership to sign on to DORA. Obviously, any discussion of research assessment must also include the researchers themselves. Attendees planned to return and have those conversations at their own institutions.

Next steps

DORA encompasses more than we may think initially. In Canada, DORA can be a complementary approach to strategic goals, such as equity, diversity, inclusion and accessibility (EDIA), Open Research/Science, and doing Indigenous research in a good way. Together, we must continue the conversation, sharing the opportunities and “wins” that DORA-aligned research assessment affords and the challenges that emerge along the way.

The organizers of this session are committed to further unpacking what we heard, and working together on practical resources, outputs and engagement opportunities that will benefit the Canadian research community.

The post The DORA Movement in Canada: Working Together to Advance Assessment of Research Excellence appeared first on DORA.

A DORAat10 Local Event Report

In May 2023, DORA celebrated it’s 10th Anniversary with two plenary sessions and a decentralized weeklong program of local events organized by community members from around the world. Event organizers were given the option to write brief reports on their events that summarize key takeaways and recommendations.

By Tilmann Kiessling

A hybrid panel discussion hosted by EMBO and EMBL on 12 May marked the anniversary of DORA, a worldwide initiative aiming to advance approaches to the assessment of scholarly research. The panel discussed issues with current methods of research assessment, as well as solutions and actions for improvement in which panel members have been involved.

Wolfgang Huber, co-Chair of the EMBL responsible research assessment working group, kicked off the discussion. “Research assessment is an integral core aspect of doing science as it helps decide on the recruitment of the next generation of scientists and the allocation of funding,” he said. “Publications are actually not the scholarship itself. They are more of an advertisement of scholarship as the actual scholarship consists of the complete set of reagents or data analysis code, for instance in my field, that generates a paper. We should think about research outputs in a broader way than just the papers,” Huber said. He cited what is known as Goodhardt’s law: Whenever a measure becomes a target, it ceases to be a good measure. The real value would lie in a culture change, Huber said.

Bernd Pulverer, Head of EMBO Scientific Publishing and DORA co-founder, talked about the development DORA underwent from its inception during the 2012 meeting of the American Society for Cell Biology. “At the meeting we quickly converged that the journal impact factor as a single metric is at the heart of some of these issues,” he said. Today, DORA has become a global initiative endorsed by more than 23,000 individuals, institutions, publishers, and funders: “DORA has turned into an advocacy group and is developing tools for more balanced research assessment.” Practices and policies at EMBO include: Applicants for fellowships and grants are not allowed to indicate journal impact factors or other metrics in their applications; instructed reviewers not to use journal impact factors in the evaluation; guidelines for reviewers and applicants are published on the EMBO website; clear-cut conflict of interest policies are applied throughout selection committees; no journal names in files of the candidates shortlisted for the EMBO Gold Medal; reviewed preprints made it realistically possible to assess research outputs at a much earlier stage than journal publications. “And we dropped impact factors for the promotion of the EMBO Press journals,” he said.

Guillermina Lopez-Bendito, from the Institute of Neuroscience in Alicante, Spain, and Chair of the EMBO Young Investigator Committee, emphasized the lack of standardized and comprehensive methods for evaluating quality and impact of scientific work as a major obstacle to advancing research assessment. “Contributions to science go beyond research papers alone. We need to consider mentoring, outreach activities, peer review, and evaluation. We should incorporate assessments of whether researchers have translated their results and discoveries in ways other than publishing,” Lopez-Bendito said. Candidates’ narratives in evaluations are important, as they provide an opportunity for candidates to explain the impact of their research. “DORA is refocusing the attention of reviewers and evaluators on what truly matters, which is the quality of the work.”

Karim Labib, from the University of Dundee and Chair of the EMBO Installation Grant Committee, addressed the assessment of research outside one’s own field. “A key challenge is how best to assess research in areas that one is not extremely familiar with, without relying solely on simple metrics,” he said. Labib supported the idea of interviewing all shortlisted candidates, as it provides equal opportunities for candidates. Labib also emphasized the role of the lead reviewer in interview panels. He advised that lead reviewers should wait until after the interview before sharing their views with the panel, so that the panel members can assess the candidate’s performance in a less biased manner. The panellists agreed with Labib’s view that another bottleneck in research assessment is time. “Scientists are generally interested in participating in research assessment, but lack of time is the primary constraint. “

Brenda Andrews, from the Donnelly Center at the University of Toronto, Canada, and Vice Chair of the EMBL Scientific Advisory Committee, is actively involved in research assessment. As the founding editor of the open-access journal G3 Genes Genomes Genetics, her goal is to publish valuable research findings without considering the impact factor or subjective opinions about the importance of the work. “Senior colleagues bear a significant responsibility in leading by example and changing how we think about research assessment,” Andrews stated. However, she acknowledged that impact factors are still discussed in evaluations. Review committees have become increasingly aware of this issue in recent years. Andrews explained that there is now a clear emphasis on the description of the work and the progress made in setting up labs and training people, rather than solely focusing on the publication venue.

Cecilia Perez, postdoctoral researcher at EMBL, became interested in topics related to social justice in research assessment during her PhD studies. She shared her experiences when applying for scholarships and the biases present in assessment processes. “I would like to highlight the arbitrary nature of assessment processes and emphasize the need for fairness to achieve greater diversity and equality,” Perez expressed. She reflected on the challenges of organizing authors on papers, particularly in collaborative projects that are becoming more common agreeing that we need to move away from simplistic assessments based on authorship order and focus more on author contributions.

The panel discussion on the occasion of the DORA anniversary was co-organized by EMBO (Sandra Bendiscioli, Senior Policy Officer) and EMBL (Katherine Silkaitis, Strategy Officer) and chaired by Sandra Bendiscioli.

This is a cross-post (original post here) that has been lightly edited for length. The quotes from the panel discussion were edited.

The post How adorable is DORA? appeared first on DORA.

With the evolution of a research landscape driven by technological progress and societal needs, there has been a growing demand to rethink and reform research assessment that could enhance the quality and culture of a scholarly ecosystem. The Declaration on Research Assessment (DORA) is a worldwide initiative to address these needs.

To celebrate its 10th Anniversary, DORA hosted a plenary session for the Africa, Americas, and Europe region on Tuesday 16 May 2023. The event started with opening remarks from Stephen Curry, DORA’s Chair, followed by a keynote address by Sarah de Rijcke, Leiden University, Netherlands. The second half of the session was composed of a panel discussion by Andiswa Mfengu of University of Cape Town, South Africa; Karen Stroobants, Vice-Chair CoARA, UK; Judith Sutz of Universidad de la República, Uruguay; and Stephanie Warner of University of Calgary, Canada. DORA’s Acting Program Director, Haley Hazlett, shared closing remarks at the end of the session. Chris Hartgerink, Liberate Science, and Queen Saikia, DORA Policy Associate, provided technical support during the session.

Opening remarks: DORA into the next decade

To recognize the depth and breadth of research and innovation talent, the scholarly community needs to advocate for best practices with an understanding that responsible research assessment is a systems challenge composed of interlocking and intersecting issues. Stephen Curry began the plenary session with a discussion of the history of DORA, which was drafted in 2012 at the annual meeting of the American Society of Cell Biology (ASCB) in San Francisco and published in 2013. Ten years later, DORA has been signed by over 23,000 signatories worldwide and its recommendations have been adopted by research funders, publishers, academic institutions and professional societies. Curry highlighted one key and recent initiative of DORA, the Tools to Advance Research Assessment (TARA)  Project, which aims to support community implementation of responsible academic assessment through the creation of tools and resources for hiring, promotion, and tenure decisions. The key objectives of Project TARA include an interactive web resource (Reformscape) which will help scholarly actors discover policies and processes for the implementation and uptake of responsible research assessment (RRA) practices worldwide. Project TARA also includes a survey of US institutions to understand their approaches to RRA, and a toolkit to address identified issues, which is currently under its development process. Details about the significant milestones and activities that have shaped DORA’s progress can be found in the recent retrospective blog.

Keynote address: The future of research evaluation

The future of research evaluation is likely to encounter unforeseeable changes driven by advancements in technology, evolving scholarly practices, and the need for more comprehensive and transparent assessment methods. In her keynote lecture, Sarah de Rijcke pointed to a few trends and possibilities that could shape the approaches to future assessment reform. For example, de Rijcke discussed the move to open science, technological developments such as Artificial Intelligence (AI), and the rise of social media. Despite the perks that technological advancements could bring, research communities, governments, civil societies and other actors governing the space should keep thinking about how these align with their values and norms around, for instance, integrity.

Signing DORA is a mechanism to demonstrate support for RRA and, for organizations, signals intent to implement reform. Over 2800 organizations from across the world have signed DORA, many of them in Europe. One striking trend that has been consistently observed among DORA signatories is in the relative lack of signatories from the US. For example, as of June 2023 there are 258 organizational signatories in the UK and 166 in the US. This suggests a need to understand institutional attitudes and approaches to qualitative assessments over quantitative indicators in the US. To meet this need, Alex Rushforth from Leiden University developed a survey and held interviews as part of Project TARA that spotlighted plurality in opinions of US researchers and evaluators. de Rijcke, also a member of the Project TARA team, outlined these results. Some respondents showed strong endorsement towards responsible metrics; some showed partial alignment but were unprepared to make it happen; and a fraction of them showed pragmatic rejection expressing no desire to move away from research evaluation “norms”. The survey results suggest that it is essential to raise awareness among US institutions to adopt evaluation principles aligning with the values of RRA, scientific excellence, innovation and the broader goals of the society. However, it is noteworthy that some other initiatives such as HELIOS, PTIE, and HuMetricsHSS have been successfully accepted in the US and are actively working to address these challenges. On being asked if we need a consensus before driving for a change in the context of US, and how DORA could help: de Rijcke said that consensus is need-driven and there was no overall consensus from the US study, however there’s a progressive change for the ones that are supportive of research culture. DORA can help by asking questions, putting forward concrete practices, and collaborating with US-based initiatives.

de Rijcke later brought the audiences’ attention to the discussion paper, The Future of  Research Evaluation that presents an outlook on research assessment reforms across the globe. Out of several case studies, de Rijcke highlighted a case study on China’s biggest funding organization, the Natural Science Foundation of China (NSFC),that reflects important shifts in the systemic reforms by moving away from bibliometrics to a system that encourages curiosity driven research, to fit the broader goals of the society.

Panel discussion: Research assessment practices across the globe

The second half of the plenary session was focused on understanding RRA and the impact of DORA in a range of different regional contexts. Andiswa Mfengu shared that changes are happening but currently in a budding stage around her region, and that her University of Cape Town is engaged in fostering conversations and setting up working groups with a view to creating awareness and developing an impact assessment framework for holistic evaluation of researchers. However, commitments and translational changes are still afar, and it is still a challenge to rule out systemic norms like ‘publish and perish’ that govern University rankings. According to Mfengu, initiatives like DORA can serve as an ‘idea of hope’ that can inspire and support researchers are working to champion RRA. Mfengu envisions research assessment to be laying fairer principles in future, by decolonizing knowledge systems in order to meet local needs of the community.

Research assessment reforms vary across different regions of the world due to diverse institutional attitudes, cultural norms and priorities. In Europe, as Karen Stroobants mentioned, organizations are starting to move away from heavy reliance of quantitative metrics like university rankings, the JIF, H-index etc., to more holistic evaluation practices such as use of Narrative CVs and evaluating societal impact.  Along with DORA, initiatives like CoARA are driving increased momentum for assessment reform in (and increasingly beyond) Europe. Some countries, including the Netherlands, the UK, Finland and Norway have already progressed considerably in the implementation of new approaches. When asked what she envisioned for the future of research assessment, Stroobants foresees 1) assessment practices that would balance between achieving organisational strategies and leaving space for individual creativity and 2) organizations and individuals not just to be assessed   based on their track record but also on their potential and competencies to create a future impact.

Judith Sutz discussed that RRA in Latin America has been and continues to develop heterogeneously. There is great appetite for reform supported both by researchers and by international organizations like CLACSO-FOLEC. Public universities are gaining autonomy from government restrictions and are driving research and excellence these days. Sutz mentioned that in Uruguay, DORA has helped set up alternative indicators of research evaluation such as qualitative narration of work progress as opposed to quantitative scales. On being asked her vision on research reforms in next years, Sutz laid out seven points:

1. “Narratives will become the standard way of documenting research performance
2. Researchers should be encouraged to build research problems based on the needs of common people with the help and guidance from non-academics, government experts or civil society. This is important because “unearthing” problems is a complicated task and requires a researcher’s involvement, otherwise suffering of common people will continue to be invisible to research
3. Ethical aspects of research will thoroughly be assessed and researchers would be asked to reflect upon those aspects
4. A sound heterodoxy will be rewarded, the power of mainstream scholars would be diminished, and experimentation (in the sense of following uncertain paths) will be encouraged through an evaluation system that assesses the processes not the outcomes
5. A vibrant international community of scholars through common procedures of RRA will be free to set research agendas that would address fundamental problems of time without any fear of punishment in choosing too difficult problems in small harvest of papers
6. Many more women will be enrolled and enjoy doing research because harmonization between work and life will be at last be possible
7. Everybody will find a place in the wonderful enterprise of research, every talent will be appreciated, and the differences notwithstanding will be appreciated in their due value”

Judith Sutz

The Canadian academic environment too is gaining momentum on holistic assessment. Several national funding organizations are now signatories of DORA and have incorporated DORA’s recommendations into their practices, along with academic institutions, including the University of Calgary. These organizations are recognizing different pathways to impact, as well as understanding what ‘impact’ or ‘high quality’ means in different disciplinary contexts. Stephanie Warner hopes to see a more equitable and transparent research landscape in a decade’s time, with open practices and policies for individuals to be chosen on the basis of their merit by committees well-versed in metrics literacy, and where there will be true respect of different world views, particularly Indigenous and community values.

Conclusion

Conversations surrounding RRA should be a cultural norm within the research community, especially among young researchers that are currently at the forefront of driving culture change tomorrow. There was strong agreement between all the panellists that conversation and awareness alone is not enough; training, capacity building, and understanding research literacy is equally important. At the same time, local advocates of systemic changes should be encouraged to ask questions, carry their values forward, and stand strong even in solidarity. DORA will be working in the coming years to address just such points.

Reading List

Non-DORA resources referenced or shared during the call:

1. Canadian funders and DORA: https://www.nserc-crsng.gc.ca/NSERC-CRSNG/policies-politiques/DORA-DORA_eng.asp
2. University of Calgary and DORA https://research.ucalgary.ca/research/our-impact/DORA
3. University of Cape Town’s statement for the principles of DORA https://uct.ac.za/research-support-hub/outputs/research-assessment
4. Coalition for Advancing Research Assessment https://coara.eu/
5. The Latin American Forum on Research Assessment (FOLEC-CLACSO) https://www.clacso.org/en/folec/

DORA resources:

1. The Declaration on Research Assessment: https://sfdora.org/read/
2. Fritch R, Hatch A, Hazlett H, and Vinkenburg C. (2021). Using Narrative CVs. https://zenodo.org/record/5799414#.YeM-41lOlPY
3. Ginny Barbour and Haley Hazlett (2023). From declaration to global initiative: a decade of DORA. https://sfdora.org/2023/05/18/from-declaration-to-global-initiative-a-decade-of-dora/
4. Anna Hatch, Ginny Barbour and Stephen Curry (2020). The intersections between DORA, open scholarship, and equity. https://sfdora.org/2020/08/18/the-intersections-between-dora-open-scholarship-and-equity/

Queen Saikia is DORA’s Policy Associate

The post DORA at 10: A look at our history and the bright future of responsible research assessment (Africa, Americas & Europe Plenary) appeared first on DORA.

A DORAat10 Local Event Report

In May 2023, DORA celebrated it’s 10th Anniversary with two plenary sessions and a decentralized weeklong program of local events organized by community members from around the world. Event organizers were given the option to write brief reports on their events that summarize key takeaways and recommendations.

By Gerald Beasley, Sally Wilson, Jo Wixon, Rebecca Kirk, Victoria Gardner (HESI SDG Publishers Compact Fellows); Shane Rydquist (Cactus); Monica Granados (Open Climate Campaign); Tim Koder and Joana Osório (Open Pharma)

Since its inception, the Declaration on Research Assessment (DORA) has made great progress in stimulating discourse and driving change around academic reward and incentives structures, moving away from a focus on the journal Impact Factor as a proxy for the quality or impact of research to a broader concept of the impact of scholarly outputs of research. The Declaration has attracted a huge number of signatories over the past ten years and provides them with a forum to share ideas and best practice, and to work collaboratively to foster change. 

In the same spirit, the HESI SDG Publishers Compact Fellows formed a collaboration and convened an event in partnership with the Open Climate Campaign and Open Pharma to provide advice and guidance to researchers on writing for non-academic audiences. Our focus was on highlighting why writing for non-academic audiences is vitally important to drive progress towards the UN’s Sustainable Development Goals, and aligned with the aims of DORA in terms of improving the ways in which the outputs of scholarly research are evaluated. We agree on the need to move beyond the current academic focus of impact to a greater understanding and recognition of the impact that research can have on the broader world. We all believe that research can help us solve the challenges that we are dealing with globally. The mindset change that is is at the heart of DORA is essential as we collectively work to solve these challenges – be that good health and well being, climate change, food security, inequality, or any of those reflected in the Sustainable Development Goals and more widely. 

Our event on May 17 was titled: How to write for non-academic audiences to achieve progress towards the UN Sustainable Development Goals (SDGs). We defined non-academic audiences as any reader who does not share the specific academic knowledge of the research output. This includes the general public, practitioners, patients, advocacy organisations, and policy or decision makers. During the session, our roster of speakers and facilitators provided institutional, industry, advocacy, and publishing perspectives on how to help research do more to reach non-academic readers and users. For example, writing plain language summaries or succinct practitioner or policy action points. We covered why writing for these audiences is important, how it can help widen the reach and impact of research, and outlined some practical tools and tips to support the writing process, including Top Tips created by the HESI SDG Publishers Compact Fellows. Participants had the opportunity to try some writing of their own, based on a worked example, which gave them the chance to put some of the tips and guidance of the session into practice. We had high levels of engagement from our audience throughout the event, who shared their thoughts and experiences with the group. The slides and handout with links to related resources can be found on the SDG Publishers Compact Fellows website.

The organisers didn’t just convene the event, but learnt a lot from our audience members, who were generous and candid with sharing their thoughts and experiences, including barriers to communicating in this way for these audiences. 

The convenors came away from the session with ideas on future events (including a possible debate on the role of AI in supporting with writing these formats), and with a clear sense of the challenges around writing for non-academic audiences. Ensuring that rewards and incentives structures, alongside training and support, are in place to support researchers in this endeavour is one challenge. Many attendees noted that while they were keen to communicate with wider audiences, current incentives structures do not encourage them to devote time and energy to writing these kinds of outputs. DORA is playing a vital role in addressing this and there is lots more that can be done. 

We are keen to accelerate change and help research to realise its possible real world impact. We plan further collaboration across our groups and DORA as we develop practical solutions to support broader forms of impact and engagement, including writing for non-academic audiences to drive progress towards the SDGs. Keen to find out more and to support our goals?  Join the SDG Publishers Compact Fellows Group Virtual Community for news about  activities and future events and to take part in our work to support the aims of DORA and drive progress towards the SDGs!

The post How research assessment reform can help research to do more! Reflections from the  SDG Publishers Compact Fellows, Open Climate Campaign, and Open Pharma appeared first on DORA.

A DORAat10 Local Event Report

In May 2023, DORA celebrated it’s 10th Anniversary with two plenary sessions and a decentralized weeklong program of local events organized by community members from around the world. Event organizers were given the option to write brief reports on their events that summarize key takeaways and recommendations.

Scroll down for the English translation of this report: Advances in gender equity in academic evaluation: Panel “Gender and science evaluation“

By Sistemas Conocimientos Venezuela

El pasado 19 de mayo del 2023, se celebró un panel en conmemoración del décimo aniversario de DORA (Declaración de San Francisco sobre la Evaluación de la Investigación). ). El evento fue realizado en modalidad virtual y co- organizado por FOLEC/CLACSO, el Centro de Estudio de Trasnformaciones Sociales/ IVIC, la Red de Antropologías del Sur, el FIS Diversidad y el GT CLACSO “Ciencia abierta como bien común”. Reunió a expertas en temas de evaluación académica y ciencia, para analizar y discutir propuestas en mejora de la equidad de género en la evaluación académica y en la investigación científica.

Laura Rovelli, coordinadora del FOLEC CLACSO e investigadora argentina de la Universidad Nacional La Plata y el CONICET, centró su participación en analizar los incentivos a la equidad de género en materia de evaluación académica. Rovelli abordó las distorsiones observadas en la presencia de mujeres en la producción académica, como criterios sesgados, subrepresentación en procesos de evaluación e inequidad de resultados, y propuso introducir la perspectiva de género en tres dimensiones: criterios de evaluación, procesos de evaluación y resultados obtenidos. También presentó hallazgos sobre cómo algunas agencias de financiación de la región consideran la perspectiva de género en la asignación de fondos, promoviendo la paridad en grupos de investigación y comités de evaluación, y teniendo en cuenta aspectos relativos a las tareas de reproducción social que ocupan principalmente a mujeres.

Por su parte, la profesora uruguaya, adjunta de la Unidad Académica de la Comisión Sectorial de Investigación Científica (CSIC) de la Universidad de la República, UDELAR, Cecilia Tomassini, presentó una ponencia titulada “¿Cómo afecta la maternidad y la paternidad las trayectorias académicas? Evidencias a partir del caso de Uruguay”. En esta investigación, donde se evaluó cómo las responsabilidades de cuidado afectan la carrera académica de las mujeres y hombres en Uruguay, considerando aspectos sociodemográficos en comparación con resultados académicos obtenidos; se muestra que el ámbito de productividad por publicaciones es donde se ha evidenciado más una penalización de la carrera científica de las mujeres en función de las cargas de cuidado, en especial en relación con el número de hijos.

La investigadora propuso incorporar otros tipos de trayectorias académicas válidas y premiadas que permitan apuntalar procesos de corresponsabilidad sobre los cuidados. En conclusión, la investigación de Tomassini destacó la importancia de considerar la perspectiva de género en la carrera académica y la necesidad de políticas que apoyen a las mujeres en la continuidad de su carrera de investigación.

Florencia Fiorentin, argentina y becaria doctoral CONICET, perteneciente al Instituto de Industria, Universidad Nacional de General Sarmiento (IdeI–UNGS), doctoranda en Economía, magíster en Gestión de la Ciencia, la Tecnología y la Innovación, y licenciada en Economía Política (UNGS); participó con una ponencia titulada: “El efecto Matilda en la política científica y tecnológica. Brechas de género en la asignación en los Proyectos de Investigación Científica y Tecnológica”. En su participación, mostró hallazgos de cómo el efecto Mateo solo se evidencia en varones, siendo más alto en estos que en mujeres. Tuberías con fugas y techos de cristal: Fiorentin enfatizó que las mujeres publican menos que los varones y alcanzan escalafones más altos en menor proporción, aunque en el ámbito científico, en general, existe paridad. Coexisten varias brechas, dijo: salarial, de evaluación y de participación en el mercado laboral, entre otras.

Fiorentin destacó que la política de Ciencia y Tecnología (CyT) no pone énfasis en estas brechas preexistentes, las cuales se refuerzan entre sí. Frente a esto, apuntó, es importante no solo conocer el impacto de las políticas, sino también atender las etapas previas de evaluación y asignación. Al cierre, Fiorentin sostuvo que se requieren políticas transversales para alcanzar la equidad, ya que existen brechas de género previas al inicio de la carrera académica, así como hacen falta indicadores con perspectiva de género para abordar estas problemáticas de manera efectiva.

El foro panel regional, conmemorativo del 10º Aniversario de DORA, se cerró con la participación de las investigadoras venezolanas Ximena González-Broquen (CETS-IVIC), Eisamar Ochoa Contreras (CETS-IVIC) y Annel Mejías Guiza (ULA-Red de Antropologías del Sur), quienes presentaron una ponencia titulada “Apuntes sobre Género y evaluación científica en la Red de centros CLACSO de Venezuela”. Esta investigación, realizada con el apoyo de DORA durante el 2022 y en la cual participó también Mariángela Petrizzo Páez como coordinadora (UNATUR – IAET), analizó la representación de género y los enfoques feministas en los centros de investigación de la Red CLACSO en Venezuela.

Estas investigadoras destacaron que, a pesar de que la mayoría de los centros están dirigidos por mujeres, sólo el 7% de los temas abordados se enmarcan en el feminismo. Esto demuestra que la participación de más mujeres no garantiza necesariamente una mayor igualdad de género en la investigación científica. Sin embargo, los centros de la red CLACSO muestran un interés en avanzar hacia una ciencia más justa e inclusiva, incorporando indicadores como la equidad de género, la interseccionalidad, la pertinencia social y la ciencia abierta en sus procesos de evaluación académica.

Además, un número importante de centros participantes en el estudio afirmaron estar abiertos a incluir a miembros de las comunidades en los procesos de evaluación, incorporando la perspectiva de género y favoreciendo prácticas científicas situadas en cuanto a los objetos de estudio y los métodos utilizados. La falta de mecanismos de evaluación en algunos centros brinda una oportunidad para introducir una evaluación responsable que atienda las consideraciones de género y supere las dinámicas excluyentes establecidas por los procesos de evaluación académica dominantes.

Las expertas concluyeron que es necesario un cuestionamiento profundo del sistema para considerar los procesos de reproducción social de la vida como elemento transversal en la producción académica y reconocer la diversidad de pensamiento y fuentes de producción de conocimiento. Este enfoque, dijeron, pudiese contribuir a una ciencia más inclusiva y equitativa en Venezuela y en la región latinoamericana en general.

El video completo, con las participaciones del foro panel, fue transmitido por el canal CLACSO TV de YouTube, y puede ser visto desde este enlace: https://www.youtube.com/live/Eh8aDgssVrM?feature=share

Advances in gender equity in academic evaluation: Panel “Gender and science evaluation”

On May 19, 2023, a panel was held to commemorate the tenth anniversary of DORA (San Francisco Declaration on Research Assessment). The event was held online and co-organized by FOLEC/CLACSO, the Center for the Study of Social Transformations/ IVIC, the Southern Anthropology Network, the FIS Diversidad and the GT CLACSO “Open Science as a Common Good”. It brought together experts in academic evaluation and science to analyze and discuss proposals to improve gender equity in academic evaluation and scientific research.

Laura Rovelli, coordinator of FOLEC CLACSO and Argentine researcher at the National University of La Plata and CONICET, focused her participation on analyzing incentives for gender equity in academic evaluation. Rovelli addressed the distortions observed in the presence of women in academic production, such as biased criteria, underrepresentation in evaluation processes, and inequity of results, and proposed introducing the gender perspective in three dimensions: evaluation criteria, evaluation processes, and results obtained. She also presented findings on how some funding agencies in the region consider the gender perspective in the allocation of funds, promoting parity in research groups and evaluation committees, and taking into account aspects related to social reproduction tasks that mainly involve women.

Cecilia Tomassini, Uruguayan professor, and assistant of the Academic Unit of the Sectorial Commission for Scientific Research (CSIC) of the University of the Republic, UDELAR, presented a paper entitled “How do motherhood and fatherhood affect academic trajectories? Evidence from the case of Uruguay”. This research evaluated how caregiving responsibilities affect the academic careers of women and men in Uruguay, considering sociodemographic aspects in comparison with the academic results obtained. It shows that the area of productivity through publications is where there is more evidence of penalization of women’s scientific careers in terms of caregiving responsibilities, especially in relation to the number of children.

The researcher proposed incorporating other types of valid and rewarded academic trajectories that allow underpinning processes of co-responsibility for care. In conclusion, Tomassini’s research highlights the importance of considering the gender perspective in academic careers and the need for policies that support women in the continuity of their research careers.

Florencia Fiorentin, Argentinean and CONICET doctoral fellow, belonging to the Institute of Industry, the National University of General Sarmiento (IdeI-UNGS), Master in Science, Technology and Innovation Management, and BA in Political Economy (UNGS); participated with a paper entitled: “The Matilda effect in science and technology policy. Gender gaps in the allocation of Scientific and Technological Research Projects”. The Matilda effect is a bias misattributing women’s achievements to their male colleagues.

In her participation, she showed findings on how the Matilda effect is only evident in males, being higher in males than in females. Leaky pipelines and glass ceilings: Fiorentin emphasized that women publish less than men and reach higher echelons in a lower proportion, although in the scientific field, in general, there is parity. Several gaps coexist, she said: salary, evaluation, and participation in the labor market, among others.

Fiorentin pointed out that Science and Technology (S&T) policy does not emphasize these pre-existing gaps, which reinforce each other. In view of this, she said, it is important not only to know the impact of the policies but also to pay attention to the previous stages of evaluation and allocation. In closing, Fiorentin said that cross-cutting policies are needed to achieve equity since gender gaps exist prior to the start of the academic career, and indicators with a gender perspective are needed to address these issues effectively.

The regional panel forum, commemorating DORA’s 10th Anniversary, closed with the participation of Venezuelan researchers Ximena González-Broquen (CETS-IVIC), Eisamar Ochoa Contreras (CETS-IVIC) and Annel Mejías Guiza (ULA-Red de Antropologías del Sur), who presented a paper entitled “Notes on Gender and scientific evaluation in the Network of CLACSO centers in Venezuela”. This research analyzed the representation of gender and feminist approaches in the research centers of the CLACSO Network in Venezuela. It was carried out with the support of DORA in 2022 and in which Mariángela Petrizzo Páez also participated as coordinator (UNATUR – IAET).

Despite the fact that most of the centers are directed by women, only 7% of the topics addressed are framed within the framework of feminism. This shows that the participation of more women does not necessarily guarantee greater gender equality in scientific research. However, the CLACSO network centers show an interest in advancing towards a more just and inclusive science, incorporating indicators such as gender equity, intersectionality, social relevance, and open science in their academic evaluation processes.

In addition, a significant number of centers participating in the study stated that they were open to including community members in the evaluation processes, incorporating the gender perspective and favoring situated scientific practices in terms of the objects of study and the methods used. The lack of evaluation mechanisms in some centers provides an opportunity to introduce responsible evaluation that addresses gender considerations and overcomes the exclusionary dynamics established by the dominant academic evaluation processes.

The experts concluded that a profound questioning of the system is necessary to consider the processes of social reproduction of life as a transversal element in academic production and to recognize the diversity of thought and sources of knowledge production. This approach, they said, could contribute to a more inclusive and equitable science in Venezuela and in the Latin American region in general.

The complete video, with the panel forum’s participation, was broadcasted on the CLACSO TV YouTube channel and can be viewed from this link: https://www.youtube.com/live/Eh8aDgssVrM?feature=share

The post Avances en equidad de género en la evaluación académica: Panel “Género y evaluación de la ciencia” appeared first on DORA.