Embedding DORA: improving how a research track record is considered in BHF funding decisions

Towards our goal of embedding a greater appreciation of the value of all research outputs, from January 2025 BHF is amending how we ask applicants to describe their track record in research funding applications.

In 2021, BHF signed the San Francisco Declaration of Research Assessment (DORA) and committed to the DORA principles. These principles include transparency in how funding decisions are made, responsibility in the use of bibliometrics and a greater appreciation of the value of all research outputs. At the core of the DORA principles is the desire to ensure that the quality and impact of scientific outputs are measured and evaluated appropriately. By becoming a signatory and publicly committing to these principles, BHF aims to help support researchers to develop to their full potential and encourage nurturing and open research environments for good science and research integrity to thrive. This article outlines our latest efforts, working to fully embed the DORA principles in BHF funding and research evaluation processes.

Capturing wider outputs

Working to embed a greater appreciation of the value of all research outputs, BHF has reviewed how an applicant evidences their track record in its application forms. Whilst some BHF application forms offer applicants the opportunity to highlight wider research outputs, most application forms focus on publication and grant histories. To better capture evidence for an applicant’s track record, BHF has revised some targeted questions within its FlexiGrant grants management platform; in particular the universal “My CV” section, case for support and the main application form.

Most application form amendments, detailed below, will empower applicants to evidence their track record using any relevant research output, rather than publications alone. Furthermore, research outputs will be limited to six, encouraging applicants to consider the relevance and impact of the research outputs described. Limiting the number of research outputs to six should also balance the additional effort to tailor CVs to individual applications. To help applicants emphasise their involvement in creating the research output, applicants will be asked to explain in a single sentence their contribution to the research output.

Applicants will also be given the opportunity to describe any factors that provide additional context in the assessment of their track record. This section will not be mandatory but will give applicants the chance to describe any career breaks, part-time working, time taken out of active research to pursue training or other relevant experiences that may have impacted their track record. From early 2025, BHF will make the following changes to the application forms for BHF research grant funding: 

“My CV” section

• Publication history – Rather than listing up to 30 papers from the past 10 years, applicants will be asked to evidence their ability to deliver the proposed work through six relevant research outputs, which may include research papers. They should articulate in a single sentence their contribution to each research output. Applicants will be asked to provide a publicly accessible link to their full publication list. This link may be an institutional webpage, PubMed/Europe PMC search link, ORCID, Google Scholar or another relevant platform.
• Additional information – Applicants will be given the opportunity to provide additional context for their track record. This section is not mandatory and should not be used to describe additional skills, scientific experience or outputs.
• Memberships and honorary contracts – To better understand any conflicts of interest, applicants will be asked to declare any honorary contracts outside of their host institution, any funding Committee/Board memberships or Editorial board membership.
  
  

Case for Support (applicable to some funding committees)

• Chairs and Programme Grants committee – For most schemes, the primary applicant and co-applicants will be asked to provide up to six relevant research outputs and grants, and articulate in a single sentence their contribution to each research output.
• Fellowships, Project Grants and Translational Award committees – Information from the “My CV” section will be included in the application papers, so additional information will only be requested when required. PhD studentship application forms will also request research output information for proposed named students.
• Clinical studies committee – Application forms will continue to ask for a maximum of six most recent/relevant publications and six grants per applicant.
    

Main application

• Publications from previous funding – Similar to the “My CV” section, when the application is a continuation of previous BHF funding, applicants will be given the opportunity to report on broader research outputs.
  
  

Considering the Narrative CV

The Narrative CV, in particular the Royal Society’s Résumé for Researchers, offers researchers the opportunity to showcase their contributions to research in a systematic and standardised manner. Aligned with the DORA principle of ‘a greater appreciation of the value of all research outputs’, the Narrative CV holds the potential to capture a wide range of research outputs. 
However, whilst the Narrative CV could enhance the value of non-traditional research outputs, there has been some concern that certain groups may be disadvantaged by its use. For example, scientists for whom English is not their first language may struggle to describe their contributions as eloquently as native speakers. Moreover, there is evidence that women and applicants from some ethnicities may be more modest, and less likely to promote their achievements. These concerns could be more of an issue in longer narrative answers compared to a traditional CV that has a list of roles and publications.

BHF evaluated the use of the Narrative CV alongside other funders  and recognised that its adoption would be a major change in how BHF evidences an applicant’s track record and ability to deliver a project. As such, we felt that any decision should be made in consultation with our BHF research community. In 2023, we launched a consultation on the Narrative CV, which included an anonymous survey and themed discussions during BHF Fellows and Annual Committee Members meetings. 

The results of the consultation confirmed that implementation of the Narrative CV would be divisive, and overall, results were inconclusive for the acceptability of the Narrative CV to our researchers. For example, early to mid-career researchers were somewhat open to introducing the Narrative CV, but raised serious concerns regarding additional workload and pressures to “fill blank boxes”. Furthermore, more established researchers felt the Narrative CV would not reduce bias and improve equity, diversity and inclusion, but may add to inequities for applicants who are less confident in writing in English.

BHF has therefore decided not to implement the Narrative CV at this stage, but will review this position in 2026. Whilst it is clear that the Narrative CV offers applicants and independent expert reviewers an opportunity to consider a wider range of skills, experiences and research outputs, there is a lack of evidence on the wider implications for inclusion and unintended negative consequences. Therefore, in the short term, BHF has focused instead on amending targeted questions in our existing application forms, to help applicants to showcase the breadth of their track record.

Overall progress with DORA

Since becoming a DORA signatory in 2021, we have worked to ensure the DORA principles are embedded in our research funding and evaluation processes. To this end, we conducted an internal review to assess opportunities for DORA implementation. As outlined in the figure below, this entailed looking at an applicant’s journey from submission to outcome to monitoring research outputs.

BHF progress with DORA at a glance (green sections to be introduced from January 2025).

Following this review, BHF has been working to improve transparency and has published application pathways, funding committee memberships, conflicts of interest policy, independent expert review guidelines and review forms. By making these documents publicly accessible, we want to help our researchers understand how their proposals will be assessed, before they apply. Similarly, we have updated reporting progress and research evaluation webpages to include information on all types of BHF progress and final reports. We hope that this increased transparency around reporting will help our researchers better understand how BHF uses their reports and Researchfish submissions.

Finally, our independent expert review guidelines include a reminder of the BHF expectations of our independent expert reviewers and highlight that written reviews will be included in feedback to the applicants. Alongside our efforts to increase transparency, we have also started an evaluation of our review forms. This will include a review of the current confidential comments section, which is excluded from applicant feedback, and the overall language used in the form. We expect that amendments to BHF review forms will be introduced in 2025.