Blog
DORA updates to BHF research assessment
It has been one year since the BHF signed the San Francisco Declaration of Research Assessment (DORA) - a set of recommendations designed to ensure that the quality and impact of scientific outputs are measured accurately and evaluated appropriately. By ensuring the DORA principles are embedded in BHF’s research funding and evaluation practices, we are working to make our processes more inclusive and transparent. In becoming a DORA signatory and adopting the principles, our aim is to help support our researchers to develop to their full potential and encourage nurturing and open research environments for good science and research integrity to thrive.
One year on – our work to date
DORA principle: Transparency in how funding applications and grants are assessed. Applicants should understand how their grant applications are assessed, and once awarded, how progress is evaluated.
Improving transparency in how funding decisions are made at the BHF has been a key goal in our internal review. As a start, we have formalised guidelines for good practice in expert review and will make examples of our review forms available on our website. By helping applicants understand how their proposals are assessed and providing clear guidance to expert reviewers, we are working towards a more inclusive, more transparent expert review system. In combination with publishing application pathways for all our funding committees, we are helping to clarify how the BHF reaches its funding decisions.
Whilst we try to minimise administrative burden for researchers, many of our funding schemes require the completion of reporting forms, such as milestone, midterm, or annual reports. How these are assessed post submission is often unclear and can vary between funding schemes. Our reporting progress webpages have been updated to include information on all types of progress and final reports, and clarity on how these reports will be assessed. Similarly, our research evaluation webpage has been updated to help researchers understand how we use their Researchfish submissions.
DORA principle: Responsible use of bibliometrics. Guarding against the inappropriate use of journal impact factor is the founding principle of DORA. A research article should be judged by its own scientific merit, not that of the journal that it is published in.
Over the course of the next two committee funding cycles, the DORA principles will be introduced and embedded in our committee procedures, with a statement that outlines the DORA principles and why these are important at the BHF. The statement will be highlighted by the committee Chair or Vice Chair, who with support from the BHF staff team, will ensure that committee discussions maintain the DORA principles and values, such as ensuring an applicant’s work is not assessed based on the impact factor of the journal it is published in.
DORA principle: An appreciation of the value of all research outputs. Narrow measures of academic success have been shown to have a detrimental influence on research integrity and environment. A greater appreciation of all research outputs is therefore recommended.
In the next phase of our work, we will be reviewing BHF application forms to take into account a more diverse range of research outputs, so that we consider not just ‘what’ researchers have achieved but ‘how’ they have achieved it. In the near future, rather than asking applicants to simply list their publications, we will be asking applicants to articulate their specific contribution to the publication. In many of our application forms, publication lists will be complemented with a request for wider research outputs e.g., public or policy-maker engagement.
Last year, we also agreed – together with UKRI, NIHR, Wellcome and Cancer Research UK and others - to explore the use of ‘narrative CVs’ in grant applications. Narrative CVs, which are currently being piloted by other funding bodies such as the NIHR, involve individuals providing written overviews of their career path and the achievements and contributions arising from the activities they have been involved in – allowing them to showcase a broader range of outputs and outcomes from their work, beyond factors such as their number of publications or grants. A draft BHF hybrid CV (part narrative CV, part traditional academic CV) has been developed based on the Royal Society’s Resume for Researchers format. However, it is important that we work with our researchers on this initiative and any subsequent changes to the BHF CV. To this end, in Winter 2022 we will be reaching out to our research community to help us understand what the challenges are in introducing the hybrid CV and how we can help smooth the adoption for researchers in our funding processes.
Beyond DORA – EDI at the BHF
The BHF funds over half of all non-commercial cardiovascular research in the UK. As a major funder, we recognise the part we need to play in nurturing a positive research culture and supporting an inclusive and diverse research community. Our DORA work is part of a broader commitment to improve equality, diversity and inclusion (EDI) in cardiovascular research and the research workforce. In May 2022, we launched our 3 year EDI strategy, which outlines our specific ambitions and objectives around addressing inequalities in the current system.
If you have any questions about our work on DORA please contact the Research team at [email protected].