Proposal evaluation is a key first step in deciding the success or otherwise of a research project. Most funding bodies work with peer review as their main evaluation mechanism. Research proposals are normally written by researchers and most evaluators are active researchers, therefore there can be a competence gap in evaluating CSO participation. Additionally, one way of ensuring that CSO involvement is valued and explicitly considered by project partners is to separately focus on CSO engagement during post-project evaluation of projects.
If funders want to ensure that CSO engagement is planned and implemented in a way that fulfils their objectives they should make sure that the individuals evaluating the proposals and final reports both understand those objectives, as well as the practicalities of involving CSOs in research. It is important that evaluators understand the nature and motivations of CSOs and the organisational practices that can determine the success or failure of CSO collaboration. This means that evaluators need training and instruction to ensure that they can evaluate relevant proposal appropriately. Further information on this front is contained within the Evaluators and Reviewers recommendations section.
What are the expectations around evaluating the success of the CSO involvement? Is this explicit within the funding call?
Are the existing evaluators aware of the issues relating to CSO involvement in research, or is it necessary to provide separate guidance / briefing?
Is the existing evaluation pool broadly representative of stakeholders' interests; should more civil society representatives be encouraged to contribute?
In one of the national funding streams investigated in one of the CONSIDER case studies, the funder set up a panel consisting of users and their representatives with a view of ensuring that their views were considered. The funding stream focused on technology development for dementia patients and the panel represented patients, carers and other stakeholders involved in dementia care. This user panel that comprised several CSO representatives evaluated both proposal and project outcomes.
'I think there were about - I am not sure but maybe 50 projects or so we advised. And the good thing was that the scientists were thinking 'oh yeah we have to go to this group, it’s important for them so we can arrange our research in this way. I think it’s a success because now they come voluntarily to us for advice'. (CSO Representative)