Monitoring and evaluation
Monitoring is the crucial routine collection of information about the work involved.
Evaluation is the collection of judgements you make about your service or project, how efficiently it is working and what areas could be improved.
You must do both on an ongoing basis throughout the lifetime of your project or service. The process for this should be built into the project at the planning stages to establish accountability and help learning and development.
Five steps to effective monitoring and evaluation:
1. Setting outcome measures
To make an effective business case, you need to use the information collected during the life of your project or service to show that:
- It has made a positive difference to prevention, diagnosis rates, patients, productivity, waiting times, accessibility, equality, hospital admission rates, finances or whatever is relevant.
- It is in line with national targets and standards.
- It is in line with local targets and standards – align your data collection and evaluation with your organisation’s three or four priority performance indicators for the year.
- The standard of service is not sustainable without your project.
- Patients sustain better health outcomes, decreasing the number of hospital readmissions.
- Patient anxiety, depression and quality of life have improved: if patients do not have your support, these improvements may reverse and result in unnecessary hospital admissions.
- Patients are pleased with the service and believe it is beneficial – you can measure patient satisfaction through patient feedback, the number of complaints received, or a survey of patient waiting times and facilities.
See: building evidence
2. Consider these key questions:
- What is the project or service trying to achieve?
- What effect will it have on staff or service users, or the environment?
- Who will benefit and who will not?
- What resources will be used?
- What specific steps will contribute to your overall aims?
- How will you know whether you have met your aims and objectives?
3. Devise a plan to help manage your evaluation:
- Decide on outcome measures/Key Performance Indicators.
- Set a realistic timetable – decide what you can achieve in the given time with the available resources.
- Determine what information you need and who you will collect it from.
- Identify who will collect the information and how they will do this.
- Decide who is going to analyse the information and how it will be done.
- Identify how you will use the results of your analysis; for example, to give information to the community, to improve your project or to produce a report for the funding body.
4. Getting the right data
Data should be collected on an ongoing basis from the start of the project. The information you collect should be determined by the aims and objectives of your service.
Process data describe what your project or service does. Outcome data demonstrate the short, medium or long-term change your project or service caused. Both types of data can be collected in the following ways:
This is information that can be analysed statistically and usually answers questions such as ‘How many?’ ‘How often?’ and ‘Where?’. Examples include recording the number of people attending an activity or receiving a service, their ages or how often they attended.
Potential sources include:
- records of requests for assistance from partner organisations
- number of people involved in various elements of the project such as participants at project events
- data reports
- number of hospital admissions
- demographic information of participants such as age and gender (this can be both quantitative and qualitative data).
This is information that can’t be quantified in numerical terms and usually answers questions such as ‘How?’ and ‘Why?’. An example could be first-person records of people’s views about an activity.
Potential sources include:
- training course evaluation forms
- minutes of meetings
- views of the project participants – what they learned or enjoyed about an activity, how the project has helped them individually and within the family
- letters of thanks from satisfied clients or results of client-satisfaction surveys
- views of agencies working with the project
- knowledge or quality-of-life questionnaires (baseline and follow-up)
- good practice – what has worked well
- constraints – areas where the project has faced obstacles or barriers
- problems encountered and solutions found
- benefits to individuals, groups and communities.
5. Be prepared to present
You may be asked to present a summary of your business case to a panel that will be involved in making the final funding decisions. This will usually involve presenting the key elements of your business case and then answering questions about it – similar to a job interview. Here’s how to prepare:
- Decide on the format for your presentation and what visual aids you will be using.
- Make it a group effort – it will potentially have more impact if you present alongside a few other key stakeholders who have supported you in putting the case together, such as a consultant or nurse for some of the clinical components and someone from your finance department for the budget elements.
- Consider presenting a case study or inviting a patient representative to join the panel if you feel this is appropriate.
- Make sure that the key elements and benefits of your case come across clearly.
- Ensure that written visual aids are spell-checked and proofread.
- Think what questions you may be asked by the panel and prepare your answers in advance.
- Practise your presentation several times beforehand and ask for feedback from stakeholders who have supported you in developing the case.