What you need to know about monitoring, evaluation & learning.

In this blog, Emily Burn, THET's Evaluation & Learning Officer, walks you through the key monitoring, evaluation and learning issues health partnerships should be addressing when planning projects. 

Recently I attended an M&E workshop held by INTRAC which brought together many M&E practitioners from the NGO and public sector to share challenges and solutions to common issues in M&E.  The detailed discussions led me to think about the over-arching principles that we should return to when we plan our M&E. Here I summarise the components of a good system for monitoring, evaluation, and learning, which health partnerships should think through when planning their projects.

Your Monitoring & Evaluation needs a rigorous system underpinning it

Your monitoring and evaluation activities need to be underpinned by an efficient system that enables you to focus your efforts and translate data into evidence and lessons.  An efficient system is particularly important to health partnerships which are operating with limited resources for M&E but which need to meet reporting requirements, and make the most of opportunities for learning.

So what does an efficient M&E system look like?

It is thoroughly planned

The project plan clearly articulates the change you want to achieve.  A Theory of Change approach will help you to see the logical flow of your objectives. For more information and a tool for using this approach see: http://www.thet.org/hps/resources/good-practice-guidance/project-planning-theory-of-change-1

You have discussed who your stakeholders are and the types of results they will be interested in, which means that you can plan the analysis, focussing on a limited number of questions that the data could answer, rather than all possible questions.

Each objective has ‘indicators’ (also known as signs of success or measures of progress) that are appropriate and feasible given the time, funds, expertise, and data (etc.) that you have access to.  Also, you know who will gather the indicator data, with what tools, and how frequently.  You know if the data collector has the expertise to do this or you have a plan in place to provide training. You know if you need to create a new data collection tool or if an adequate system is already in place that you can use.

You know how the data will be brought together in a central place. For example, you have an M&E focal person in each project site whose responsibility it is to submit health worker logbook data on a monthly basis (via email ) and then you have a central site where these data are entered into a spreadsheet, ready to be analysed.

M&E planning tool: http://www.thet.org/hps/resources/good-practice-guidance/monitoring-evaluation-plan

It is carried out consistently and in partnership

All those who are part of project implementation understand the objectives and buy in to the indicators.  This may require on-going review and adjustment of the indicators and milestones. 

You have a process in place to disseminate the project’s progress regularly so that those affected by the project understand what is going well and what is not.  The channels for communicating the results take into account the different project stakeholders, from NHS board members, to the staff on the ward, where the former may be most receptive to a presentation from the UK partner, and the latter may prefer to see improvements in practice displayed on a poster or mapped out on a chart in a staff area.

Lastly, an efficient M&E system facilitates learning by making full use of the findings to question how things are going, pinpoint problems, and so make changes based on evidence. At THET, our M&E system includes an analysis process that aggregates qualitative data into a simple spreadsheet where we group findings thematically. We see the benefits of this: when we write reports for our donors; when we need evidence of how best to support health partnerships; or when we need examples of, for example, positive changes in practice. In this way, our system for analysing and recording the data from health partnerships helps us to make use of it, in multiple formats, and for all our stakeholders. 

Discussing protocols at Kambia Government Hospital, Sierra Leone.

Anaesthesia training in Zambia - a volunteer perspective - part 2

Dr Lowri Bowen is currently in a 6 month volunteer post with the Zambian Anaesthetic Development Project (ZADP) working at University Teaching Hospital, Lusaka. In this blog she reflects on her time in Lusaka so far and the importance of not taking things for granted. 

My last blog centered firmly on the educational role of the Zambian Anaesthetic Development Project (ZADP), so this time I thought I would cover a different aspect of my work, which runs alongside the teaching and clinical supervision aspects.

I am sure that most people are familiar with the United Nation’s Millennium Development Goals – essentially a blue print of areas that’s been agreed on by all the world’s countries as essential to improve upon. There are eight of these but I wanted to focus on number 4 – decreasing child mortality.

There are many different ways to combat child mortality and there are many fantastic programmes going on all over Zambia and the world to this very end; however this is the start of our contribution towards getting to the goal of making University Teaching Hospital, Lusaka a safer and better place for children to have their anaesthetics. This is a particularly important topic, so why have I chosen to talk about this now, almost 4 months into my time here?

Well…..   Last week highlighted some of the most exciting times for me as a ZADP trainee. Despite plenty of interesting clinical work and really fulfilling teaching sessions over the course of that week I got incredibly excited about a door.  It does seem like quite a crazy thing to get excited about – I mean it’s something we use on a daily basis – it opens and it shuts and it serves it purpose but it’s hardly the highlight of anybody’s week usually.

So what type of a door was it?

The door in question is a plain, brown wooden door which I confess does not fire up the imagination. So why have I become so excited by this particular door?

If you think about it carefully a door is actually something that appears quite a lot in children’s stories and they are significant not by their appearances, but by what they contain behind them:

There is the wardrobe door to enter Narnia in ‘The Lion, the witch and the wardrobe”, the changing room door where Mr. Ben disappears into his various different worlds wearing his fancy dress outfits and, of course, who could forget the Alice in Wonderland door down the rabbit hole!

Behind my featured door is the new anaesthetic store room for the paediatric (children’s) theatre block.  It is where we will be able to safely store and register all the required equipment for the safe delivery of anaesthetics to children in UTH.  It means that we will no longer have to scramble around searching to find items that may or may not be present or kept elsewhere. We will be able to keep a detailed inventory of all the equipment which will allow timely ordering of things before they run out and also allow the theatres to have less equipment strewn around them and become a more ordered place to work.  This is a project that was started over a year ago. The main focus initially was to secure a storage area in the main theatre complex as there was nowhere for anaesthesia equipment to be kept apart from a small toilet block that regularly leaked and caused damage to vital equipment.

After much work and tireless negotiation by my previous colleagues, this was finally secured last June and since then has become well stocked and ordered with 2 of the anaesthetic MMeds in charge of it. Since this time we have secured a similar area in the emergency theatres and so when I arrived in August my remit was to focus on getting a safe area of storage for the paediatric theatre block. We quickly identified a store area but required a door to serve as a secure area for us to develop. Since this time (3 and a half months) it has taken painstaking negotiations with more people than I could ever have imagined being linked with the hanging of a door! Many a letter and meeting and cajoling visit to the workshops and purchasing departments all added to the momentum that could also be called just plain nagging! The help of the theatre matron was critical and with her additional nagging I think we managed to get the ‘project’ up and running. Initially the victory was getting a door frame placed, as there wasn’t one.

The frame getting placed and held up by batons as the concrete dries.

Once the frame was placed there was then the inevitable three week wait to get the plastering dry and the door frame procured  – no such DIY shop down the road here! So Friday was the celebration of conquering the beaurocracy and of all those frustrating weeks of doubt by the visualization of an actual door sitting in it frame!

The door! I am hopeful there may be a paint touch up at some stage, hopefully after the lock appears. 

There are still things to do – the little problems of a lack of a lock, clearing the space out as it has accumulated lots of ‘junk’ over the years, sort out some shelves, stock up and make an inventory are all in front of us… but this is most certainly a step forwards and a great achievement for the improvement of patient safety in UTH. To celebrate, we managed to decorate the walls of the recovery room and holding bay with some wall stickers that some friends had brought out for me – ok, this is not strictly a UN MDG 4 goal but it is important to provide stimulation and create a child-friendly atmosphere in the theatre block whilst the children wait for their surgeries. It was great to get all of the staff – from theatre porters, nurses, trainees, consultants and surgeons to give their opinions and to help in placing the decorations and become enthused over such a simple but important aspect of emotional care for the children.  As you can see from the snaps below the place looks a lot brighter now! I think both children and staff will enjoy being in the department a lot more now!

To find out more about THET's programme work in Zambia please visit our THET.org