June 18, 2020

The BEARR Trust usually awards ten or more small grants (of up to £3,000) each year to small civil society organisations working in partnerships to try out new approaches to health and social welfare problems in Eastern Europe and Central Asia.

How do we assess the impact of the projects we fund?

Initial feedback

BEARR asks grant recipients to complete monitoring forms in addition to their project reports. We ask grantees to be candid about how their projects went – we want to demystify the process and encourage an open and honest approach to learning from things that did not go as well as hoped. Grantees write their project reports immediately – a project’s duration is normally six months – so we have quick feedback about their activities, which we often publicise in newsletter articles. This enables others with similar interests to make contact and exchange experience. It also enables us to describe to our own donors how their money was spent

Accountability to communities

However, accountability to funders is only part of the monitoring and evaluation process. BEARR also sees evaluation as an ethical issue about power and purpose and we have therefore started to focus on a different direction of accountability – accountability to the communities being worked in.

We are developing an approach to evaluation that aims to transform it from a mechanical auditing exercise into one where we help our grantees to reflect on their projects and to raise the profile of their project achievements. We want to encourage them to take responsibility for and initiatives in adapting their work, in the light of the project results, to better suit the needs of their communities and of particular beneficiaries.

Visits to projects

One way in which we do this is to visit grantees in order to do our own assessment of a project’s early impact, and to examine how effective our approach of funding pilot and experimental projects is. We prefer to let some time elapse in order to make this more of a reflective process about any real change that has happened either to the beneficiaries or the organisation itself as a result of the project.

We do not have the means to visit every project, so we try to take advantage of opportunities as they arise – for example, trustees and volunteers sometimes add project visits to their own travels in the BEARR region. In this way, since 2016 we have visited projects in progress or made contact with past grantees in Russia, Ukraine and Armenia – sometimes more than once. Unfortunately, some visits in Moldova, Kyrgyzstan and Ukraine that were planned to take place in May and June this year have had to be postponed because of the COVID-19 pandemic.

Here are brief snapshots from some of BEARR’s project visits:

Visit to Lviv, Ukraine May 2016

In May 2016 BEARR trustee Janet Gunn was in Lviv for BEARR’s regional conference, and used the opportunity to visit a project run by the Women’s Perspectives Centre. BEARR had just awarded them a grant to fund an expansion of their training in textile and woodworking to provide skills and some income to internally displaced (IDP) women and men. Janet’s visit was early in the project, but it was clear that activity was well under way.

Visits to Kyiv and Brovary, Ukraine May-June 2018

Two years after BEARR’s 2016 Small Grants Scheme for projects to help IDPs, BEARR’s Small Grants Officer, Anna Lukanina-Morgan, took time out from a private visit to Ukraine to catch up with grantees Tolerspace, Donetsk Youth Debate Centre, Business Perspective, Free People Employment Centre and SOS Children’s Village. Two years on, Anna’s visits gave us a good insight into the challenges faced by many organisations in obtaining follow-up funding to keep their projects going (see The BEARR Trust Annual Report 2018). Nevertheless, grantees had often adapted and sometimes developed new activities.

Visit to Armenia August 2019

BEARR Information Officer Sophie Vandyck used a private visit to the region to make a second BEARR visit to ISHR in Yerevan – previously visited by former BEARR Chairman Robert Brinkley in May 2017. Sophie also visited Improve Our Village in Kotayk. This NGO had received two grants from BEARR: one in 2016 to teach beekeeping skills to refugees (featured on the cover of BEARR newsletter No 69 in September 2017), and one in 2019 for their ‘Women’s Rights Kitchen’ project.

Visit to Kyiv, Ukraine, February 2020

Before the Covid-19 pandemic curtailed travel this year, Anna Lukanina-Morgan was able to visit three grantees from the 2018 and 2019 schemes. Anna visited Aratta (2018) and Zhytomyr (2018), and made a second visit to Tolerspace, who had been awarded a further grant in 2019 to produce a comic strip-style book to teach adolescent girls to protect themselves from harassment.

A collaborative process

Our project visits are not an auditing inspection. We are more interested in helping our grantees to consider all aspects of their project’s activities and outcomes, and to take ownership of the evaluation process. We enjoy the visits, as they are a great opportunity to get to know civil society organisers in the region, and to develop the sort of relationship that might lead to future collaboration. Some past grantees with good experience and expertise to share have travelled to London to speak at BEARR’s annual conference, or have participated in the conference via video link.

What we look for and the questions we ask

Given the pace of political, social and economic change in many of the countries we work in, it is not surprising that some organisations that have received grants from BEARR no longer exist, or have transformed in some way. We are interested in finding out whether an organisation has changed course and whether it has grown in staff numbers and in its ability to attract volunteers. We want to know whether the partnership that carried out the original BEARR-funded project worked well, and whether the partners still work together. We explore whether BEARR’s project changed the way the organisation works, and whether it opened up new avenues of activity, or developed new networks. For projects that took place a few years earlier, we are especially interested in whether the activities led to changes in attitudes or in official policy.

Some organisations that receive a BEARR grant find that it helps them to attract further funds, because they have already demonstrated their credibility to an experienced grant-giver. However, many organisations tell us that they struggle to find follow-up funding. We are interested to understand how an organisation would have spent any available follow-up funding, and how this might have added value to the original project. We try to find out the areas of need that might be slipping through the net, and use this knowledge to inform our future grant schemes.

Trying out alternative approaches: peer evaluation

As an alternative to visiting projects we have experimented with peer evaluation: another approach to evaluation that is also intended to strengthen local leadership and ownership of project results. This has been our most significant investment to date in an evaluation exercise. Our peer evaluation of projects concerned with domestic violence in Central Asia took place in May 2017, in collaboration with INTRAC in Kyrgyzstan.  Grantees from Kyrgyzstan travelled to Almaty, and their Kazakh counterparts made a return visit to Bishkek the following week. It was a novel experience for the participants, for several of whom it was a helpful training exercise in structuring a dialogue between organisations with different ideologies and project activities. An important outcome was the realisation of how much more needed to be done to address the issue of domestic violence – an insight that led BEARR to focus its 2019 and 2020 small grants scheme on this area.

Nicola Ramsden, Chairman of the BEARR Trust