Improving Our Understanding of Impact - Experimenting with Surge Measures

This past week I spent two days with 11 leaders who are funding solutions to social challenges and know that strong social sector partners are essential to their success. They spent two days together with me and my colleagues in the Social Sector Accelerator’s first Organizational Strengthening Program Design Workshop discussing strategies for helping the organizations they work with have even greater impact.

 Brian Treece - Program Director of Community and Organizational Development and Evaluation at Findlay-Hancock County Community Foundation

Brian Treece - Program Director of Community and Organizational Development and Evaluation at Findlay-Hancock County Community Foundation

Each of the leaders who participated in the workshop already invests, in different ways, in strong partners. Brian Treece is one of them. Brian is the Program Director of Community and Organizational Development and Evaluation at Findlay-Hancock County Community Foundation in Findlay, Ohio. For the last 10 years, Findlay-Hancock has supported nonprofits in their community to become stronger, more effective organizations by providing in-depth coaching, workshops and targeted support such as board development and strategic planning. With a budget of $100,000, this “organizational strengthening” program supports around 30 organizations each year.

Brian and I had the good fortune of meeting in Chicago at the GEO Funders 2017 Learning Conference.  Brian attended a breakfast discussion I led where we talked about how to improve our understanding of the impacts of organizational strengthening.  Our discussion focused on how to understand what we call a “Capacity Surge” – the changes in their organization that support greater mission impact. At the breakfast I shared the “Capacity Surge” measures we had developed. These measures were drawn from our own experience and a range of evaluations, studies, and assessments that described the changes organizations and grant makers sought when building stronger nonprofits. Surge measures include organizational changes such as increase in overall budget size, engagement of stakeholders in the design and implementation of their services or number and quality of Board meetings. Not rocket science, maybe, but the essential building blocks of a strong organization delivering on their mission and helpful for clarifying what nonprofit leaders and their supporters are trying to achieve when they make investments in an organization’s strength.

After the breakfast, Brian asked if he could use the Surge Measures to help assess the effectiveness of Findlay-Hancock’s organizational strengthening program.  At the time, they were interested in three measures - grantee satisfaction, quantity of support they provided and and the impact of that support on the organizations. Though Brian and his colleagues were ahead of many foundations, they were not satisfied with the way they had been measuring the effectiveness of their organizational strengthening program. They had been asking a lot of open-ended questions but wanted to target their measures to ensure their support was achieving what they intended.  

Last year Brian organized the Surge Measures into five categories:

  1. Financial
  2. Quality and Quantity of Program or Service Offerings
  3. Collaboration
  4. Systems Thinking and Advocacy
  5. Internal Process Improvement

He then created an online survey and asked their grantees how useful the Organizational Strengthening (OS) support they received had been in each of the 5 categories. Changes to Internal Processes received the highest marks followed closely by Systems Thinking and Collaboration. 

Respondents were asked to rank the top 3 categories where the OS support had the greatest impact for their organization. With their top 3 in mind they were asked to select 2-3 specific Surge Measures where they saw improvement and describe the change.  For example:

1. Surge Measure - Our overall budget increased.

6 of 14 attributed this type of change to the OS support they received. One grantee reported that because of the support to improve their fundraising strategy their overall budget increased from $240k to $540k per year.

2. Surge Measure - We understood and influenced the system in which we are working.

6 of 13 attributed this type of change to the OS support they received. One grantee reported, “We've engaged in more advocacy at the state level in Columbus meeting with legislature addressing concerns with SNAP, TEFAP and farm bill. Held face-to-face meetings with our state legislatures at the West Ohio Food Bank and participating with a food distribution and interaction with constituents and clients.”

3. Surge Measure - We demonstrated consistent quality financial reporting. 

6 of 14 attributed this type of change to the OS support they received.  One grantee report, “From our individual coaching sessions, we determined we needed a higher-level Finance Director in lieu of an accountant. We made the staff transition and have improved financials and reporting to our board and staff.”

4. Surge Measure - We created new services.

10 of 15 attributed this type of change to the OS support they received. One grantee report, “We didn’t expand but instead we streamlined services and cut back on programs that have measurably been unsuccessful.”

5. Surge Measure - the number and quality of the board meetings we held increased.

13 of 19 attributed changes in their internal processes to the OS support they received. One grantee reported, “Board meetings are more effective because committees meet prior to board meetings and committee chairs send reports to board members in advance.” 

6. Surge Measure - We incorporate the perspective of stakeholders who are directly affected. 7 of 16 attributed changes in their collaborative practices. One grantee reported, “We have incorporated stakeholder stories and quotes into our marketing materials and use their feedback from regularly administered surveys to improve and adjust services.”

Besides the improved quality of response this survey received the highest response rate from all previous surveys (there was a 90% response rate!) and generated the greatest detail on the impact of the Findlay-Hancock OS program. It provided Brian, his colleagues and their Board with the information they needed to better understand the impact of their OS support.  The results sparked an internal discussion about how they can align their OS support with other initiatives of their foundation.

At the Accelerator, we care deeply about increasing investment in strong social sector organizations who are capable of tackling some of the world’s greatest social challenges.  One path to increasing investment is improving our understanding of what types of organizational strengthening support does and does not work. Part of our time together in the Design Workshop was dedicated to developing program specific Learning Agendas to support the leaders in better understanding which of their interventions were generated the greatest impact. 

Brian’s effort is a modest first step to building a framework for learning about OS programs. We all recognize the shortcomings of funder administered surveys. They must be complemented with other learning opportunities especially those that help to rebalance power in the funder/grantee relationship. Next Brian plans to share the data back with their grantee partners and ask them for their feedback on how to improve the way he used the measures. He wants to try at least one more survey for 2018. He’s particularly interested to see if the workshop topics they have planned for 2018 will influence the responses on the Surge Measures (in the case of this year, will they see an uptick in responses around fundraising, planned giving, storytelling, and advocacy), or if those remain stable regardless of the workshop topics.

We'd love for others to experiment with new ways of learning about the impact of their Organizational Strengthening support using Surge Measures. You can download our 'Capacity Surge" measures and the Organizational Strengthening Learning Questions generated by grantmakers at the Learning Conference.