Home Library Translate
A A A
Share »
Follow us on Facebook Follow us on Twitter Follow us on LinkedIn
Connect »

Blog: Nonprofit Capacity Building

Menu

  • This Blog's Home
  • Guest Writer Submissions
  • Policies
  • To Subscribe to a Blog
  • About
  • Feedback

How to Evaluate on a Budget: DIY or Outsource?

By Priya Small on May 29, 2012

This question has nagged me for a while.  It emerged to the forefront as I recently considered the most efficient way to carry out my responsibilities, spurred on by the 20-80 rule:  80% of our outcomes come from 20% of our efforts.  So how do you evaluate on a budget?  Should you do-it-yourself (DIY) or outsource?  We’ll begin by considering some of the advantages of either option.

Advantages of the Do-It-Yourself Approach

Firsthand Knowledge of Issues

We are sometimes our own worst critics.  But we may have more to offer than we’d imagine.  Recently I had the privilege of working with stakeholders who have personally experienced the challenges  their program addresses.  I emerged from that meeting with a deep appreciation for the collective wisdom of program  participants and those who work closest with the people who receive a non-profit’s services.  It is also vital to fully coordinate evaluation efforts with existing program operations, especially to plan program evaluation during program planning stages.

Save Money

It goes without saying that you can streamline evaluation-related costs and increase efficiency by using in-house staff who are most familiar with the way a program operates.

Save Time

In-house staff already have relationships with program participants and thus need not go out of their way to build trust.  Building trust takes time!  Good evaluations are founded on relationships of trust with program participants.  If they do not like or trust the person who is administering the evaluation, can we truly expect genuine and forthcoming answers?  Also, staff can gather data (via surveys, interviews, etc) in the course of carrying out their responsibilities. Since they know the ins and outs of a program, program staff’s expertise is vital to the success of any program evaluation.

While in-house staff bring indispensable strengths to an evaluation, please also consider the advantages of using an external evaluator.

Advantages of Using an External Evaluator

Expertise

We’ve all probably had these thoughts at some point or the other:

“Evaluation is not rocket science, anyone can do it!”

“What is so hard about designing a survey, distributing it and analyzing the data?  Even a high-school student can do it.”

This topic was recently broached within an online  American Evaluation Association  Thought Leader Discussion Group.  The danger is that evaluation can seem a lot easier than it actually is.  But if you look at  Program Evaluator Job Announcements, you’ll notice specific qualifications.  The following is a sample of what you may see for a junior level position:

  • at least a Masters in Social Sciences or Public Health, Ph.D. preferred
  • at least 1-2 years of experience conducting program evaluations
  • strong communication and interpersonal skills
  • experience with quantitative data analysis and with data analysis software, such as SPSS

The general consensus seems to include the above qualifications as well as engagement in professional continuing education in the field of program evaluation.  (Before you stop reading this post and throw up your hands in despair because you do not have the budget to hire another person, please keep reading.  we’ll get to possible solutions to this dilemma soon!)

There have been reams of papers and chapters written on program evaluation, specific models, quantitative and qualitative methods and survey design.  Enough to make even the most experienced evaluator feel that there is always so much more to learn!  Other evaluation colleagues often call attention to mistakes that well-intentioned novices have made, for example,  in sharing evaluation results and data.

Sure, it is important to maintain a level of healthy skepticism:  of course, any evaluator will want to make a case for why you should hire a professional to do the job right.  But I’d encourage you to also consider how much risk  you are willing to take if an evaluation is not done properly.  What do you stand to lose?  Think about all the ways that misrepresented data could hurt your program.  Think about all the opportunities to grow and improve your program that may be lost through a botched evaluation.

Objectivity

Again from first hand experience, I’ve heard stakeholders closest to the program express appreciation for the objectivity and fresh perspective that an external evaluator (also known as an independent evaluator) brings to the table.  Program participants may feel awkward being forthcoming with those actually delivering the services.  Due to their existing relationships with program staff, participants may feel subconsciously pressured to give answers that they think staff want to hear.

Ability to Coordinate

Having a skilled evaluator coordinate an evaluation effort will spare you headaches and worry.  This may be a better option over merely dividing up responsibilities among maxed-out staff  or having to worry about coordinating an extra project on your own.

Possible Solutions

Alright, now we come to the simple solutions.  Coordination implies that you do not have to hire the evaluator to do everything!  This translates into cost savings.  Carter McNamara, Ph.D. astutely applied the 20-80 rule to program evaluation in his article Basic Guide to Program Evaluation.  Since 20% of the effort can produce 80% of the outcomes, a good option may be to contract with an external evaluator just for the 20% of effort that produces the 80% of outcomes.  I agree with Dr. McNamara that the best responsibilities to outsource would be the design of the evaluation and surveys.  I’d also add data analysis and reporting.

Data collection can be most time-consuming and expensive.  Using program staff for these functions may be a good compromise, as long as measures are taken to encourage objectivity, for example, a volunteer placing surveys in sealed envelopes.

If you still find the notion of contracting with an independent evaluator daunting, consider the following solutions that I have observed other non-profits use.  Once you have clearly defined the scope of a very feasible evaluation project, carefully recruit and choose a well-qualified:

  • graduate student whose rates and schedule may be more flexible
  • pro bono evaluator who may be trying to gain expertise in a new area or may just be interested in giving back
  • a stay-at-home parent who may be willing to trade their program evaluation expertise for more flexible terms.

You may considering beginning your recruiting efforts at the  American Evaluation’s Association’s Career Center or on  AEA’s LinkedIn Group.  Here is some further reading on using an external evaluator from the Centers for Disease Control and Prevention.

Evaluating on a budget doesn’t have to be an unattainable dream and neither does it have to be a do-it-yourself disaster.  With creative and strategic solutions and careful planning, it can be a practical reality.

——————

For more resources, see our Library topic Nonprofit Capacity Building.

_______________________________________________________________________________________________

Priya Small has extensive experience in collaborative evaluation planning, instrument design, data collection, grant writing and facilitation.  Contact her at priyasusansmall@gmail.com.  Visit her website at http://www.priyasmall.wordpress.com.  See her profile at http://www.linkedin.com/in/priyasmall/

« Previous Next »

Search Our Site

Meet the Blog’s Co-Hosts

Marion is a popular facilitator of workshops and Board retreats. Her expertise includes strategic planning, board development, governance and harnessing the power of technology. [Read more ...]


Carter McNamara of Authenticity Consulting, LLC, provides organizational development, consultant training and Action Learning services, and is developer of the Free Management Library. [Read more ...]

Recent Blog Posts

  • The 2015 Wishlist of Books for Nonprofit Folk Is Here
  • Summer Reading for Nonprofit Folk
  • The Interim Between CEOs Is Important – Make the Most of It
  • Founders of Nonprofit Organizations
  • Nonprofits: Businesslike, But Not Businesses
  • The 2013 Millennial Impact Research Report – Focus on Benchmarks
  • How to Get out of the Evaluation Report Writing Slump
  • You and Your Nonprofit Board – New and Recommended
  • Five Tips on Making Your Evaluation More Systematic
  • 2013 Nonprofit Finance Fund Survey Highlights

Categories of Posts

  • Basics and Overviews
  • Boards
  • Collaborations
  • Evaluations
  • Finances
  • Fundraising
  • General Resources
  • Human Resources
  • Marketing
  • Programs
  • Social networking
  • Staffing
  • Strategic Planning
  • Technology
  • Uncategorized

Blogroll

  • two sets of questions

Related Library Topics

  • Assessments (Org'l)
  • Boards of Directors
  • Capacity Building
  • Consultants (Using)
  • Executive Dir. Role (CEO)
  • Group/Team Skills
  • Finances
  • Fundraising
  • Leadership
  • Management
  • Marketing
  • Program Management
  • Program Evaluation
  • Social Enterprise
  • Staffing
  • Succession Planning
  • Starting Nonprofits
  • Strategic Planning
  • Taxation
  • Volunteers

Library's Blogs

  • Boards of Directors
  • Building a Business
  • Business Communications
  • Business Ethics, Culture and Performance
  • Business Planning
  • Career Management
  • Coaching and Action Learning
  • Consulting and Organizational Development
  • Crisis Management
  • Customer Service
  • Facilitation
  • Free Management Library Blogs
  • Fundraising for Nonprofits
  • Human Resources
  • Leadership
  • Marketing and Social Media
  • Nonprofit Capacity Building
  • Project Management
  • Quality Management
  • Social Enterprise
  • Spirituality
  • Strategic Planning
  • Supervision
  • Team Building and Performance
  • Training and Development
About Feedback Legal Privacy Policy Contact Us
Free Management Library, © Copyright Authenticity Consulting, LLC ®; All rights reserved.
  • Graphics by Wylde Hare LLC
  • Website maintained by Caitlin Cahill

By continuing to use this site, you agree to our Privacy Policy.X