In the Fall 2014 YALSA journal (vol 13, number 1), I published an article about creating outcome measurement tools collaboratively with staff and participants for a teen program (Measuring Outcomes for Teen Technology Program, p. 25). The program I discussed is the Teen Tech Squad, tech workshops for teens led by teens at Hennepin County Library.
When I began working with the teen librarians to identify outcomes and measurement tools, an important step was relying on the expertise of the teen librarians. I did not assume that I knew what teens were doing in the workshops or what skills they were gaining. I relied on the expertise of the teen librarians to identify these things. I worked with them to make sure that they understood what outcomes are and we collaboratively created the outcomes and survey questions. We also took the time to get teens opinions on the questions we asked so we knew our questions would be understandable and effective. I empowered staff to take the lead on implementing the evaluation and continue to offer my assistance as they discover what is working and what isn’t.
This approach to evaluation is called “developmental evaluation,” a concept developed by program evaluation consultant Michael Quinn Patton. Developmental evaluation differs from traditional evaluation in many ways. For example, one way is the role of the evaluator. Traditional evaluation positions the evaluator as an outsider from the program they are evaluating while developmental evaluation positions evaluation as a job duty of the program deliverers. Developmental evaluation is most suited to programs that are innovative and adaptable; that is, not static.
Why this is important is that I see a need for libraries to have an in-house evaluation expert. It may seem easier (although more expensive) to hire an outside firm to evaluate. What library staff miss out when they do this is learning how to evaluate on their own. Knowing how to evaluate means that you can work evaluation into the biggest and smallest projects at your library. It can help you design projects intentionally, evaluate them, and decide what should continue, what should change and what can come to an end.
YALSA’s tenth teen programming guideline directs library staff to, “Engage in youth-driven, evidence-based evaluation and outcome measurement.” Now that we have measurement tools in place and used teen feedback to develop the tools, our teen workers are charged with using them and evaluating their results to improve workshops. Our library staff also use measurement tools to evaluate our teen workers’ experiences and improve the project.
I’m very lucky to be sharing my experience in developing the teen tech evaluation tools at this year’s Libraries Leaders Summit at the 2015 Computers in Libraries conference in Washington, D.C. (April 27-29). If you’ll be there, I’d love to talk to you about evaluations. If you can’t make it, there are many resources you and your colleagues can access to learn more about creating evaluations. The Research and Institute for Public Libraries summer conference is focusing on outcomes and how to measure library impact. There are also a few books I’ve listed below. Finally, reach out to nearby universities to find out if they offer trainings for measuring outcomes — I caught the evaluation bug after attending a one-day training at the nearby University of St. Thomas. It’s great to be able to talk to expert evaluators about your library’s work and get their feedback. Remember YALSA’s seventh teen programming guideline, “Participate in targeted and ongoing training to build skills and knowledge relating to programming.” You may intuitively know what works and doesn’t work about your teen programs. It’s also important to evaluate in a more formal manner to get new insights.
Dynamic Youth Services through Outcome-Based Planning and Evaluation, by E Dresang, M. Gross, L Edmonds, Holt. ALA Editions, 2006.
Evaluating Teen Services and Programs, by S. Flowers. ALA Editions, 2012.
Getting Started with Evaluation, by P. Hernon, R. Dugan and J. Matthews. ALA Editions, 2014.
Johannah Genett is the resource services division manager for Hennepin County Library. She can be reached at jrgenett@hclib.org.
