BOSTnet Quality Forum 2009

The Quality Forum, held at The Boston Foundation, was attended by over a hundred program staff, directors, administrators, board members, and funders. It is very telling that even with the financial crisis and all the pressure on organizations and their programs that quality matters. Not just doing the job. Beyond just "trying our best," each attendee seemed committed to making their program a quality service for the children and youth it was designed to serve.

What did this quality look like in the many diverse programs represented at the forum? It is hard to say. Outside of the forum and in many meetings leading up to it, we have heard program directors say they want more information on what children and youth need, what families think about their program, and ways to measure their impact not just for grant requirements, but to answer the questions, "how is my program doing" and "how do I know?" There are a great many program directors feeling stressed by demands for further and more in-depth assessments, reportage, and being held responsible for youth outcomes that they have little direct control over, such as in school attendance or performance. While the definitions of "quality" as a unified front continue to grow (when they do not ebb for lack of funding), staff at the program level continue to do their best to examine their programs and the needs of those they serve.

What was presented at the Quality Forum was not an attempt to promote a unified message around quality, but rather an examination of several views. The view of quality started with research. Dr. Gil Noam, Founder and Director of Program in Education, Afterschool & Resiliency (PEAR) presented a quality view that looked at programs addressing developmental needs of children and youth first and foremost in program design. Young children need different supports than older. The atmosphere of a middle school program needs to be social and active. Older youth need to choose activities and need to be motivated to attend rather than compelled or mandated. This work will soon be made more available to a wider audience but for the meantime, it shows that research is supporting the social-emotional power of out-of-school time as a unique environment for youth development.

BOSTnet presented its own research-based framework. This framework looks in depth at the environment and approach of programs. Based on BOSTnet's 21 years of work, this framework is very simple for those who have worked a long time in the field--almost too simple according to the evaluations returned. However, to those who are new to the field or those who continue to not understand the "story of out-of-school" this framework is accessible and demonstrates how the variables of the environment and the approach meet at the point of service to create a quality environment for youth. If this framework appears simple, the mechanics of making it happen are complex and take a great deal of work. The presentation and tools from the 2008 - 2009 field work were disseminated for use by other programs.

Corey Zimmerman, Director of Strategic Planning and Analysis of the Department of early Education and Care (EEC) presented the state's answer to quality through its developing system of quality improvement and program assessment that may be mandated as part of licencing requirements for school-age programs. The Quality Ratings and Improvement System (QRIS), is a "fast-spreading policy innovation because they align standards, supports to programs, and accountability efforts into one non-duplicative system." This effort appears to need a unified system to support it which the out-of-school time field has yet to develop (and often the field asks whether this should ever be developed). This innovation is perhaps useful to staff at programs but also is seen by many in the field as an unfunded mandate since some of the quality improvements don't appear to come attached to additional funding needed for staff time or other expenses. Of course, this is a work in progress, so we cannot point to the nature of the QRIS innovation nor its impact.

The final part of the day was a panel of directors. For Kids Only Afterschool (FKO) and Bird Street Community Center (BSCC) both discussed specific examples of quality in their organizations. For FKO the issue was investment in staff development and an allowance to work with staff at their particular stage in their career and educational growth. This meant a great deal of professional development where their time was paid for. This meant swing time to attend college courses or support to gain a certificate. A great deal of training was done internally so that the FKO approach was reinforced and outside trainers brought in for targeted work such as behavior or curriculum development. BSCC also invested in staff. BSCC worked hard to rais funds to increase the number of full time staff so that the positions and the work could be the primary focus of staff and the job as a career was feasible. Staff were supported internally and also provided professional development since now with the extra hours, program staff were paid for these trainings. Both these quality strategies focused on staff. Without good consistent and prepared staff, program quality would suffer. While many in the audience said their organizations could never cover the funding needed to provide such supports, it demonstrated that to achieve quality programming perhaps piecing a workforce together and expecting full time commitment at part time pay is not feasible.

In all, the Quality Forum presented a lot of information in too little time. This is also a sign of the times. Programs have reported less time to devote to outside meetings, conferences, even training. It was decided that it was better to fit in too much information - allowing learning points for everyone - rather than a brief survey of information surrounded by ample coffee breaks. Also, as another sign of the time, there was no catering budget.

We welcome further evaluation or response to our Quality Forum. There will be a more formal discussion of this event in the BOSTnet BUZZ, BOSTnet's e-newsletter.

Quality is "Job One"?

The days to the Quality Forum are short - the questions around "quality" as it relates to out-of-school time seem to grow. "Quality" is one of those terms that we hear a great many times - from corporations, from product salespeople, from former automotive giants. Whether "job one" or "continuous improvement" or "quality improvement" or other flashy catch phrases and academically or research supported systems, the question remains - what is a quality out-of-school program?

Also, built into this question is other question. Who asks for this "quality."

If "out-of-school time" is any program that works with children but is not run directly by public education employees then these program include more than "after-school" and reach into museums, summer camps, and enrichment programs - even if some part of these programs take place within the confines of a public school facility or during the hours of school and in collaboration with school teachers or officials. Out-of-school is an approach that transcends programmatic focus, mission, or population as it is a number of very diverse programs bringing ways of learning that may differ (not detract but differ) from the approach the public system is currently taking.

Perhaps, then, quality is different for different programs but can be measured if those measurements are directed to the focus of the program or the ability of staff to create the type of environment where children and youth are cared for and can learn - differently.

And who asks for quality? Yes, regulators, funders, parents, politicos, endorsers, and sundry other influences that have piled on to the backs of programs.

However, it is children and youth that ask for this. True, no seven year old will say, "can I have a quality program and how are you going to assess this so that I know both empirically and based on research drawn from data that was appropriately cleaned and triangulated that my youth outcomes improve based on my attendance" - which seems what many are asking site coordinators and program directors - but "What am I doing today?"

That's at the center of quality, a child or youth asking - "what am I doing today?" Right now, in the moment.

Our job is to make that "doing" fun, engaging, learning, positive, supportive, and safe.

Each one of these elements may be measured in great detail - what is "fun" how do we know when we are having "fun" - however, is that appropriate for a field where the work is not to answer philosophical conundrums but to provide a point-of-service.

How about measuring:

Children or youth participation by attendance of program
What days do children or youth come? When are they picked up? How much of the year or season do they stay? Do they also use the program for other times of the year?

Children or youth return rates
Do children and youth keep coming back?

Parent or family involvement
Do parents pick their children up early? Do they attend events or assist in the program?

Staff retention
Do staff work the duration of the program? Do they return again to work another season or year? When they leave, do they continue to work in a similar position or move on to a higher position?

Children or youth satisfaction
What do they say about the program? Do they feel stuck there or can't wait to attend?

Perhaps by focusing on measurements that are obtainable by programs, meaningful data can be gathered and program quality can be assessed not by a multitude of indicators, but by a finite number of contact points that can span programmatic type, approach, and environment.

Coalition We Must?

There is a great deal of talk around building quality programs and presenting a united front to funders. In the funder community - and amongst government officials - there is an increasing call by the state and city for organizations to stop their "siloing" and to come together in increased collaborations - or coalitions.

Coalitions are built on common purpose. Despite differences in special interests and approaches, everyone is gathered to bring resources to children and develop quality environments - yet these days it seems hard to keep coalitions together as so many of us are working from the same limited and proscriptive funding sources. How can we both compete and work together? Perhaps it was not a good idea of make non-profits more "corporate" in making funding and the culture of organizations more competitive. Along with competition comes... not working together but working for self interest and organizational survival.

True coalitions are built from common practices and good ones are those that agree on common language and concepts and can martial resources around concepts that hold true to the work on the ground as they do in expressing the complexities at the administrative level. Coalition we must. Yet, where are we with building those coalitions when out-of-school time field cannot agree on what the field offers as a product. We seem not to be able to agree how to measure the quality of that service or which organization gets to set standards, competencies, and the direction of the field (this had been done through committees at times, but committees grind so slowly the field conditions change before we get the first public draft of whatever is being... "committee'd".

Coalition we must. However, we need to do some damage control in our field - and non-profits in general. The attitude of the free market has gotten us into a great deal of trouble. Out need for competition has led to less cooperation and more duplication of services, our love of free will has led to a rogue attitude of self-interest, and accountability seems something fostered on the non-for-profit community because it has no use in other more lucrative industries. Today, perhaps we can again capture the excitement of a new field of work and connect to our like-minded friends doing this work with us and turn a new page in Out-of-School Time as well as breath a little life back into the Great Society we have allowed to falter and may yet be able to rebuild.