BOSTnet Quality Forum 2009

The Quality Forum, held at The Boston Foundation, was attended by over a hundred program staff, directors, administrators, board members, and funders. It is very telling that even with the financial crisis and all the pressure on organizations and their programs that quality matters. Not just doing the job. Beyond just "trying our best," each attendee seemed committed to making their program a quality service for the children and youth it was designed to serve.

What did this quality look like in the many diverse programs represented at the forum? It is hard to say. Outside of the forum and in many meetings leading up to it, we have heard program directors say they want more information on what children and youth need, what families think about their program, and ways to measure their impact not just for grant requirements, but to answer the questions, "how is my program doing" and "how do I know?" There are a great many program directors feeling stressed by demands for further and more in-depth assessments, reportage, and being held responsible for youth outcomes that they have little direct control over, such as in school attendance or performance. While the definitions of "quality" as a unified front continue to grow (when they do not ebb for lack of funding), staff at the program level continue to do their best to examine their programs and the needs of those they serve.

What was presented at the Quality Forum was not an attempt to promote a unified message around quality, but rather an examination of several views. The view of quality started with research. Dr. Gil Noam, Founder and Director of Program in Education, Afterschool & Resiliency (PEAR) presented a quality view that looked at programs addressing developmental needs of children and youth first and foremost in program design. Young children need different supports than older. The atmosphere of a middle school program needs to be social and active. Older youth need to choose activities and need to be motivated to attend rather than compelled or mandated. This work will soon be made more available to a wider audience but for the meantime, it shows that research is supporting the social-emotional power of out-of-school time as a unique environment for youth development.

BOSTnet presented its own research-based framework. This framework looks in depth at the environment and approach of programs. Based on BOSTnet's 21 years of work, this framework is very simple for those who have worked a long time in the field--almost too simple according to the evaluations returned. However, to those who are new to the field or those who continue to not understand the "story of out-of-school" this framework is accessible and demonstrates how the variables of the environment and the approach meet at the point of service to create a quality environment for youth. If this framework appears simple, the mechanics of making it happen are complex and take a great deal of work. The presentation and tools from the 2008 - 2009 field work were disseminated for use by other programs.

Corey Zimmerman, Director of Strategic Planning and Analysis of the Department of early Education and Care (EEC) presented the state's answer to quality through its developing system of quality improvement and program assessment that may be mandated as part of licencing requirements for school-age programs. The Quality Ratings and Improvement System (QRIS), is a "fast-spreading policy innovation because they align standards, supports to programs, and accountability efforts into one non-duplicative system." This effort appears to need a unified system to support it which the out-of-school time field has yet to develop (and often the field asks whether this should ever be developed). This innovation is perhaps useful to staff at programs but also is seen by many in the field as an unfunded mandate since some of the quality improvements don't appear to come attached to additional funding needed for staff time or other expenses. Of course, this is a work in progress, so we cannot point to the nature of the QRIS innovation nor its impact.

The final part of the day was a panel of directors. For Kids Only Afterschool (FKO) and Bird Street Community Center (BSCC) both discussed specific examples of quality in their organizations. For FKO the issue was investment in staff development and an allowance to work with staff at their particular stage in their career and educational growth. This meant a great deal of professional development where their time was paid for. This meant swing time to attend college courses or support to gain a certificate. A great deal of training was done internally so that the FKO approach was reinforced and outside trainers brought in for targeted work such as behavior or curriculum development. BSCC also invested in staff. BSCC worked hard to rais funds to increase the number of full time staff so that the positions and the work could be the primary focus of staff and the job as a career was feasible. Staff were supported internally and also provided professional development since now with the extra hours, program staff were paid for these trainings. Both these quality strategies focused on staff. Without good consistent and prepared staff, program quality would suffer. While many in the audience said their organizations could never cover the funding needed to provide such supports, it demonstrated that to achieve quality programming perhaps piecing a workforce together and expecting full time commitment at part time pay is not feasible.

In all, the Quality Forum presented a lot of information in too little time. This is also a sign of the times. Programs have reported less time to devote to outside meetings, conferences, even training. It was decided that it was better to fit in too much information - allowing learning points for everyone - rather than a brief survey of information surrounded by ample coffee breaks. Also, as another sign of the time, there was no catering budget.

We welcome further evaluation or response to our Quality Forum. There will be a more formal discussion of this event in the BOSTnet BUZZ, BOSTnet's e-newsletter.

Quality is "Job One"?

The days to the Quality Forum are short - the questions around "quality" as it relates to out-of-school time seem to grow. "Quality" is one of those terms that we hear a great many times - from corporations, from product salespeople, from former automotive giants. Whether "job one" or "continuous improvement" or "quality improvement" or other flashy catch phrases and academically or research supported systems, the question remains - what is a quality out-of-school program?

Also, built into this question is other question. Who asks for this "quality."

If "out-of-school time" is any program that works with children but is not run directly by public education employees then these program include more than "after-school" and reach into museums, summer camps, and enrichment programs - even if some part of these programs take place within the confines of a public school facility or during the hours of school and in collaboration with school teachers or officials. Out-of-school is an approach that transcends programmatic focus, mission, or population as it is a number of very diverse programs bringing ways of learning that may differ (not detract but differ) from the approach the public system is currently taking.

Perhaps, then, quality is different for different programs but can be measured if those measurements are directed to the focus of the program or the ability of staff to create the type of environment where children and youth are cared for and can learn - differently.

And who asks for quality? Yes, regulators, funders, parents, politicos, endorsers, and sundry other influences that have piled on to the backs of programs.

However, it is children and youth that ask for this. True, no seven year old will say, "can I have a quality program and how are you going to assess this so that I know both empirically and based on research drawn from data that was appropriately cleaned and triangulated that my youth outcomes improve based on my attendance" - which seems what many are asking site coordinators and program directors - but "What am I doing today?"

That's at the center of quality, a child or youth asking - "what am I doing today?" Right now, in the moment.

Our job is to make that "doing" fun, engaging, learning, positive, supportive, and safe.

Each one of these elements may be measured in great detail - what is "fun" how do we know when we are having "fun" - however, is that appropriate for a field where the work is not to answer philosophical conundrums but to provide a point-of-service.

How about measuring:

Children or youth participation by attendance of program
What days do children or youth come? When are they picked up? How much of the year or season do they stay? Do they also use the program for other times of the year?

Children or youth return rates
Do children and youth keep coming back?

Parent or family involvement
Do parents pick their children up early? Do they attend events or assist in the program?

Staff retention
Do staff work the duration of the program? Do they return again to work another season or year? When they leave, do they continue to work in a similar position or move on to a higher position?

Children or youth satisfaction
What do they say about the program? Do they feel stuck there or can't wait to attend?

Perhaps by focusing on measurements that are obtainable by programs, meaningful data can be gathered and program quality can be assessed not by a multitude of indicators, but by a finite number of contact points that can span programmatic type, approach, and environment.

Learning in Out-of-School

Project-based learning has been around for a long time. Many of the early progressive educators were indeed doing projects. The educational philosopher John Dewey was looking to make learning more real than the factory-style learning he saw about him. In the decades since, there has been innovation in teaching and learning inside and outside of the classroom.

Today, there are student centered classrooms, reading corners, rocking chairs, and carpet areas and many other innovative strategies and frameworks being used inside the school day. There is still a great deal of traditional structure and those who see a further erosion of free time, recreation, and creative problem solving in the school day. There is a great deal of pressure on schools. They are asked to carry many of the traditional roles assigned to parents, and yet, teach a specific body of information and create a common culture. With schools mandated to have delivered outcomes for every hour of the day, cannot out-of-school programs assist by creating a different learning environment not to augment the school-day tasks, or in opposition to the learning styles of traditional education, but to create that social-emotional playground where children and youth can learn by hands-on activities and the socialization that contextualizes all understanding?

Project-based learning in out-of-school is not just about dissecting eels, improving the process of making clay dough, or creating a model of the neighborhood to understand the importance of places, it is also about the process. A time to see what others know, understand how actions impact the group as well as to take individual initiative. The papier-Mâché float, or the community map, or the fundraiser run by the children and youth is not always the largest, or finished on time, or as grand as the original plans, and the outcomes cannot always fit the metrics of the school day. Nevertheless, when the framework is clear, when adult staff are energized about leading projects that dig into a topic over roughly connected activities (theme activities, arts and crafts without context, etc) there is a learning environment that can reach out to children and youth and really dig into the benefits of out-of-school time as a place of social contact and deeper learning.

At a recent training, a group of group leaders were learning how to become project specialist. Being a project specialist means not just leading fun activities, but sequencing activities to answer or explore big questions and fun and engaging topics. The group was led on a simple activity. Make pasta drawings. Everyone seems to have remembered what this meant. Some asked questions about whether they could draw lines first, plan, or what the picture was to be about. They were told that they could do this activity any way they wanted. They were even provided tooth picks to add into their pasta drawings if they wanted. After a short period of time, the group was gathered together. They were told to get up and look at other tables, look at other works, and if need be, make constructive comments. When they were sitting, the group was asked - was this a project.

The unanimous opinion was "yes." It was hands-on, it was an open ended activity, they shared the materials and showed each other what they made. However, what was the deeper learning? "We learned to work together" one staff member said. However, did they now know that before the start of the activity? It may be that it reinforced what they knew, which is not a bad thing, but what new learning did it lead to? Also, did the product they made matter. "No. I'm going to throw it out as soon as we leave." This is similar to how many children respond to the things they are asked to make in out-of-school. The "done and dump" outcome.

The group was then led through what project elements were in the activity but how that activity did not reach as far as we can go in out-of-school - even though it was a fun activity in-of-itself.

The day before at the training one project example given was zombies.

Yes, zombies.

The group revisited that topic - used as a comical example that this group of adult learners - because it spoke to them. "Zombies" then became the focus of our project simulation, even if it was not a child-ready topic. It was engaging for the adults, therefore, they could better see the process and experience some of that fun in learning and exploring we hope children and youth will engage in but which many staff cannot find then discussing clay modeling as if they were themselves children. The group discussed what they would make in a zombie project. The title selected was "Zombie Attack." The final product an escape plan. The group talked and discussed what they needed to learn. "What do zombies eat," "why are they here," "are they fast or slow." The group put down some of those questions as well as "how do we work together" an then looked at the project. The group made a list of things they could do as activities as well as resources. When this was done, they again looked at the learning goals. Were some of those questions short-lived or limited? They asked larger questions which would naturally capture smaller fact-based questions. "What do we need to know about zombies?" was the first learning goal, collecting the other zombie fact questions into one bucket. The next was "what do we need to survive?" This question is not just about zombies, but can be taken as "what do we bring" or at a deeper level "what are the basics of life or things that are important to us?" The Third question was "how do we make a plan?" This came up because the group was divided between where to go (Home Depot or Walmart) and how to get there (run as a group, everyone for themselves, cars, etc). This also meant we needed to revise the "how do we work together" to "How do we help each other survive?" A deeper question and one that goes into community at a much deeper level.

With these questions, we looked at the activities and revised several based on what we could do (simple materials or low schedule impact) and what would take more funds than we wanted to use (e.g. renting a helicopter and getting MTV to film out process). There could be research (zombie movies, books, common lore, internet sites), discussion (debate, shared experience, etc), and field trips (trip to Home Depot, Walmart, looking at the building for how to defend or escape from it). To show how this was done, the group actually was charged with leaving the building simulating a zombie attack. After much discussion, the group did not choose a leader, and when time was called two groups formed and left from two different directions. The escape was hampered by confusion. When we regrouped everyone was sure they would be the survivor, but others would succumb to the zombie hordes. The group then discussed the experience. We were no longer talking about escape and planning, we lived an experience and could see first hand the flaws in our planning ability, our need for leadership or shared decision making, and a need to learn how to help others survive. This comical example was fun, and that is also central to the project. At the end of this process, we could all see that as adults we'd love to do a zombie project - and with children and youth we can see that we want to go deeper into questions and use sequenced activities that get children and youth to move, to laugh, to bring in their own knowledge and to experience something they may not do on their own.

There will be other trainings and who knows what topics will be used as an example. With practice, it seems that this group of learners will now be project leaders and while perhaps avoiding the entire topic of zombies, create fun projects that matter.

Learn more about project-based learning for out-of-school time at www.bostnet.org

Communication and Out-of-School Time

"Communication is key," we are always told. The Not-For-Profit field, and especially Out-of-School Time (including from early care through youth work), relies on "relationship building" and often those relationships are built over time and use a great deal of resources to manage. Site visits, meetings, cohorts, conferences, phone conferences, trainings, forums, mailings, events, and other point of organizational and professional contact take time and money. Recently, much of that money has evaporated. Increasingly (or is it immediately?) dollars are put to point-of-service and skipping over entire networks that have been grown or maintained over the past decade. Of course point-of-service money is needed, however, the infrastructure that was being assembled or had already been in place is also needed. The communications infrastructure of point-of-contact needed to maintain quality programs, a broad base of stakeholders, and professional development. Some of these points-of-contact may be provided by digital media.

Digital communications are not just communicating to funders, but looking at technology to reinvent how non-for-profits engage their stake holders as well as the fluidity that Out-of-School Time programs need to respond to as they meld their program or organizational message with the real-life cultures of the populations they serve. This is perhaps especially challenging for youth-serving organizations since the culture of youths are..... often incongruous with the culture of an organization, especially older mission driven organizations used to the brick and mortar one way control that print media allowed. Youth culture is garish, often filled with crude media images, commercialization and and now able to be uploaded to Youtube and linked to your organization at the click of a button. Scary stuff for staff used to well managed community bulletin boards or quarterly newsletters filled with passe clip art.

The change in communications does not just swap paper and ink for bites of information and blocks of color on a screen, it alters how information is created, consumed, and reinvented. The best example is how bloggers have eroded the traditional news media especially print. It is not just the Boston Globe feeling the burn, it is story after story being broken or covered by bloggers long before reporters with advanced degrees in journalism and years of experience ever get it off the wire. To communicate today is to do so faster, but also giving up a great deal of control over how content is used - or abused. There are risks to this move beyond the traditional fears of Out-of-School Time professions that youth served online may be connected to dangerous elements in society (time and again studies show youth to be more savvy on the WWW. than adults give them credit for). There is an unfamiliar feel to the way an organization has to expose more of itself as well as understand that associations will have to have increased management as well as more awareness by funders that they are seeing a fuller spectrum of who organizations serve and the connections they make - whether the organization enjoys that connection or not.

At the Roundtable a number of leaders and a handful of direct staff attended. It was telling that few direct service staff attended and that some who RSVPed were actually removed from the list by their supervisor. It is interesting to see that the more progressive programs saw all their staff as included in message creation and communication while the traditional organizations continue to think in terms of media marketing and message control - similar to how print has resisted digital and still does not know how to respond (outside of several newspapers shuttering their operations). This may be more of a style issue with organizations, with some able to support a common message while other more grassroots organizations allowing for a faster and looser control and more user generated/ staff generated content. It is hard to say which way is "correct" or whether both are appropriate for their audiences and typology of program.

Several organizations presented their work in communications to give some ideas about how to use the services out there as well as some challenges. Massachusetts Afterschool Partnership presented their MAP Facebook fansite work, while Boston Afterschool and Beyond discussed both the search features created by the BostonNavigator project as well as new work in using digital tracking to monitor youth using social services within the city. Teen Empowerment discussed their Teen Empowerment Youtube Channel and how this is used both by their youth as well as how more professional products are created and shared with funders. The Boston Children's Museum shared their free on-line curriculum especially made for and field tested by Out-of-School educators as well as highlighting that this work may expand - but will always remain free to the educators who need it. What is an interesting link between all these initiatives is that the users - viewer or whatever the new word may be - is not charged. The communication is not to create content that is paid for at the point-of-contact and this is in keeping with the spirit of the internet and new media as well as the challenge to the digital age - how do we pay for all these services?

Of those who RSVP'ed a very unscientific survey was taken using Survey Monkey. It showed that although Facebook and Linkedin use is high, few other communication services are used. This means that perhaps these services are not fitting to the organizations or perhaps the field, but also it may suggest that there is an opportunity for growth in Out-of-School Time communications.

The BOSTnet Roundtable Year in Review

2008/2009 Leadership Roundtable Series Reflection

Overview
For fourteen years, BOSTnet’s Leadership Roundtables have been one of the many professional development services we provide to the out-of-school time (OST) field. They are a key activity in our approach to quality improvement that focuses on building staff skills around identified field competencies. This past year, we updated the format and began holding Roundtables in communities outside of Greater Boston that have limited networking and professional development opportunities. In the past the Roundtable relied on panel discussions in which participants discussed effective practices used in their programs. This season’s offerings combined networking opportunities with research-based training presentations and collegiate conversations to foster a more inclusive learning community. Training topics were sequenced to build a body of core knowledge and to meet the needs of program staff for professional development credits through the Department of Early Education and Care. Thanks to the generous support of For Kids Only After School, Inc., BOSTnet re-launched the Leadership Roundtable on the North Shore while the Shapiro Foundation, the Boston Foundation, State Street, and others provided support for the Boston series to continue into its fourteenth year.

Review of Activities
The sequencing of events was an original concept. It has long been recognized that staff at OST programs experience turnover during or after the summer and there is a need for refresher materials at the beginning of the academic year with older staff in new positions or assuming new program projects. To address this need to orientate OST staff (including directors, coordinators, and new staff), BOSTnet launched the fall series with Quality Environments for Youth, an overview of what makes OST programs a unique developmental setting for youth, how it differs from formal education environments, and its potential as the primary pathway to healthy youth development. This environmental approach to building quality programs represents BOSTnet’s view of the field as a complimentary support for children and youth from what they receive during the school day.
The series continued with Promoting Positive Behavior, purposely front-loaded at the beginning of the academic year to address behavioral and group management issues as programs are assembling. Behavioral support training is consistently one of most requested and well-attended trainings we offer. As the year continued, Project-Based Learning provided a working structure for programs to intentionally frame enrichment activities, meet academic standards, and allow program staff to develop their own fun and engaging projects. The beginning of the Spring Series we had the opportunity to dig down into more complex issues that are a perpetual challenge for providers, including accountability and inclusion. To help support programs faced with shifting expectations and increasingly complex requirements, BOSTnet surveyed the many different tools for measuring program quality and provided a forum for programs to discuss some current issues they are facing in obtaining meaningful data. Inclusion also raised many questions for programs that are seeing many disabilities manifest as behavioral challenges – especially as the definition of disability is shifting to include social-emotional or cognitive disorders. The final Roundtable in the 2008–2009 series set for May 7th, Effective OST Communication in a Digital Age, will discuss program communication, outreach, and new web-based technology.

The presentations provided built upon the knowledge we have gained over the past few years running the Lead to Opportunities for Youth with Disabilities (LOYD), Promoting Positive Behavior, Engaging Families, the Facilities Initiatives, as well as best practices gathered from BOSTnet’s fieldwork. Leadership Roundtables on the North Shore were hosted by different organizations such as Girls Inc., the YMCA of Greater Lynn, Boys and Girls Club of Salem, Community Teamwork, Inc., and the North Shore Community College department of Education who also generously provided refreshments. The Boston Public Library where wireless Internet allowed for “real time” web searches and video feeds hosted Boston Roundtables. Every roundtable was evaluated and reflected upon on the day of the event, elevating key issues and capturing quotes and conversations, while providing transparency as to effectiveness and participant satisfaction. Reflections were posted on the BOSTnet blog at www.bostnet.blogspot.com and indexed according to topic or initiative. Comments were welcome to these pages by interested parties and attendees, allowing for anonymous comments to create a higher level of academic freedom and dialogue in the field. Research support was provided by Michael Bennett Monica Zgola handled logistics. Evaluations were developed and reviewed by Manosi Datta.

Observations
Over the past year we have connected to many great programs and staff and heard inspiring stories from staff that are working hard to make a difference in the lives of children and youth. Nearly 500 attendees participated in the 2008/2009 Leadership Roundtables in Boston and the North Shore. The level of experience of attendees was on average ten years or more in the field. Some commented that they felt most professional development was not raising their abilities but “refreshing” strategies or approaches they already felt confident they knew. There was also consistent representation by “line staff” however; this was more the case for Boston than the North Shore.

Programs are stressed by shifting and increasingly unrealistic expectations, inconsistent directions set by funders and policy makers, and, of course, the economy. Many see a diminishing of the community they found in the OST movement, and are interested in continuing to network and share ideas. A major obstacle to this is the limited funding strategies available to programs that creates a more competitive environment and less cohesion in the movement. Many current leaders (many attendees over 20 years in the field) wonder about the upcoming leaders that are needed to keep the field growing and strong. Response to the Leadership Roundtables was positive, as evidenced by solid attendance, increases in blog readership, and positive reviews collected in evaluations. With fewer professional development dollars available and increasing professional development requirements for licensed programs, many providers see BOSTnet’s Roundtables as their primary staff-training option. We will continue to use participant’s feedback to improve our offerings to better serve the field. A few of the suggestions we are currently evaluating include providing specific Roundtables for different level staff, and providing more afternoon trainings for direct staff when they have paid program time.

The Coming Year
We are currently working with partners to plan the upcoming series and expand the Roundtables in the South Shore and Central Massachusetts for the 2009/2010-year. As BOSTnet assumes its role as the Massachusetts’ affiliate for the National Afterschool Association, some changes may be made to both the delivery and geographic reach of our Roundtables. We are constantly developing new topics and refining old ones based on lessons learned from our fieldwork and research. In collaboration with the DEEC, Roundtables will be evaluated according to standard expectations of training organizations and other entities may be involved in delivering these events or informing the content. Funding for these events may be provided by organizations acting in unison, as was demonstrated by organizations on the North Shore this past year, or assistance may come from private or governmental channels as has traditionally supported the Boston series. The series topics with locations and hours will be listed on www.bostnet.org.

Technology and Out-of-School Time

There is a great deal of information out there through various media. We hear about newspapers being shuttered while also the ascendancy of new forms of communication that not only replace the paper format, but redefine how we communicate and construct knowledge.

That is not to say that the only thing we are constructing is knowledge. There is a great deal of clutter and with each advancement in communication, there seems to be an equal advancement in people's ability to turn something good into yet another stream of trash. The academic papers and open discussions of the Internet became chat rooms and.... pictures of questionable taste. E-mail became SPAM. Viral ideas, became viruses and Trojans or other malware. "Maleware" became a word.

Some react to these forces with a heightened sense of danger. Filter, restrict, create hoops to gain privilege to the network - and this is program staff not an attitude to children or youth. Others have taken the everything new is a new opportunity and jumped further and further to the "bleeding edge" (beyond the "cutting edge" for those of you old enough to remember proper grammar). Some organizations have unloaded millions of hard-raised dollars for web pages, search features, and Java script and encouraged others to "join the digital age." However, the non-profit sector in general and Out-of-School Time in particular seem to have not found a way to raise themselves above the clatter, hold on to a certain amount of control while realizing that new-technology redefines host and user content and may offer a more transparent window into their work - whether they like what outsiders are seeing or not.

Out-of-School programs are slowly coming into the digital age - as are training providers, OST researchers, and other supporting institutions as organizations jump in often using models applied by the corporate world or from personal experience (the hey, I use myspace with my friends so....). This topic will be explored at an upcoming Boston Roundtable organized by BOSTnet Thursday May 7th from 9:30 - 12PM at the Boston Public Library. If you would like to discuss what you do to promote your organization, meet funder needs, or integrate technology into what you bring to children and youth, please attend and be prepared to visit your website and tell others about how this or that aspect of technology captures an audience - or you feel drives away community.

Over ten years after the dot-com bubble/boom/bust, we continue to ask ourselves, is Twitter really going to help us? and then Twitter that exact message so our friends all know what we're doing now is wondering.

For more information on the May 7th Boston Leadership Roundtable or to participate go to www.bostnet.org.

Program Profiles on Youtube

As part of BOSTnet's on-going series of on-line program profiles, KidsArts! was featured. A program that was founded by families almost two decades ago to provide arts education and child care, KidsArts! is currently operating in the recreational area of an historic church in Jamaica Plain. The program has an arts focus, but at the center of this program is building a community of learners and taking care of children. The staff are all artists who are trained to provide educational enrichment and act as teachers. The program takes in children from many schools as well as children who are home schooled. This program is an example of a small scale independent program that fills an important role in the community and may represent the kind of program that appears to have a high level of quality, but that quality may look very different to a more standardized approach. That the program has lasted almost two decades (with one staff member there from the beginning of the program) demonstrates that individual community-based solutions are not necessarily temporary situations depending on a certain group of parents or a limited scope of activities. While KidsArts! may have been created to fill a void left by the cuts in arts funding and programming in the day school so many years ago, the program is no longer about the deficits of formal educational programming. KidsArts! today provides parents of different economics as well as parenting strategies a vibrant learning community for their children. There may be hundreds of similar programs throughout the Commonwealth that do similar work.

Diversity of Children - Diversity of Programs?

What is a quality program? Can there be a single definition of quality for all programs, or does it depend on what a program offers (childcare, arts, sport, academics, socialization, etc.) and the children and youth it serves? We hear time and again that children and youth have diverse needs. If there are different programs, approaches, and environments are there also different standards that need to be applied? Over the past several years, it seems that the public educational system has been searching to become more uniform in approach and measurement. These are but one environment, however. What measurement is given to parochial schools, or the different private schools that espouse every shade of educational approach from the most radical progressive education to the most structured curriculum. How are these measured, or, because they are not "public" - that is run by the government - these other "publics" need not be considered because they fall outside the keen of observation, even if the children and youth of those systems are part of our main-stream American society. What are their quality environments. Considering Out-of-School, will there appear a similar system of quality following the two realms of those programs receiving public - that is government either state or federal - money and another public that uses strictly private funds and is not recognized nor beholden to the standards that are developed. It can be considered that this second group, like the confederation of private schools (these need not be exclusive of rich but may be lower and middle class), be another world?

These questions are part policy, yet belong in a conversation about quality at the point of service because that is where structures and budget expectations come into play with expectations. If all children must do all things, or allowing children to self-select, creates a different set of challenges. It may impact behavior and interactions between staff and child. If certain programs have to align while others opt out through different methods (or programs say they are aligning but find clever ways to go around formal expectations), what does that mean for the quality of experience for the children and youth? This is not to propose any one way has been shown to be better than another.

However, when Johnny hates art but must do it because that is what the program expects of all kids... Does that recognize and meet the needs of the diversity of children and youth? Seeking quality and having standards need not be mutually exclusive. Identifying which standards measure what environment may be important, not just in theory, but impacting the point of service where a child meets the program.

LOYD: Inclusion Roundtable

"There seems to be a growing number of kids with special needs" one program coordinator says. "In the [years] I have worked here, I don't know what it is... they just seem to need more and more." What do you do when all your kids have special needs? One site director reported that in the same program were children presenting ADD, OCD, PTSD, and Spectrum Disorders. Another program director asked whether how to practice inclusion when what seems to work is to remove the child from time to time. "If they're not included in gym, then what? If they are just put on the computer because that's the only way not to cause trouble... are we doing a service just babysitting?" What if the funding has remained flat (or vanished) while the needs increased as well as higher standards of quality. These questions are far above the old format of Inclusion that were challenges are posed and then strategies are employed. When people know several strategies but are overwhelmed, what then? "How do typical kids fit in when many children have challenges"

It appears from comments at the Roundtable that Inclusion is still part of the wealth of services that Out-of-School Time programs can provide and researchers such as Gary Siperstein of the Center for Social Development and Education claim that many school environments can learn from the success of Inclusion in Out-of-School. This is in keeping with the strong relationship-base within many programs and the flexibility that programs have in creating activities that instruct about life as well as socialize. Out-of-School staff want an open environment. They want to make accommodations as well as many at the Roundtable knew the basics about most disabilities and some had worked in therapeutic Out-of-School programs for years. There was a healthy feeling of "try and try again" in letting kids come to the program as who they were and to move them into forming new relationships with their peers as well as push out into new experiences. Nevertheless, there was frustration as it seems that many directors are beyond the "basics" of inclusion and are ready for the next level - a level of complexity that no longer allows for solutions on the cheep but a more thoughtful approach and consistent high-level supports to staff and understand the limits of Inclusion.

One limit would be over-representation of children and youth with special needs within a program. The other limit may be resources. A program that takes in a child or youth who otherwise has a one-on-one during the day may not be able to absorb that child or youth. Programs themselves need to have policies in place that structure how program directors can communicate to families the benefits of including their child in the program as well as some of the limitations. However, if there is a language barrier or family members are opposed to discussing a perceived condition, what then? When a parent demands that their child is included but not with a realistic timeframe or perhaps without taking into consideration the needs of other children or youth at the program, what then? A child with Autism may not be able to run into the program and be included in the first day, or three weeks, but may need time as well as the right amounts of inputs from staff. Too much and the staff forces relationships - too little and there is risk of a benign neglect. This time to negotiate with one family or parent may be available, but what if there are a majority of parents and families with similar demands but divergent issues. Again, the "all or nothing" approach to Inclusion may put ideology over practicality.

One interesting observation from this discussion and the many questions was that the majority of issues were with emotional or emotional disabilities. In an on-line survey of participants 82% wanted assistance on children and youth with ADD/ADHD. Emotional disabilities came in second at 73% and trailing that was working with children with undiagnosed disabilities at 55%. Amongst the "traditional" disabilities such as mobility, cognitive only 9% felt they needed support in those areas.

Programs are asking for increased assistance in providing quality training to staff (78%) and assistance in funding to pay for additional services (82%). This later one is such a complex subject it will be addressed in another posting at a later date.

There is good news and many free resources on-line. BOSTnet has some resources available on Inclusion that may help in training staff and funding opportunities are always included in the BUZZ and the website. Resources on the website are divided into different areas, the most predominant being sports and arts inclusion. These areas were central to a series of BOSTnet events in previous years under the LOYD initiative. The idea that sports (physical activity) and arts can serve as a way to include more children and youth with special needs and that these not form the basis of the program but can be engaged as needed. These areas are useful no matter what the program's own content is - such as academics, because with the flexibility that programs have, setting up time to work on an arts project or engage in physical activity may serve to get children to work together. BOSTnet is also working on the 5th All Means All Conference for 2009 which will be shaped in part by conversations in the field.

While there was a good deal of information covered in the presentation, this Roundtable seems to have raised more complex questions than answered them. We hope that we can work on these issues and find solutions that reinvigorate the field to reach out to children and youth with special needs and not be overwhelmed.

Quality and Accountability - and then there is what you do each day for the kids

When the question was posed to a room of out-of-school program directors, site-based coordinators, researchers, graduate students, and direct-care workers as to what is the intended outcome of their program, the room fell silent.

There was that uncomfortable pause - the one that if cut short would produce cut-and-paste answers, but if held too long, would cause a meltdown in energy and perhaps get participants of the Boston Roundtable heading prematurely for the door.

Then one voice spoke up. "We want to hear kids laugh. We want them to have fun." Then the answers came more easily. "We want to hear that our kids feel self-esteem and make healthy choices outside our program." "We want kids to ask whether the program is running on days it isn't." "I don't want to write kids up [behavior reports]." The entrance to the conversation on evaluation and outcomes in Out-of-School Time was about what people saw at their programs and a sense of what they knew was good for children. This was both something that occurred at their point-of-service as well as beyond their direct control. It was, however, in contrast to the many benchmarks that are increasingly being set for programs linking them to various outcomes and creating an increasing amount of assessment burden on staff already handing the daily tasks of care and enrichment in the face of dwindling resources.

The presentation on tools covered only a certain amount of evaluations and assessments. Some of these "assessments" were actually evaluations (were monitors for program quality or accountability) while other "evaluations" were actually assessments (focused on child outcomes over program competencies). There was discussion about what tools were mandated by which funder or agency and whether certain tools were still required or were about to be phased out in light of new regulations from the Department of Early Education and Care or increasing interest in ELT by the Department of Elementary and Secondary Education (formerly the DOE). This confusion was indicative of the issues of quality ratings system development and the increasing confusion - even cynical burn-out - of many in the field who look for answers and find a multitude of shifting concerns. Many in the group asked who was going to advocate for them in the face of these changes in funding and the increase of required assessment and evaluation alongside a stagnation or decrease of funding.

There were no easy answers.

The presentation of the Boston Roundtable was intended to provide an approach to shopping for the right assessment tools as well as a critical discussion on how programs need to stop consuming every tool that comes their way and to start educating funders (one attendee said "argue with funders" and that may be the case, but we'll stake our middle ground with "educate funders") as to the realities on the ground around key issues of quality improvement. Many people wanted to know which tool was the preferred or met approval by either this organization or that trainer, however, the presentation was not a sales pitch for any one tool - but a call for people to go back to basics and ask, "what are we trying to do with all these things?" Are we researching, or managing our program? If we are using assessment and evaluation as a management tool, who do we really need to be talking to? Is there more value in self assessments? More questionnaires to children and families? Is there a balance between using face validity (going with our personal/professional observations and "gut") and formal processes?

Participants of the Boston Roundtable raised a number of issues that linked to or were directly related to the topic. These included:
The new regulations by the EEC for school-based centers, family child care, and Afterschool and Out-of-School programs do not have a clear enough plan for how to train staff (provide funding for extra time) to administer the child assessments
That arts programs are expected to deliver academics rather than focusing on their individual expertise (this may be extended to sports programs but none were represented at the meeting)
The connections to school may be valid for one kind of program but may not be the model for all programs given the diversity of program type and approach in out-of-school time (perhaps there needs to be a taxonomy of programs was not mentioned but perhaps should be discussed)
Some programs are asked to do one kind of assessment on year as a requirement and then that is dropped for another product
Program directors and on-site coordinators need to focus on staff development and that the most useful tools are those that assess staff competencies and help reduce employee turn-over - especially direct service
Programs need to stop following the system and must look to become more self-reliant economically so that they can meet the needs of the community they are in rather than attempt to align with a standard way of "doing things"

Again and again, there was a call to get organizations to take a stand on obtaining funding for the field to meet new of increased outcomes or adjust expectations to a reasonable level - focusing on what programs have historically done and can provide to our children and youth - supporting quality environments. There seems to be the subject of "quality and accountability" as it is seen through the lenses of several assessment and evaluation tools, and then what you do each day with the kids. It looks like there is a great deal more that has to be done to create tools that assist in managing a program, monitoring quality at point-of-service, and helping stakeholders have confidence in the quality and value of the program.

Perhaps we can all stand up together to bridge this gap between what and why we measure and the job we do every day working with children in our various settings.

YOU CAN FIND OUR QUALITY SELF ASSESSMENT TOOL HERE

North Shore Roundtables: Outcomes and Evaluations

There is a great deal of discussion at the funder level - and the policy - about how programs should assess themselves and how the diverse field of Out-of-School Time (that many say spans all ages and types of programs from year around to summer and weekend) needs to be evaluated.

There are basically two modes of thought: One proposes that Out-of-School Time needs to be evaluated vis-a-vis the school day. That improving programs means aligning them with school methods, norms, and expectations. This is a de facto if not in sutu way of extending the school day - which is a fancy way of saying Out-of-School Time becomes like-school-time. The other mode of thought is that programs need to be assessed based on the specific and unique elements that define a Out-of-School Time program and that evaluation of programs need only be against other programs. If there is improvement in academics in school and that student is a child in an Out-of-School program that is good, but not a benchmark of program quality.

The group gathered on the North Shore were lively and there were many ways people took up the idea of assessment and evaluation. There was discussion that assessment and evaluation are often used interchangeably - or at times mistakenly - to mean a similar thing (just as many competencies or outcomes are not on students but part of elements of effective management) One program director remarked that there is a distinction and one way to avoid confusion is to consider any tool to be assessment as long as it measures specific aspects of the program (an arts project, family engagement, etc) and larger program-wide data to be part of evaluating the program or organization. An interesting discussion was also about what tools promoted an academic-focus or put links to the school day or subjects as major elements of quality - missing the focus on the social-emotional development of children or the long-term values added to the community. In all, most agreed that they were not professional evaluators and that sometimes they will just collect data and "let the funders make sense of this all."

Evaluation and assessment are part of our popular mind. They are enforced by a culture of customer service that has been so pervasive in the past thirty years many of us seem not to remember a time when our civil society gave us what was good for us rather than constantly asking us what we want. Whether this is a positive or negative development has yet to be seen. Years of viewing social services as customer driven and creating an attitude that the public does not have a commonwealth but is comprised of a diverse network of consumers has indeed changed our landscape and how we interact with one another and run our programs. The resources we have placed into assessment and evaluation may perhaps give us a better point of view - or at least numbers to toss around and fun data charts we can post on our walls. However, has this process of constant assessment and evaluation led to an increase of quality at our programs? This question may actually require further evaluation. Perhaps we need to ask only those questions we want to answer - we are consumers after all...

Field Work: ASPIRE

As part of BOSTnet's on-going field work, we are profiling programs as well as collecting individual voices from the field from leaders in Out-of-School Time. This allows unfiltered voices to come through, allowing program staff to speak about their experiences and views as well as making this work more visible to a general audience.

Last week ASPIRE After School Program staff were interviewed about their program. They did not respond to any re-scripted questions but spoke about what they did at the program as well as share some of their passion for youth work. We hope to increase the number of profiles in the coming weeks and to present this work at our Quality Forum in June.

LOYD: What is Inclusion?

What is inclusion? In the classroom, inclusion means to put disabled students in typical classrooms rather than special education settings. Broadly speaking, Inclusion means to provide access to all people no matter what their ability. The issues is then, what is Inclusion when it comes to Out-of-School Time? Is it the "inclusion" of the classroom, or the "inclusion" of the wider disability rights movement?

There remains a controversy to Inclusion - there are many camps in this issue. There are those who claim that inclusion strips services and does not provide the needed supports. There are those who argue that inclusion can be more effective in certain environments. Then, there are those who look at inclusion as more than just traditional disabled and view inclusion as expanding to all needs and abilities in attitude and support. As we discuss inclusion, perhaps we will think more about the division on the issue as well as ways that programs can come together and provide quality supports.

If we look at inclusion as a fundamental aspect of a quality Out-of-School Time program, we then have to tackle what this inclusion means. More people are seeing inclusion as that of including the "traditional disabilities" (e.g. physical impairment, mobility, physical trauma, developmental), and the "new" disabilities (e.g. emotional, social, cognitive). Researchers, such as Dr. Siperstein, director of the Center for Social Development and Education at the University of Massachusetts Boston, Out of School programs may be able to offer a version of inclusion that is broad in definition as it is flexible in getting children and youth with disabilities not only to be next to typical children and youth but to become friends and develop meaningful long-term social interactions with their peers.

This growing evidence demonstrates that Out-of-School practitioners are on to something in seeing their programs as a space that is more fitted to using inclusion as a strategy than more formal settings. It may be that Out-of-School Time programs can adapt to the needs of children and youth and offer a wide range of activities for a version of inclusion - that itself is broad and inclusive.

These are, but the first words in what looks like a longer conversation.

STEM in Out-of-School Time

I love science, and it pains me to think that so many are terrified of the subject and feel that choosing science means you cannot also choose compassion, or the arts, or be awed by nature. Science is not meant to cure us of mystery, but to reinvent and reinvigorate it. -Robert Sapolsky

Science is a wonderful subject to teach children and youth. Earth science almost writes its own curriculum - and the materials are always at hand: water, dirt, organic materials, air, etc. Many people, such as the Coalition for Science After School think that Out-of-School time is a perfect place to do science projects. In a recent development, progressive educators have conceptualized the Science Technology Engineering and Math approach - or STEM for use in and out of school. The STEM initiative seems to thrive in the after school environment and there appears to be increasing research supporting this.

That is exciting that the learning that occurs in OST is beginning to be recognized and the unique developmental setting being used for what it can be rather than molded into an additional classroom experience. With STEM (as with project-based learning or the social emotional development) OST can serve to compliment formal education and in this way both institutions lead to positive youth outcomes. Learning, after all, is academic skills and social emotional development. As Sapolsky suggests above, these need not be mutually exclusive.

However, it will take further investigation to see how STEM can be implemented in programs. BOSTnet is working with several other agencies to realize this goal. As part of a statewide project the STEM pilot is a joint initiative of DHE and EEC. BOSTnet has selected four programs in and around Boston (the other programs are being selected by another organization out in the most western part of the state). These programs will be receiving units of instruction from the Museum of Science's Engineering is Elementary curriculum originally designed for use in the classroom but adapted for use by Out-of-School Time. Program staff will earn how to do a simple engineering project with children and test the idea that these programs can implement a fuller set of STEM initiatives.

The time for science education is always now. It does not matter whether we as a nation are doing worse than other nations in science or better than other nations in science,what matters is that all children know the stages of water, the theory of gravity, and perhaps be able to understand that plastic takes 20 generations to break down into non-complex hydro carbons and to think twice about disposing of thoseplastic toys.

We will move ahead with our own investigation and post further outcomes as we see how STEM can be brought to life in our programs and reinvent and reinvigorate the mystery of the world for all children.