Quality is "Job One"?

The days to the Quality Forum are short - the questions around "quality" as it relates to out-of-school time seem to grow. "Quality" is one of those terms that we hear a great many times - from corporations, from product salespeople, from former automotive giants. Whether "job one" or "continuous improvement" or "quality improvement" or other flashy catch phrases and academically or research supported systems, the question remains - what is a quality out-of-school program?

Also, built into this question is other question. Who asks for this "quality."

If "out-of-school time" is any program that works with children but is not run directly by public education employees then these program include more than "after-school" and reach into museums, summer camps, and enrichment programs - even if some part of these programs take place within the confines of a public school facility or during the hours of school and in collaboration with school teachers or officials. Out-of-school is an approach that transcends programmatic focus, mission, or population as it is a number of very diverse programs bringing ways of learning that may differ (not detract but differ) from the approach the public system is currently taking.

Perhaps, then, quality is different for different programs but can be measured if those measurements are directed to the focus of the program or the ability of staff to create the type of environment where children and youth are cared for and can learn - differently.

And who asks for quality? Yes, regulators, funders, parents, politicos, endorsers, and sundry other influences that have piled on to the backs of programs.

However, it is children and youth that ask for this. True, no seven year old will say, "can I have a quality program and how are you going to assess this so that I know both empirically and based on research drawn from data that was appropriately cleaned and triangulated that my youth outcomes improve based on my attendance" - which seems what many are asking site coordinators and program directors - but "What am I doing today?"

That's at the center of quality, a child or youth asking - "what am I doing today?" Right now, in the moment.

Our job is to make that "doing" fun, engaging, learning, positive, supportive, and safe.

Each one of these elements may be measured in great detail - what is "fun" how do we know when we are having "fun" - however, is that appropriate for a field where the work is not to answer philosophical conundrums but to provide a point-of-service.

How about measuring:

Children or youth participation by attendance of program
What days do children or youth come? When are they picked up? How much of the year or season do they stay? Do they also use the program for other times of the year?

Children or youth return rates
Do children and youth keep coming back?

Parent or family involvement
Do parents pick their children up early? Do they attend events or assist in the program?

Staff retention
Do staff work the duration of the program? Do they return again to work another season or year? When they leave, do they continue to work in a similar position or move on to a higher position?

Children or youth satisfaction
What do they say about the program? Do they feel stuck there or can't wait to attend?

Perhaps by focusing on measurements that are obtainable by programs, meaningful data can be gathered and program quality can be assessed not by a multitude of indicators, but by a finite number of contact points that can span programmatic type, approach, and environment.

Learning in Out-of-School

Project-based learning has been around for a long time. Many of the early progressive educators were indeed doing projects. The educational philosopher John Dewey was looking to make learning more real than the factory-style learning he saw about him. In the decades since, there has been innovation in teaching and learning inside and outside of the classroom.

Today, there are student centered classrooms, reading corners, rocking chairs, and carpet areas and many other innovative strategies and frameworks being used inside the school day. There is still a great deal of traditional structure and those who see a further erosion of free time, recreation, and creative problem solving in the school day. There is a great deal of pressure on schools. They are asked to carry many of the traditional roles assigned to parents, and yet, teach a specific body of information and create a common culture. With schools mandated to have delivered outcomes for every hour of the day, cannot out-of-school programs assist by creating a different learning environment not to augment the school-day tasks, or in opposition to the learning styles of traditional education, but to create that social-emotional playground where children and youth can learn by hands-on activities and the socialization that contextualizes all understanding?

Project-based learning in out-of-school is not just about dissecting eels, improving the process of making clay dough, or creating a model of the neighborhood to understand the importance of places, it is also about the process. A time to see what others know, understand how actions impact the group as well as to take individual initiative. The papier-Mâché float, or the community map, or the fundraiser run by the children and youth is not always the largest, or finished on time, or as grand as the original plans, and the outcomes cannot always fit the metrics of the school day. Nevertheless, when the framework is clear, when adult staff are energized about leading projects that dig into a topic over roughly connected activities (theme activities, arts and crafts without context, etc) there is a learning environment that can reach out to children and youth and really dig into the benefits of out-of-school time as a place of social contact and deeper learning.

At a recent training, a group of group leaders were learning how to become project specialist. Being a project specialist means not just leading fun activities, but sequencing activities to answer or explore big questions and fun and engaging topics. The group was led on a simple activity. Make pasta drawings. Everyone seems to have remembered what this meant. Some asked questions about whether they could draw lines first, plan, or what the picture was to be about. They were told that they could do this activity any way they wanted. They were even provided tooth picks to add into their pasta drawings if they wanted. After a short period of time, the group was gathered together. They were told to get up and look at other tables, look at other works, and if need be, make constructive comments. When they were sitting, the group was asked - was this a project.

The unanimous opinion was "yes." It was hands-on, it was an open ended activity, they shared the materials and showed each other what they made. However, what was the deeper learning? "We learned to work together" one staff member said. However, did they now know that before the start of the activity? It may be that it reinforced what they knew, which is not a bad thing, but what new learning did it lead to? Also, did the product they made matter. "No. I'm going to throw it out as soon as we leave." This is similar to how many children respond to the things they are asked to make in out-of-school. The "done and dump" outcome.

The group was then led through what project elements were in the activity but how that activity did not reach as far as we can go in out-of-school - even though it was a fun activity in-of-itself.

The day before at the training one project example given was zombies.

Yes, zombies.

The group revisited that topic - used as a comical example that this group of adult learners - because it spoke to them. "Zombies" then became the focus of our project simulation, even if it was not a child-ready topic. It was engaging for the adults, therefore, they could better see the process and experience some of that fun in learning and exploring we hope children and youth will engage in but which many staff cannot find then discussing clay modeling as if they were themselves children. The group discussed what they would make in a zombie project. The title selected was "Zombie Attack." The final product an escape plan. The group talked and discussed what they needed to learn. "What do zombies eat," "why are they here," "are they fast or slow." The group put down some of those questions as well as "how do we work together" an then looked at the project. The group made a list of things they could do as activities as well as resources. When this was done, they again looked at the learning goals. Were some of those questions short-lived or limited? They asked larger questions which would naturally capture smaller fact-based questions. "What do we need to know about zombies?" was the first learning goal, collecting the other zombie fact questions into one bucket. The next was "what do we need to survive?" This question is not just about zombies, but can be taken as "what do we bring" or at a deeper level "what are the basics of life or things that are important to us?" The Third question was "how do we make a plan?" This came up because the group was divided between where to go (Home Depot or Walmart) and how to get there (run as a group, everyone for themselves, cars, etc). This also meant we needed to revise the "how do we work together" to "How do we help each other survive?" A deeper question and one that goes into community at a much deeper level.

With these questions, we looked at the activities and revised several based on what we could do (simple materials or low schedule impact) and what would take more funds than we wanted to use (e.g. renting a helicopter and getting MTV to film out process). There could be research (zombie movies, books, common lore, internet sites), discussion (debate, shared experience, etc), and field trips (trip to Home Depot, Walmart, looking at the building for how to defend or escape from it). To show how this was done, the group actually was charged with leaving the building simulating a zombie attack. After much discussion, the group did not choose a leader, and when time was called two groups formed and left from two different directions. The escape was hampered by confusion. When we regrouped everyone was sure they would be the survivor, but others would succumb to the zombie hordes. The group then discussed the experience. We were no longer talking about escape and planning, we lived an experience and could see first hand the flaws in our planning ability, our need for leadership or shared decision making, and a need to learn how to help others survive. This comical example was fun, and that is also central to the project. At the end of this process, we could all see that as adults we'd love to do a zombie project - and with children and youth we can see that we want to go deeper into questions and use sequenced activities that get children and youth to move, to laugh, to bring in their own knowledge and to experience something they may not do on their own.

There will be other trainings and who knows what topics will be used as an example. With practice, it seems that this group of learners will now be project leaders and while perhaps avoiding the entire topic of zombies, create fun projects that matter.

Learn more about project-based learning for out-of-school time at www.bostnet.org

Summer Is Here, Again

Summer is here and city children will soon be spotted wearing agency tee shirts and filling parks and recreational places. Other children will be enrolled in arts camps, band camps, and other camps where the tee shirt is optional or non-extant. Leaders will again discuss key issues about where children and youth go for the summer and what they should be engaged in. There is a great deal of discussion on "summer learning loss" and how summer programs geared towards academics can prevent this loss from occurring. These discussions are generally aimed at urban children and youth, since the data suggests the loss is greatest in urban minority populations. That there is a loss over the summer for children may be true for all children but the effects of this impact those who struggle the most at school or have language or cultural barriers. What to do about this, is less clear.

It may be that all children benefit from great activities that make them think, meet new friends or engage with others in structured an unstructured ways. It would be a shame if inner-city children get a version of summer school while others get:

Trips to a zoo or visits by a nature program
Getting away to a camp for the day or a sleep away
Engaging in a project that involves trips to museums or cultural centers
Cleaning up a neighborhood park or other service projects
Sports programs and athletics

It seems that there is a great deal of discussion between those who want to see summer as an extension of the school process - think "summer school" and those who want to see youth be active in recreation - the "send this kid to camp" tradition. There are those who are looking to meld the two into a hybrid form where children and youth learn formal topics but do so in a more active and participatory learning format. There are many programs that already do participatory learning, however, the press is filled with more conversations about childrens' "time" or following "youth outcomes" (these more than not include events outside the control of the out-of-school program like attendance in school over attendance in the program) or looking at links to this or that curriculum.

Again, summer is here. Children are growing, their minds changing and the road to adulthood is set according to a calendar. We seem to know we want to do something positive with our kids, but as in summers past, we may not agree on what this positive thing looks like.

The BOSTnet Roundtable Year in Review

2008/2009 Leadership Roundtable Series Reflection

Overview
For fourteen years, BOSTnet’s Leadership Roundtables have been one of the many professional development services we provide to the out-of-school time (OST) field. They are a key activity in our approach to quality improvement that focuses on building staff skills around identified field competencies. This past year, we updated the format and began holding Roundtables in communities outside of Greater Boston that have limited networking and professional development opportunities. In the past the Roundtable relied on panel discussions in which participants discussed effective practices used in their programs. This season’s offerings combined networking opportunities with research-based training presentations and collegiate conversations to foster a more inclusive learning community. Training topics were sequenced to build a body of core knowledge and to meet the needs of program staff for professional development credits through the Department of Early Education and Care. Thanks to the generous support of For Kids Only After School, Inc., BOSTnet re-launched the Leadership Roundtable on the North Shore while the Shapiro Foundation, the Boston Foundation, State Street, and others provided support for the Boston series to continue into its fourteenth year.

Review of Activities
The sequencing of events was an original concept. It has long been recognized that staff at OST programs experience turnover during or after the summer and there is a need for refresher materials at the beginning of the academic year with older staff in new positions or assuming new program projects. To address this need to orientate OST staff (including directors, coordinators, and new staff), BOSTnet launched the fall series with Quality Environments for Youth, an overview of what makes OST programs a unique developmental setting for youth, how it differs from formal education environments, and its potential as the primary pathway to healthy youth development. This environmental approach to building quality programs represents BOSTnet’s view of the field as a complimentary support for children and youth from what they receive during the school day.
The series continued with Promoting Positive Behavior, purposely front-loaded at the beginning of the academic year to address behavioral and group management issues as programs are assembling. Behavioral support training is consistently one of most requested and well-attended trainings we offer. As the year continued, Project-Based Learning provided a working structure for programs to intentionally frame enrichment activities, meet academic standards, and allow program staff to develop their own fun and engaging projects. The beginning of the Spring Series we had the opportunity to dig down into more complex issues that are a perpetual challenge for providers, including accountability and inclusion. To help support programs faced with shifting expectations and increasingly complex requirements, BOSTnet surveyed the many different tools for measuring program quality and provided a forum for programs to discuss some current issues they are facing in obtaining meaningful data. Inclusion also raised many questions for programs that are seeing many disabilities manifest as behavioral challenges – especially as the definition of disability is shifting to include social-emotional or cognitive disorders. The final Roundtable in the 2008–2009 series set for May 7th, Effective OST Communication in a Digital Age, will discuss program communication, outreach, and new web-based technology.

The presentations provided built upon the knowledge we have gained over the past few years running the Lead to Opportunities for Youth with Disabilities (LOYD), Promoting Positive Behavior, Engaging Families, the Facilities Initiatives, as well as best practices gathered from BOSTnet’s fieldwork. Leadership Roundtables on the North Shore were hosted by different organizations such as Girls Inc., the YMCA of Greater Lynn, Boys and Girls Club of Salem, Community Teamwork, Inc., and the North Shore Community College department of Education who also generously provided refreshments. The Boston Public Library where wireless Internet allowed for “real time” web searches and video feeds hosted Boston Roundtables. Every roundtable was evaluated and reflected upon on the day of the event, elevating key issues and capturing quotes and conversations, while providing transparency as to effectiveness and participant satisfaction. Reflections were posted on the BOSTnet blog at www.bostnet.blogspot.com and indexed according to topic or initiative. Comments were welcome to these pages by interested parties and attendees, allowing for anonymous comments to create a higher level of academic freedom and dialogue in the field. Research support was provided by Michael Bennett Monica Zgola handled logistics. Evaluations were developed and reviewed by Manosi Datta.

Observations
Over the past year we have connected to many great programs and staff and heard inspiring stories from staff that are working hard to make a difference in the lives of children and youth. Nearly 500 attendees participated in the 2008/2009 Leadership Roundtables in Boston and the North Shore. The level of experience of attendees was on average ten years or more in the field. Some commented that they felt most professional development was not raising their abilities but “refreshing” strategies or approaches they already felt confident they knew. There was also consistent representation by “line staff” however; this was more the case for Boston than the North Shore.

Programs are stressed by shifting and increasingly unrealistic expectations, inconsistent directions set by funders and policy makers, and, of course, the economy. Many see a diminishing of the community they found in the OST movement, and are interested in continuing to network and share ideas. A major obstacle to this is the limited funding strategies available to programs that creates a more competitive environment and less cohesion in the movement. Many current leaders (many attendees over 20 years in the field) wonder about the upcoming leaders that are needed to keep the field growing and strong. Response to the Leadership Roundtables was positive, as evidenced by solid attendance, increases in blog readership, and positive reviews collected in evaluations. With fewer professional development dollars available and increasing professional development requirements for licensed programs, many providers see BOSTnet’s Roundtables as their primary staff-training option. We will continue to use participant’s feedback to improve our offerings to better serve the field. A few of the suggestions we are currently evaluating include providing specific Roundtables for different level staff, and providing more afternoon trainings for direct staff when they have paid program time.

The Coming Year
We are currently working with partners to plan the upcoming series and expand the Roundtables in the South Shore and Central Massachusetts for the 2009/2010-year. As BOSTnet assumes its role as the Massachusetts’ affiliate for the National Afterschool Association, some changes may be made to both the delivery and geographic reach of our Roundtables. We are constantly developing new topics and refining old ones based on lessons learned from our fieldwork and research. In collaboration with the DEEC, Roundtables will be evaluated according to standard expectations of training organizations and other entities may be involved in delivering these events or informing the content. Funding for these events may be provided by organizations acting in unison, as was demonstrated by organizations on the North Shore this past year, or assistance may come from private or governmental channels as has traditionally supported the Boston series. The series topics with locations and hours will be listed on www.bostnet.org.

Technology and Out-of-School Time

There is a great deal of information out there through various media. We hear about newspapers being shuttered while also the ascendancy of new forms of communication that not only replace the paper format, but redefine how we communicate and construct knowledge.

That is not to say that the only thing we are constructing is knowledge. There is a great deal of clutter and with each advancement in communication, there seems to be an equal advancement in people's ability to turn something good into yet another stream of trash. The academic papers and open discussions of the Internet became chat rooms and.... pictures of questionable taste. E-mail became SPAM. Viral ideas, became viruses and Trojans or other malware. "Maleware" became a word.

Some react to these forces with a heightened sense of danger. Filter, restrict, create hoops to gain privilege to the network - and this is program staff not an attitude to children or youth. Others have taken the everything new is a new opportunity and jumped further and further to the "bleeding edge" (beyond the "cutting edge" for those of you old enough to remember proper grammar). Some organizations have unloaded millions of hard-raised dollars for web pages, search features, and Java script and encouraged others to "join the digital age." However, the non-profit sector in general and Out-of-School Time in particular seem to have not found a way to raise themselves above the clatter, hold on to a certain amount of control while realizing that new-technology redefines host and user content and may offer a more transparent window into their work - whether they like what outsiders are seeing or not.

Out-of-School programs are slowly coming into the digital age - as are training providers, OST researchers, and other supporting institutions as organizations jump in often using models applied by the corporate world or from personal experience (the hey, I use myspace with my friends so....). This topic will be explored at an upcoming Boston Roundtable organized by BOSTnet Thursday May 7th from 9:30 - 12PM at the Boston Public Library. If you would like to discuss what you do to promote your organization, meet funder needs, or integrate technology into what you bring to children and youth, please attend and be prepared to visit your website and tell others about how this or that aspect of technology captures an audience - or you feel drives away community.

Over ten years after the dot-com bubble/boom/bust, we continue to ask ourselves, is Twitter really going to help us? and then Twitter that exact message so our friends all know what we're doing now is wondering.

For more information on the May 7th Boston Leadership Roundtable or to participate go to www.bostnet.org.

LOYD: Boston Inclusion Roundtable

The BOSTnet Inclusion Roundtable brought together a small number of program directors, direct care staff, and Inclusion specialists. The presentation initiated a great deal of conversation around the sorts of supports that Out-of-School Time can offer children and youth with disabilities and the need for increased funding of staff development to ensure that programs can provide a quality environment.

What was interesting was that unlike the last roundtable, this group did not say that they lacked the same supports as the school day because they questioned the supports of the school day itself. One support was the "one-to-one aide." It is common to hear many Out-of-School professionals mention that children they include in their programs are done so without the support of these aides and this reduces the quality of their program both for the individual who requires the support by day and for the other program participants. At the Roundtable, many questioned whether the aide model was good for the child in school or whether it set that child apart and allowed teachers and students to remove themselves from care-giving. One OST program staff said that it was a good thing that they did not have aides and that these children who were provided aides or sent to special programs within the school for the majority of the academic day were included in an authentic way during their program time (some mentioned that severe mobility impairment or intense mental or cognitive disabilities that lead to aggressive behavior may need additional supports). This was an interesting to hear since it is very common to hear lists of program challenges around Inclusion rather than a confident point of view that said "schools should learn from us about what we do" rather than "we are deficient in this area without funding."

There was a great deal of talk about how many schools continue to exclude children and youth with disabilities - and that these programs while they may not take place during the entire day are often in re-purposed areas of the school facility; basements, former bathrooms, former closets, former mechanical rooms. While many said that their programs operated in less-than-optimal facility environments, many programs felt that their program did a better job of including children or youth with special needs so that the physical space did not move these members of the community "out of sight." Some of the attendees had personal experiences with the Special Education system and spoke directly of being in a "closet classroom." One staff member said that he was diagnosed with a learning disability while the school support staff never questioned why he had missed so much school in the previous years. So the issue treated was the inability to read "on grade level" not the social issue of truancy. He claimed that this experience of being in Special Education classes increased his difficulties with school rather than elevating them - the primary reason being social/emotional not whether the mechanics of reading were being taught in a different way. His ability to be excepted by peers was impacted as well as self esteem. Another attendee picked up this thread and asked how it was that schools seem to teach exclusion during the day and then leave it to programs to "un-teach" what they have done. "We have to much to do in just a few short hours... We got to show them a different point of view than their school... maybe their family... has on children with special needs" a staff from an arts program exclaimed. "Well, all children have special needs" another staff chimed in. "We have to see that the modifications we make benefit all the children in the program.... I mean, we do this for everyone, not to accommodate only a few."

In all, the Roundtable only briefly touched on the formal presentation. One highlight of the presentation was a short discussion of Dr. Gary Siperstein's work on Inclusion and his thoughts that Out-of-School Time programs actually are bridging the social barriers between children of different abilities and do so because the environments of these programs are focused on relationships first, activities second, and have a flexibility that many formal settings do not have (one example was a program that was in an old building so that mobility impaired children could only attend the basement. The program did not have money to get in an elevator, so they moved the most popular activities - media and computers - to the basement. This created a situation where youth who wanted to access the resources needed to confront the Inclusive nature of the program. One girl was so unfamiliar with interacting with others in wheelchairs that she refused to come in but would stand at the door. By having discussions and allowing interactions to grow organically, this child finally entered the room and got rid of her fears while the program developed a culture of Inclusion that remains to today).

It appears that while there remain challenges (especially around having funding for quality professional development) there is perhaps an emerging attitude of many program staff that they are able to provide a very high quality of service - even if that service does not look like solutions provided during the school-day. While research and the voices from the field tell us that Out-of-School Time is providing Inclusive environments, these programs in the room felt that they were not recognized for the work that they did or the successes they were having.

Diversity of Children - Diversity of Programs?

What is a quality program? Can there be a single definition of quality for all programs, or does it depend on what a program offers (childcare, arts, sport, academics, socialization, etc.) and the children and youth it serves? We hear time and again that children and youth have diverse needs. If there are different programs, approaches, and environments are there also different standards that need to be applied? Over the past several years, it seems that the public educational system has been searching to become more uniform in approach and measurement. These are but one environment, however. What measurement is given to parochial schools, or the different private schools that espouse every shade of educational approach from the most radical progressive education to the most structured curriculum. How are these measured, or, because they are not "public" - that is run by the government - these other "publics" need not be considered because they fall outside the keen of observation, even if the children and youth of those systems are part of our main-stream American society. What are their quality environments. Considering Out-of-School, will there appear a similar system of quality following the two realms of those programs receiving public - that is government either state or federal - money and another public that uses strictly private funds and is not recognized nor beholden to the standards that are developed. It can be considered that this second group, like the confederation of private schools (these need not be exclusive of rich but may be lower and middle class), be another world?

These questions are part policy, yet belong in a conversation about quality at the point of service because that is where structures and budget expectations come into play with expectations. If all children must do all things, or allowing children to self-select, creates a different set of challenges. It may impact behavior and interactions between staff and child. If certain programs have to align while others opt out through different methods (or programs say they are aligning but find clever ways to go around formal expectations), what does that mean for the quality of experience for the children and youth? This is not to propose any one way has been shown to be better than another.

However, when Johnny hates art but must do it because that is what the program expects of all kids... Does that recognize and meet the needs of the diversity of children and youth? Seeking quality and having standards need not be mutually exclusive. Identifying which standards measure what environment may be important, not just in theory, but impacting the point of service where a child meets the program.

Quality and Accountability - and then there is what you do each day for the kids

When the question was posed to a room of out-of-school program directors, site-based coordinators, researchers, graduate students, and direct-care workers as to what is the intended outcome of their program, the room fell silent.

There was that uncomfortable pause - the one that if cut short would produce cut-and-paste answers, but if held too long, would cause a meltdown in energy and perhaps get participants of the Boston Roundtable heading prematurely for the door.

Then one voice spoke up. "We want to hear kids laugh. We want them to have fun." Then the answers came more easily. "We want to hear that our kids feel self-esteem and make healthy choices outside our program." "We want kids to ask whether the program is running on days it isn't." "I don't want to write kids up [behavior reports]." The entrance to the conversation on evaluation and outcomes in Out-of-School Time was about what people saw at their programs and a sense of what they knew was good for children. This was both something that occurred at their point-of-service as well as beyond their direct control. It was, however, in contrast to the many benchmarks that are increasingly being set for programs linking them to various outcomes and creating an increasing amount of assessment burden on staff already handing the daily tasks of care and enrichment in the face of dwindling resources.

The presentation on tools covered only a certain amount of evaluations and assessments. Some of these "assessments" were actually evaluations (were monitors for program quality or accountability) while other "evaluations" were actually assessments (focused on child outcomes over program competencies). There was discussion about what tools were mandated by which funder or agency and whether certain tools were still required or were about to be phased out in light of new regulations from the Department of Early Education and Care or increasing interest in ELT by the Department of Elementary and Secondary Education (formerly the DOE). This confusion was indicative of the issues of quality ratings system development and the increasing confusion - even cynical burn-out - of many in the field who look for answers and find a multitude of shifting concerns. Many in the group asked who was going to advocate for them in the face of these changes in funding and the increase of required assessment and evaluation alongside a stagnation or decrease of funding.

There were no easy answers.

The presentation of the Boston Roundtable was intended to provide an approach to shopping for the right assessment tools as well as a critical discussion on how programs need to stop consuming every tool that comes their way and to start educating funders (one attendee said "argue with funders" and that may be the case, but we'll stake our middle ground with "educate funders") as to the realities on the ground around key issues of quality improvement. Many people wanted to know which tool was the preferred or met approval by either this organization or that trainer, however, the presentation was not a sales pitch for any one tool - but a call for people to go back to basics and ask, "what are we trying to do with all these things?" Are we researching, or managing our program? If we are using assessment and evaluation as a management tool, who do we really need to be talking to? Is there more value in self assessments? More questionnaires to children and families? Is there a balance between using face validity (going with our personal/professional observations and "gut") and formal processes?

Participants of the Boston Roundtable raised a number of issues that linked to or were directly related to the topic. These included:
The new regulations by the EEC for school-based centers, family child care, and Afterschool and Out-of-School programs do not have a clear enough plan for how to train staff (provide funding for extra time) to administer the child assessments
That arts programs are expected to deliver academics rather than focusing on their individual expertise (this may be extended to sports programs but none were represented at the meeting)
The connections to school may be valid for one kind of program but may not be the model for all programs given the diversity of program type and approach in out-of-school time (perhaps there needs to be a taxonomy of programs was not mentioned but perhaps should be discussed)
Some programs are asked to do one kind of assessment on year as a requirement and then that is dropped for another product
Program directors and on-site coordinators need to focus on staff development and that the most useful tools are those that assess staff competencies and help reduce employee turn-over - especially direct service
Programs need to stop following the system and must look to become more self-reliant economically so that they can meet the needs of the community they are in rather than attempt to align with a standard way of "doing things"

Again and again, there was a call to get organizations to take a stand on obtaining funding for the field to meet new of increased outcomes or adjust expectations to a reasonable level - focusing on what programs have historically done and can provide to our children and youth - supporting quality environments. There seems to be the subject of "quality and accountability" as it is seen through the lenses of several assessment and evaluation tools, and then what you do each day with the kids. It looks like there is a great deal more that has to be done to create tools that assist in managing a program, monitoring quality at point-of-service, and helping stakeholders have confidence in the quality and value of the program.

Perhaps we can all stand up together to bridge this gap between what and why we measure and the job we do every day working with children in our various settings.

YOU CAN FIND OUR QUALITY SELF ASSESSMENT TOOL HERE

STEM in Out of School

In a time of rather depressing news - take your pick as to what that may be for you personally or for your program or organization, there is some really interesting movement in getting Science, Technology, Engineering, and Math projects into Out-of-School Programs.

Currently the United States is not faring too wall in science education when compared to other countries. This may be due to how it is hard to measure one country against another. The difficulty to uncover standards or norms when there are so many variables and the fluid nature of knowledge when measured by strict instruments. Perhaps comparing Finland and the United States is like comparing informal learning environments to formal education. Or holding informal learning environments responsible for what occurs during the school day. Nevertheless, many researches think we can measure Finland against the United States and clean the data to give us a clear picture of the quality of our science education in this nation. If that is the case, as so many educational statisticians argue, we are not doing too well when compared against that and other nations. In fact, we rank #29th of 57 nations. Finland is #1.

Leaving the debate over evaluation and assessment of nations or learning, we can always use an extra hand in science education. Some say that our nation's financial health can only be saved by increased investment in science education.

This has led some to identify Out-of-School Time programs as the logical place for expansion of learning - and a natural place for learning that is by its nature out of the classroom and outside of the tests and answer sheets associated with current educational systems and academic-focused Out-of-School Time environments. Out-of-School Time programs have used projects for years to enrich their offerings and raise their quality.

Nevertheless, we will have to see whether STEM can work in the OST environment and the latest pilot to do so runs now until the end of summer. There are two groups in Massachusetts one in the west managed by WESTmost and the other in the east. The Eastern Cohort Group is run though a partnership of Build the Out of School Time Network, The Department of Early Education and Care, Department of Higher Education, Massachusetts Afterschool Partnership, The Museum of Science, and the Donahue Institute. The four sites and representatives of the six partners gathered today at City Year in Boston to learn about how to teach engineering using the Museum of Science's Engineering is Elementary curriculum. The training covered the fundamentals of engineering - have goals and ask questions until you solve these challenges with the materials at hand. The group was surprised at how simple materials were all that was needed. They also learned that through projects many children who may get factual answers wrong on a science actually can innovate in projects and become engaged in ways they may not through text books and discussions.

Over the next coming weeks, this group of STEM pilots will launch their projects and we will see what potential there is for this work and what challenges programs face in implementing these projects looking at what supports are needed, whether "out of the box" curriculum works, child engagement, and if these projects are fun for staff and children.

At a point in time when science, technology and engineering are opening all these incredible potentials in our world from micro-computers to green technologies, the United States is falling behind in the production of graduates in these fields. We need to do something towards growing the next generation of STEM workers. Already "by 2010, only four years from now, more than 90 percent of all scientists and engineers in the world will live in Asia." We have our work cut out for use to bring our rating up from #29th in the world science community and to make this country again the innovation powerhouse we are accustomed to.

Point of Service Quality

There is a great deal of discussion about the impact and quality of out-of-school programs - about as much as there is continued discussion on the definition of what out-of-school programs are - Early care? Elementary school? Teenager programs? Weekends? Summer programs? Are we stuck in a world of dead ideas and ways of seeing things that are out modded as Fortune Magazine's Matt Miller claims our nation as a whole is? Can we still provide quality service and keep the process of delivering that simple and effective?

Many providers in this year's BOSTnet Cohort of programs have discussed their frustration as to the definition of quality and the many and often competing tools and opinions out there vying for recognition. "I don't find much use in [an outside quality assessment tool]" one program leader claimed. "I do those [outside quality assessment tools] because I have to. Then, I create my own to learn what I need to know to improve my program." Are these tools focused on measuring dead ideas?

Many programs are asked to be held accountable to improving school-day performances either surrendering their unique offerings and substituting increased formal academics ("classroom" style learning) and linking their outcomes not to those that a unique developmental setting can provide but those improvements that may be very much out of their control - those performances of the classroom of such-and-such public school or schools depending on the composition of the program.

"We need to focus on what we do, and what we do well" seems to be the opinion of many leaders within the cohort yet upper level management often stifles innovation by holding on to older ideas of management and definitions of the field. Many emerging leaders as well as veterans see themselves needing high-level supports yet do not feel this clearly expressed need is being met. Again, older ideas of training and professional development continue to send experienced people back to the same old trainings they had the year before. How many "refreshers" are needed? "I am not sure why I am expected to attend a [professional development training] and expect to cover the same material that entry level staff are" one long-time program manager said. It is strange that in this field so many of the trainings are expected to meet the needs of all staff from the upper levels to the line-staff - not to mention the unrealistic expectations that a single trainer can "adapt" the materials to meet all these different needs in a single contact period.

Focusing on the needs of each level of staff and increasing the number of upper level opportunities may be one way that more experienced leadership can continue to grow and to shape their programs. This experienced leadership seems ready now, perhaps after years of developing the field and watching the out-of-school time community grow, to focus on what they do well, and reach to measure the sort of Point-of-Service results that are representative of their true contribution to healthy and resilient youth.

A viral video from NASA may shed some light on this situation since many issues of innovation and management cross boundaries and perhaps what is good for healthy science is good for healthy out-of-school time. Perhaps we need to throw out dead ideas and ways of framing what outcomes we are measuring and replace that with a focus on measuring the quality at the point of service and only those things we have direct control over in the multi-layered and ever complex lives of the children and youth we serve.

Building Careers in Out-of-School: Non-traditional workers

The other day the BE SAFE collaborative had their first annual meeting. BE SAFE is a group that wants to "maximize the ability of OST programs to provide positive environments for youth." This is very much a push to use OST programs as good places of social-emotional support and a means to train staff at these programs to talk about issues they may not feel confident in discussing because of a lack of knowledge or perhaps are afraid to address because of our litigious culture of liabilities and opposing cultural viewpoints. Whatever the quality of work of BE SAFE is not under review here.

What is of interest at the BE SAFE event was the group assembled on "stage" and in the room. Be SAFE appears to give simple tools that anyone who enters youth development can use. These resources are not intended to make any youth workers experts in the field of mental health but to allow them to respond to young people's needs with professional level supports. Increasingly, thought leaders and policy advocates are discussing a "star system" for program quality. One proposal floating out there is to get more staff certified or credentialed. This may require staff to commit to a regime of college classes, 45 hours or community training, professional observations, interviews, a portfolio or at least attend 5 - 40 hours of "orientation training" before their first day of work. While we want workers to be competent, can we create a competent workforce that understands the temporary nature of our workers and the limited resources of time and money for the field of OST?

The BE SAFE panel was a very diverse group of youth workers from all ages and backgrounds. A representative from a reproductive rights organization, the head of a sports/academic program - a former principal - two youth leaders, a teen center head, a representative from a domestic violence prevention organization, and a farmer.

Yes. A farmer.

This farmer, we will call her Jane the Farmer with a wink to the spirit of recent media events, had a background in organic farming. She did not think she had what it took to work with young people but seemed to have done such a good job in her short time in the field that she was featured on this panel. The organization she works for has youth development as a key aspect of their work, but it is not the only part of her organization's responsibility. Running a farm seems the primary activity. Yet, Jane the Farmer was a youth worker. OST gains an individual with knowledge of a much different field - no pun intended - and brings with her all sorts of content knowledge she and can share that with young people. In a season's time or two, she may leave youth work for more intensive farming or private farming whatever her life goals may be. In the meantime, youth benefit and the OST field of researchers, arts education specialists, inclusion specialists, and executive directors has a farmer working along side them.

OST programs need to be seen as quality programs for youth development. Professional development is a key to providing these quality services since competent staff are the life of any program. However, there needs some way to achieve this quality without closing the gates of youth development to those who can afford to commit to certificates that may take months or years. Considering that youth development work pay for direct service falls roughly between $8.25 - $11 an hour and that even program directors/coordinators may earn around $21,000 per year, are there ways of getting people to learn skills while they work? Credentials seems like an expensive and potentially risky solution. It follows the path set by colleges and universities rather than service economy jobs. The "pass/fail" certification process itself has currently only produced only 13 accredited programs within Massachusetts. Would credentials and certification of workers have a similar bottle neck effect?

Would a certification process following traditional "school of education" models prevent her and people like her from entering the field? Can the field be permeable to these skilled and caring individuals or must we close ranks as teachers have in creating their own system of "closed shops"? Upper level OST leaders may think certification will elevate the field. It may. It may also lead to a more organized workforce, and perhaps workers who cost more than the current rate of pay - robbing us of temporary talent and meaningful jobs that hire from and serve very local communities.

Boston Roundtable Reflections

The group of programs that gathered were indeed diverse in their approaches. Of the programs that attended, 74% said their staff led academic activities (homework included) 31% STEM activities (Science, Technology, Engineering, and Math), 66% visual arts, 41% music, 38% dramatic arts, and only 15 media arts. What united these programs was not their topics of choice but their approach - all these programs could and many already supported project-based learning as an approach to create some structure around very open learning.

The framework is simple. Have fun projects that are guided by staff interests rather than set curriculum. Create three or four learning goals set up as open-ended questions you can ask the children you are working with (not "you will learn about dinosaurs, but "what creatures lived long ago"?) so that children are able to begin exploring these questions with what they already know rather than waiting for you to tell them answers. Set a goal that is tangible and achievable. Change these plans to go around obstructions and challenges.

To reach this understanding the room was given some Learning Goals

1. what are fun projects I can do with children and youth?
2. how to I plan my project?
3. what will I need to make an effective project?
4 what skills do I need to work effectively with youth?

To answer this we:

Created projects using the project planning sheets (learning goals 1 and 2)
Did a hands-on activity with soda cans (learning goals 1, 3, 4)
Did a hands on activity with apples (learning goals 1 - 4)
Listened to a short lecturette about the fundamentals of project-based learning (learning goals 1 - 4)

Our final product (for many of us) was a project plan that was started we can take back and use at our sites.

Our evaluation was - well, the evaluations everyone always does at the end of any event, training, gathering, or movement. From this evaluation we learned that 94% of the people felt it was a good use of their time. We also learned that 78% of the people self-identified as "direct service" even though 48% of those people were directors and 5% were engaged in fund raising for their site or organization.

In all, what was interesting was the level of engagement - especially during the apple activity where people all made really complicated artistic creations out of their apples rather than cutting them up only in a scientific way (here, we had the flexibility to allow that rather than a narrow definition of what is appropriate). What was challenging was getting through so much material in such a short amount of time (I have in the past done entire summer institutes on this topic for OST members) and that OST has, after all these years, not embraced PBL an approach that allows learning to be done differently out of school.

The pervasive mentality appears to be that of formal education, get them to do their homework and if there are activities they are aligned with curriculum and done with expensive specialists. This prospect to link school and after school is tantalizing yet why keep linking things to create a "seamless day"? Think of yourself eating corn all day. All you eat is corn. Would you want, given an opportunity, to eat more corn or try something different? No one is saying the corn isn't good for you... But, why, if we are given the opportunity to lead new and exciting projects, do we want to struggle with our meager budgets to align with schools to do more of the same - the thing that may not be working for all children in the first place?

Project-based learning has worked in many programs, but I would be the first to admit that after ten years of teaching it, few programs see it as the center to a unique developmental setting rather than an add-on when they have solved behavior, enrollment, vouchers, and relations with the school which may or may not be assisting or hindering the growth of the community-based program. Project-based learning has, after all, been the way private progressive schools have moved learning out of books and into the minds of children. Do we at least want that opportunity for all children and can we not provide at least a taste of that approach in our Out-of-School programs?

Project-Based Learning?

This next month, BOSTnet will focus on Project-Based Learning. The idea of Project-Based Learning (PBL) has been around for a number of years and has its roots in progressive education - a formula that seems for the time being out of fashion.

In this age of MCAS and increased testing, improvement rubrics imposed for the best intentions but often resulting in a decrease in the quality of life for young people - even if there is am improvement in their test scores - there has been pressure on programs to assume more of a teaching capacity and to do so in alignment with the school day and increasingly using the tools and techniques of the school day rather than exploring how our Out-of-School time field can provide a unique developmental setting.

People at the ground level - those who work with children - are increasingly saying their children are stressed and tired by the time they get to their program. More staff are commenting aside that they are feeling overwhelmed by their programs shift towards formal academics and competing with the extend school day movement.

What are program staff to do? Can they be told to teach and not be compensated and trained as teachers? How are programs going to respond when it appears the next fad is to extend the school day and funding and opportunities all - for this moment - point them in this direction?

Schools perhaps can benefit from a more project-based approach (non-parochial private schools and home-schoolers/ unschoolers apply this method successfully) while programs can always use additional resources to attract and retain staff, additional educational materials, and sustained and sequenced training to get the most out of the kind of staff who work in Out-of-School Time.

Our next Roundtable will explore Project-Based Learning in detail and present an easy to use framework that programs can use to retain or expand the type of learning they do - even if that learning is digging for worms and asking big questions rather than memorizing hard answers and increased worksheets.

You can sign up to be part of this training and learn more about BOSTnet's professional development Roundtable Series HERE or at www.bostnet.org

North Shore Directors Roundtable Reflections

Hosted by the YMCA of Peabody, the NDR gathered together providers from the North Shore region of Massachusetts. The presentation on Promoting Positive Behavior was the second in BOSTnet's series of free professional development for this community. Behavior is a very requested topic by programs as they seek to train a staff of varied backgrounds to work with children with varied needs. Promoting Promoting Positive Behavior is an approach with supports (tools and some on-site interventions) that seek to establish a culture of prevention and recognition that many challenging behaviors stem from staff needing to improve their group management and to establish strong relationships with the children or youth, some behaviors are passing or have no "quick fix" solutions, and a few children have special needs above what an average program can provide. The presentation was perhaps too much of an overview at times since the average experience level in the room was 13 years, however 50% of attendees greatly enjoyed the round table with 80% listing the presentation as beneficial. It was surprising that 99% of attendees supervised staff but not so surprising given OST that those same people also were administrators (95%), provided some level of direct service (59%), as well as raised funds (40%), or "did everything as program director." Also not surprising was that this seasoned staff had attended "tons," or "a lot," and even "a million!" trainings on OST subjects. While many say this is a "new field," this level of experience needs to be better used so that these leaders can not only be recognized for what they already know, but can pass these skills on to workers as well as future directors.

Some interesting moments of the event were the comments time and again saying that OST needed to distinguish itself from school and to be recognized for the value it brings as youth development and that the flexibility of programs and and the element of fun and engaging activities that have, as one practitioner said, "children running into the building." "When do you see children running into school?" Many directors said that they did not feel recognized by schools and that they did not see the benefit to "become more like schools... or being put into a partnership where you are bossed around." This is interesting to hear since there is a push for increased partnerships between programs as well as more emphasis on school partnerships and programs structuring themselves as extensions of the school day. "They're robbing them of their childhood" one director said in conversation. These are very strong words and indeed, the issue at stake is whether OST remains a distinct field that continues to grow, or is absorbed into the school system.

Out-of-School Time programs generally solve behavioral issues through a more relations-based approach, that certain behaviors are accepted (skipping, laughing, having fun with friends), and some behaviors are encouraged (asking questions, having a say in choosing an activity, voicing opinions). This is in contrast to an increasing use of "zero tolerance" at younger ages and a "school to prison pipeline" within many school districts, and certainly more so in low-income districts. The behavioral approach that encourages discussion and prevention may not work in the confines of school. However, this does not mean that this approach is not valid for the kind of environment most directors want their programs to be for young people.

For any partnership to thrive, it must be between equals.