Readiness for Quality Ratings Varies among Milwaukee Afterschool
Transcription
Readiness for Quality Ratings Varies among Milwaukee Afterschool
VOLUME 100, NUMBER 4 OCTOBER 2012 Readiness for Quality Ratings Varies among Milwaukee Afterschool Programs Milwaukee’s afterschool programs vary considerably in terms of their attitudes about, understanding of, and readiness for new state quality ratings. As of July 2012, all child care and afterschool programs serving school-age children receiving state child care subsidies are required to participate in YoungStar, the state’s quality rating and improvement system. The Public Policy Forum surveyed 64 programs across Milwaukee in June 2012, finding that a significant number were uncertain or pessimistic about YoungStar’s potential impacts on quality (Chart 1). In addition, less than half of survey respondents felt certain they would participate in YoungStar. School-age programs located in group child care centers were much more likely to indicate they would certainly participate than schoolbased programs. When asked about specific elements of quality reflected in the YoungStar rating system, more than half of all programs reported having many of these characteristics, including tracking student outcomes and conducting self-assessments of program quality. Programs most likely to have these types of quality characteristics were 21st Century Community Learning Centers; school-based programs located in private schools were least likely to have many of these quality characteristics. Barriers to improving quality that were identified by survey respondents included the costs of training staff, a lack of training opportunities for staff, and pressure to keep costs down. These survey results provide a descriptive snapshot of afterschool programming in Milwaukee that should be useful for state administrators and policymakers charged with implementing and evaluating the new quality rating system, as it appears Chart 1: In your the disparate nature of the opinion, how likely is it afterschool system in Certainly Not that the application of Milwaukee may not lend 13% sure/no the YoungStar quality itself to a “one-size fits all” answer rating and approach to quality 40% improvement system improvement. At least to school-age somwhat afterschool programs likely 32% will improve the Not likely quality of the program 15% at this site? Public Policy Forum 633 West Wisconsin Avenue, Suite 406 Milwaukee, Wisconsin 53203 414.276.8240 www.publicpolicyforum.org Research by: Anneliese Dickman, Research Director Joe Peterangelo, Researcher Research funded by: The Argosy Foundation The Herzfeld Foundation LISC Milwaukee JP Morgan Chase Foundation 2 Data and methodology The 21-question survey was conducted online from June to August 2012. A mailing list of 431 probable afterschool programs/providers in the City of Milwaukee was compiled from nine sources: 21st Century Community Learning Centers, Milwaukee Public Schools School-based afterschool camps, Milwaukee Public Schools Group child care centers serving schoolage children, 4Cs for Children, Milwaukee Group child care centers serving schoolage children receiving state child care subsidies, Wisconsin Department of Children and Families (DCF) Private schools participating in the Milwaukee Parental Choice Program indicating afterschool hours, Public Policy Forum Charter schools, City of Milwaukee Charter schools, University of WisconsinMilwaukee Boys and Girls Clubs, Boys and Girls Clubs of Milwaukee Non-school agencies participating in the National School Lunch and Child and Adult Care Food Programs, Wisconsin Department of Public Instruction (DPI) All charter schools were included to capture as many school-based programs in addition to MPS as possible. Private schools previously indicating to the Forum that afterschool activities are offered were included for that reason as well. To ensure representativeness of stand-alone and other programs operating outside of school buildings, we included all group child care centers who indicated to either 4Cs or DCF that they could serve school-age children, as well as all Boys and Girls Clubs and agencies serving federally-funded afterschool meals. Because this multi-year research project is focused on afterschool programs that serve students every day for multiple hours after school throughout the school year, we did not attempt to survey programs that are known to not operate daily, operate only in summer, or not operate for the entire school year. In addition, we excluded family child care providers from the survey sample. Because of the length and complexity of the survey, administering the survey instrument online was determined to be most cost-effective. Unfortunately, email addresses were not available from all sources; thus, each potential afterschool Table 1: Survey response rate and representativeness Total 21st Century CLC School-based program Child care center Stand alone afterschool program Other program types Boys and Girls Clubs MPS school-based Charter school-based Private school-based Sample Count Distribution 64 25 39% 21 33% 13 20% 5 8% 19 29 3 11 30% 45% 5% 17% Mailing list Count Distribution 431 51 12% up to 94 22% up to 262 61% up to 24 6% 39 117 11 38 9% 27% 3% 9% Response rate 15% 49% 22% 5% 21% 49% 25% 27% 29% 3 program was mailed a large postcard via U.S. Post asking them to visit the survey website to participate in the survey. Each postcard recipient was assigned a survey ID number, which he or she was asked to enter into the website to ensure only one survey response was received from each program site. The 64 responses received equal an overall response rate of 15%. This somewhat low response may be due to the timing of the survey late in the school year when programs may have been winding down, a possible lack of internet access at some sites, and/or a significant number of sites on the mailing list not actually offering a school-age afterschool program. Thus, the true representativeness of the survey sample is difficult to measure, as non-response may or may not reflect a site’s status as lacking an afterschool program. Table 1 shows the breakdown of various types of programs reflected in the survey sample as compared to the mailing list of potential programs. The distribution of responses across program types is fairly representative of the mailing list, although 21st Century Community Learning Centers (CLCs) are over-represented. The distribution of the sample by zip code is also representative of the mailing list geographically (see Map on page 13). Table 2: Survey representativeness of students Total 21st Century CLC 7610 5262 PPF estimated total Response enrollment rate 43402 22922 Chart 2 shows the distribution of 21st Century CLCs and school-based programs in the survey sample by the type of school/sponsor site. Note that although many 21st Century Community Learning Centers are sited in school buildings, for the purposes of this report we treat them as a unique program type and do not categorize them as “school-based.” Chart 3 shows the total enrollment by program type. Chart 2: Program sites represented in sample Stand-alone program 5 Child care center 10 Public school in partnership with a community agency 28 Public or charter school Private secular school 5 2 Private religious school Table 2 shows the response rate as determined by enrollment. The programs Sample school-age enrollment responding to the survey enroll nearly one-fifth of the children we estimate to participate in afterschool programs in Milwaukee. For schoolbased programs and 21st Century Community Learning Centers added together, the survey captures responses from programs representing one-fourth of all children in the city thought to attend. 10 Chart 3: Survey sample total enrollment by program type 5262 18% 23% 1422 School-based program 1422 4127 34% Child care center 826 16353 5% Note: Calculation of estimated enrollment can be found in an earlier PPF report, “Afterschool in Milwaukee: Is it child care?” July 2012. 826 100 21st Century CLC Group child care School-based Stand-alone program program 4 On average, programs reported that about half of their students attend all of the time (five days per week) and 22% attend more than half the time (four days per week), indicating that chronic absences occur among a small portion of students, but that student engagement still could be problematic in many programs (Chart 6). There are variances in these patterns by program type and by age group, however. Chart 7 on the following page shows that among the preschool age group (four-year-old Kindergarten), school-based and 21st Century CLCs had higher attendance rates than group child care. Among elementary students, the reverse was true. At the middle school age, very few group child care centers in the survey enrolled significant numbers of children and the attendance rates at the other programs were lowest for this age group, reflecting the fact that these students are old enough to be more independent and are more likely to make choices about where and how they spend their time after school. 4000 Enrollment 3500 Capacity 3000 Attendance 2500 2000 1500 1000 500 0 Preschool Elementary school Middle school High school Chart 5: Average program enrollment, capacity, and attendance, by age group 100 86 83 Enrollment 80 60 40 Capacity 59 Attendance 39 24 27 21 34 22 23 25 20 18 0 Preschool Elementary school Middle school High school Chart 6: Attendance pattern at average program all of the time, 52% 0 20 half of the time or less, 9% more than half the time, 22% 40 60 Percent attending 80 sporadically, 5% Many programs can thus enroll over capacity because attendance is less than enrollment. Chart 4 shows the total capacity of survey respondents compared to total enrollment and total attendance. The overall pattern holds true for individual programs, as well, as shown in Chart 5, representing the average program serving each age group. 4500 Total count Research has shown that regular attendance in afterschool programs is more effective in improving student outcomes than sporadic attendance. In fact, several studies use average daily attendance as a proxy measure for quality. The nature of serving school-age students makes it difficult, however, to ensure all enrolled children are in attendance each day. Students also participating in school sports teams or arts performance groups may have conflicts after school, for example. In addition, older students who are not in need of child care may or may not choose to attend a program on a given day. Chart 4: Total enrollment, capacity, and attendance, by age group Average count Enrollment, capacity, and attendance 100 5 Chart 7: Average enrollment, capacity, and attendance by age group and program type 140 Average Attendance Average Capacity Average Enrollment Average Capacity 20 Average Attendance 40 Average Enrollment 60 Average Attendance 80 Average Capacity Average count 100 Average Enrollment 120 0 21st Century Group child School-based 21st Century Group child School-based 21st Century Group child School-based Community care center program Community care center program Community care center program Learning Learning Learning Center Center Center Elementary School Middle School With the exception of the 21st Century CLCs, very few programs reported maintaining a waiting list for enrollment (Chart 8). The 21st Century CLCs operate somewhat differently from other programs in this survey in this regard because of their funding structure, which necessitates a waiting list for some programs. See page 11 for more on program revenue. Chart 8: Programs maintaining wait lists The majority of programs reported they can serve special needs students. Table 3 provides a breakdown by program type. School-based programs in private schools and stand-alone programs were much less likely to accommodate student with special needs than other types of programs. Most programs (62%) also reported offering an afternoon snack and 42% offer an afternoon meal/dinner. Of the 25 programs operating in the morning, 64% offer breakfast. 23% Total Stand-alone School-based program 0% 5% Group child care 20% 21st Century CLC Program offerings Of the 45 respondents answering a question about program schedules, all reported having afternoon programs and 25 reported having morning programs as well. Six of the respondents do not operate on Friday afternoons, five of which are 21st Century CLCs. Thus, most respondents make programming available five days per week. Preschool 44% Table 3: Programs serving special needs students Total 65% 21st Century Community Learning Center Group child care center 75% 89% School-based program-public school 86% School-based program-private school Stand-alone school-age program 14% 0% Table 4: Number of programs offering transportation Transportation to program from home (a.m.) 4 Transportation to school (a.m.) 6 Transportation to program from school (p.m.) 7 Transportation home from program (p.m.) 9 6 Table 5: Activities and opportunities offered Percent answering “yes” Academic support/tutoring/homework help Recreation/structured physical activity Social development activities Field trips Visual and performing arts education/ enrichment Leadership opportunities/skills development Mentoring Community service projects STEM learning/projects Career and college preparation Other 21st School- SchoolStandCentury based based alone Community Group program- program- schoolLearning child care public private age Center center school school program Total 100% 100% 85% 80% 78% 100% 89% 89% 100% 100% 57% 57% 100% 40% 60% 20% 100% 67% 33% 0% 93% 89% 73% 64% 80% 22% 86% 0% 67% 58% 80% 55% 70% 44% 33% 22% 43% 57% 29% 40% 40% 20% 33% 33% 33% 58% 47% 44% 45% 40% 10% 11% 22% 11% 29% 14% 0% 0% 0% 0% 0% 0% 33% 27% 24% 9% Forty-two programs provided information on transportation, with most indicating they offer no transportation (Table 4). Five 21st Century CLCs and four child care centers provide transportation home from the program in the evening. All other transportation benefits are offered only by child care centers. With regard to specific activities offered to students, nearly all programs (93%) provide academic support, tutoring, and/or homework help (Table 5). As for other activities, there was some variance across program types. For example, while all 21st Century CLCs, child care centers, and public school-based programs offer physical activities, less than half of private schoolbased programs do. Private school-based programs also were much less likely to offer arts programming or field trips. Across all program types, few offer STEM activities (science, technology, engineering, and math) or college/ career preparation. None of the programs responding to the survey indicated a specialization in a particular activity to the exclusion of all other activities. Program quality Program quality is made up of process quality, which cannot be measured directly without observing the interactions of program staff and students; and structural quality, which is revealed by program characteristics that research has shown indicate the capacity for process quality. Structural quality indicators measure whether a program has the types of policies and organizational structures in place to help deliver activities and experiences to benefit child development and student outcomes. YoungStar is intended to measure both structural quality and process quality for most programs. Accreditation is one structural quality indicator recognized in the YoungStar rating system. Twenty-five survey respondents reported they are accredited or working on accreditation, while 14 reported they are not accredited and 19 were unsure of their accreditation status. Of those accredited or working toward accreditation, 17 reported relationships with national accrediting bodies of the type recognized by YoungStar. The others cited accreditation by agencies that do not 7 actually accredit child care or afterschool programs. Some of these agencies, such as Wisconsin Religious and Independent Schools Accreditation (WRISA), accredit schools, which may indicate that a program is located in or operated by an accredited school, but which also indicates some confusion about the type of accreditation recognized by YoungStar. Other quality indicators recognized in YoungStar include tracking and monitoring data on student outcomes and creating and keeping portfolios of student work done as part of the program. Chart 9 shows that over half of programs surveyed reported tracking and monitoring student outcomes for at least some students, while less than a third said they create student portfolios. Another 13% indicated they are planning to create student portfolios in the future, however. YoungStar also awards points to programs conducting self-assessments, which 65% of respondents reported doing. When asked how they perform the assessments, most (54%) reported utilizing a commercial or other assessment tool. The other programs reported conducting staff and/or parent surveys or other means of assessing performance. Programs most likely to conduct assessments using an assessment tool recommended by YoungStar are 21st Century CLCs and group child care centers. Chart 9: Programs monitoring student outcomes and creating student portfolios Yes Yes, for some Student outcomes Portfolios Plan to in the future 46% 7% 20% No 17% 13% 33% 57% 4% 4% Over half (55%) of respondents reported their staff are familiar with The Registry, which is the agency that tracks child care and afterschool staff credentials in Wisconsin. The state now requires program staff to register their credentials with The Registry as part of the YoungStar rating system. Programs not familiar with The Registry tended to be school-based programs in private schools. Only three programs reported being unaware of YoungStar one month prior to its implementation in school-age programs: one school-based program in a private religious school, one in a public school, and one stand-alone program run by a religious organization. This high level of awareness may be related to so many programs having structural quality elements in place. Chart 9: Number of programs having relationship with day school(s) No regular contact with day school(s) 8 Yes, co-host events for families or staff 21 Yes, employ school staff as program staff 23 Yes, attend school events or meetings 23 Yes, exchange data with school(s) regarding individual students Yes, meet with school teachers to complement learning in school day Trans portat Not sure 30 25 Number of programs Trans 8 YoungStar cannot capture all indicators of quality, however, and it does not evaluate the nature of a program’s relationship to the school(s) its students attend. Research has shown this relationship can play a large role in the program’s success in improving student academic outcomes and persistence in school. When asked about their relationship with students’ day schools, only eight programs reported they have no regular contact with the school(s); however, 22 programs did not answer the question, which may indicate that many more do not have relationships with the day schools (Chart 9). Seven of the eight programs reporting no contact were group child care centers, which may serve children from dozens of schools. The most common type of relationship reported was exchanging student-level data with the school, which was reported by 21st Century CLCs and school-based programs. These are also the two types of programs most likely to have reported meeting regularly with teachers and employing school staff as program staff. These programs may have an easier time establishing these relationships because they are located in the school building. Chart 10: Total staff in sample, by program type 250 Full-time 200 Part-time 150 100 50 0 21st Century Community Learning Center Group child care center School-based program Stand-alone program programs that lacked many of these structural quality characteristics tended to be stand-alone or located at private schools. Program staff Considering all structural quality indicators investigated, slightly more than half of the survey respondents appeared ready to participate in YoungStar at the time of the survey. The Closely related to program quality are program staffing issues. Afterschool programs utilize more part-time staff than full-time, due to the short time children are present. Chart 10 illustrates this, with only group child care centers having significant numbers of full-time staff, as they also provide a full day of care for younger children. On average, the programs that are not child care centers employed two full-time staff persons per site and four part-time staff members. Table 6: Thinking of past staff members who have left your school-age program, what have been their reasons for leaving? Percent agreeing Wages too low Insufficient hours Changing careers Going back to school Not a long-term career choice Lack of health benefits Lack of transportation Stress of job Left to work for other school age program 21st Century Community Learning Center Schoolbased program 40% 48% 52% 44% 44% 20% 8% 20% 12% 24% 19% 33% 24% 29% 14% 0% 10% 10% Group Stand-alone child care school-age center program 69% 15% 54% 31% 38% 38% 8% 31% 15% 40% 20% 40% 40% 20% 20% 0% 20% 20% 9 Table 7: Would staff benefit from additional training? Percent agreeing 21st Century Community Learning Center Schoolbased program Group child care center Stand-alone school-age program 48% 52% 48% 44% 56% 48% 44% 60% 36% 29% 5% 19% 19% 29% 29% 33% 48% 19% 31% 69% 38% 31% 54% 54% 69% 69% 62% 40% 20% 20% 40% 40% 20% 20% 20% 20% 36% 14% 46% 40% 52% 29% 69% 40% Health and safety practices Education and care of children with special needs Working with families Math skills Literacy skills (reading, writing) Arts skills (arts and crafts, music, theater, dance) Helping children get along with others Behavior management/discipline Measuring students' academic progress Measuring students' social/developmental progress Classroom/group management and organization Because of the part-time nature of most afterschool program jobs, staff turnover is to be expected. While we did not ask about turnover rates, we did ask why staff who have left a program have done so. Table 6 shows that issues related to wages, hours, and the suitability of afterschool programming as a career were common reasons for staff to leave. When asked to name the type of training that would be most beneficial to their staff, of the 27 open-ended responses, over half (55%) referred to managing children’s behavior, managing the group/classroom, or helping children get along with others. Just five programs mentioned academic-type skills such as measuring students’ progress. Staff turnover can negatively affect quality if it hinders the development of stable relationships between staff and students, and also if new staff members are not well-prepared. Therefore, we asked about the need for more staff training in specific skills. Whether or not staff would benefit from additional training in certain areas was seen differently across program types. Interestingly, the programs most likely to report having many structural quality indicators in place were also most likely to agree that their staff could benefit from more training. Table 7 shows that in several instances, more than half of the 21st Century CLCs and group child care centers see a need for more training, but in no instances do more than half of school-based or stand-alone programs see such a need. Further differences among program types are seen when providers are asked about training opportunities, as shown in Charts 11-13. Group child care centers were most likely to assist staff in obtaining training, either by paying for training costs, paying for time spent in training, or offering training in house. School-based programs were least likely to offer internal training, while 21st Century CLCs were most likely to do so. CLCs also were more likely to pay staff for their time spent in training. Looking at these three charts as a whole, however, it is clear that with the exception of 21st Century CLCs and group child care centers, the majority of programs do not assist their staff to receive more training either by helping to defray the costs or offering internal training programs. 10 Chart 11: Does your program pay for training expenses...for staff to attend training opportunities offered by outside agencies? Paid in part Paid in full Stand-alone school-age program 20% 20% 31% Group child care center 38% 5% School-based program 29% 21st Century Community Learning Center 8% 24% Chart 12: Does your program pay for staff time to attend training opportunities offered by outside agencies or educational institutions? At part wages At full wages Stand-alone school-age program 20% 20% 8% Group child care center 5% School-based program 21st Century Community Learning Center 46% 24% 0% 40% Chart 13: Does your program offer internal training opportunities? Off site On site Stand-alone school-age program 40% 40% Group child care center 38% School-based program 21st Century Community Learning Center 54% 19% 19% 56% 64% Program revenues It can be expensive for afterschool programs to provide high-quality programming and a welltrained workforce. In Milwaukee, there are several fee models used by afterschool programs to generate operating revenue. Group child care centers charge parents a fee, which can be subsidized by the state for low-income families. School-based programs also charge a fee, which may or may not be eligible for a subsidy, depending on the program. The same is true for stand-alone afterschool programs. There is a different model for 21st Century Community Learning Centers, which receive federal funds via the school district. These programs receive up to $90,000 per site, which allow them to provide services free of charge in many cases. (If a CLC student would otherwise be eligible for a state child care subsidy, however, then the CLC may choose to work with the family to access those state funds.) This funding difference is why CLCs were more likely to report waiting lists—if they do not charge a fee to participate, they are limited in the number of children they can serve. When we asked survey respondents whether they charge a fee, all group child care centers and school-based programs that responded indicated that they do so. Half of the 21st Century CLCs that responded indicated they charge a fee. The single stand-alone program to answer the question also charges a fee. Moreover, survey respondents estimated that 43% of all student fees are paid entirely by the students’ families, while another 35% are paid by state subsidy and 10% by other aid, grants, or scholarships. The relative mix of revenues varies by program type, as shown in Table 8. The fact that school-based programs and stand-alone programs rely on parent-paid fees may help explain their relative lack of structural quality indicators—cost pressures can limit quality. This dynamic is further discussed in the next section. 11 Table 8: Please estimate the percentage of school-aged students (grades K5 - 12) enrolled in your before- or after-school program at this site whose fees are fully paid for: Average percent Private pay State subsidy Other aid/scholarship 31% 17% 70% 100% 19% 61% 10% 0% 14% 4% 0% 0% 21st Century Community Learning Center Group child care center School-based program Stand-alone school-age program Barriers to quality improvements The picture painted by the survey results is one of a mixed level of quality among afterschool providers across the city. Certain types of providers, such as 21st Century CLCs, seem more ready to make the quality improvements policymakers are seeking from YoungStar. Yet, even among those providers, up to half reported not having established many structural quality characteristics. While the participation of afterschool programs in YoungStar will help parents, schools, and the community at large better understand where the strengths and weaknesses of those programs, participating in YoungStar does not guarantee that a program will be able to move up the rating scale over time. In order to help state administrators and policymakers understand some of the barriers to making quality improvements, we asked programs about their challenges. Tables 9 and 10 show that many programs experience challenges in making program improvement and obtaining training for staff. Parental involvement was cited as a challenge for over half of 21st Century CLCs and group child care centers, and nearly half of schoolbased programs. This is a concern, because parental involvement commonly is seen as necessary to ensure students have regular attendance and reinforce skills and other learning at home. The next most-frequently cited challenges were collecting fees, which directly impacts quality if revenues are negatively impacted, and keeping costs down, with reflects the need for programs to balance investment in quality with parental affordability or, in the case of many 21st Century CLCs, to work within a fixed budget. Caring for students with special needs also was seen as a challenge by most programs. Further research is needed to better understand the extent to which students with special needs are being well-served in afterschool programs. On a positive note, most programs felt they are able to provide a variety of engaging activities to students and are meeting the expectations of their school/community partners or of their agency/school district. With regard to staff training, affordability was reported to be a challenge for most programs, as was a lack of relevant training opportunities. It appears many programs also are challenged by the fact that staff may not be interested in obtaining more training (Table 10). Conclusion This survey was designed to better understand the quality status of afterschool programs in Milwaukee, as well as to paint a picture of the types of programs available to students across the city. We find a disparate system of programs in many different settings, with about half of all programs lacking certain quality indicators. As the implementation of YoungStar moves forward, we will be monitoring the quality ratings, both to measure quality improvements over time, and to gauge the impact of this increased focus on quality on the range of programs available in Milwaukee. 12 Table 9: Is your school age program currently experiencing challenges in any of the following areas? 21st Century Community Learning Center Schoolbased program Group child care center Providing a variety of engaging activities 44% 24% 38% Student discipline/behavior Caring for students with special needs Parental involvement Facilities Finding/retaining qualified program staff 56% 56% 60% 36% 48% 29% 38% 43% 29% 43% 46% 62% 69% 31% 54% Keeping costs down 56% 29% 62% Collecting fees/generating other financial support Equipment Transportation Technical assistance in meeting regulations or making improvements Meeting expectations of program partners 60% 60% 44% 38% 38% 10% 62% 54% 31% 52% 20% 43% 10% 54% 23% Meeting expectations of central agency administration 36% 24% 23% Percent responding “minor” or “major” challenge Table 10: Are any of the following items challenges for your school age program staff when trying to obtain training? Percent responding “minor” or “major” challenge Cannot afford training fees Staff not paid for time spent in training Staff not interested in training beyond the required hours Lack of relevant training opportunities Training is too elementary Lack of funding for substitutes to replace those attending training Internet access for online courses or seminars 21st Century Community Learning Center 52% 16% 48% 56% 20% Schoolbased program 29% 14% 38% 29% 14% Group child care center 69% 54% 54% 69% 38% 52% 40% 24% 19% 62% 54% 13 Map: Geographic distribution of survey respondents