Research and Evaluation Summary Report

Report Title | page 1 WestEd Year 3 Ready To Learn
Research and Evaluation
Summary Report
A Report to the CPB-PBS Ready To Learn Initiative
Betsy McCarthy, Ph.D.
September 2013
WestEd Year 3 Ready To Learn Research and Evaluation Summary Report | page 1 Overview of Year 3 Research
This report describes WestEd’s formative evaluation activities for the CPB-­‐PBS Ready To Learn Initiative from October 2012 to September 2013. Funded by the U.S. Department of Education, the CPB-­‐PBS Ready To Learn Initiative supports the development of educational television and digital media targeted at preschool and early elementary school children and their families. The goal of the CPB-­‐PBS Ready To Learn Initiative is to promote early learning and school readiness, with a particular interest in reaching children from low-­‐
income communities. In Year 3, WestEd’s formative evaluation efforts supported Ready To Learn joint awardees, the Corporation for Public Broadcasting (CPB) and the Public Broadcasting Service (PBS), as they created and delivered a next-­‐generation educational ecosystem of integrated transmedia content in literacy and numeracy that is aligned with rigorous academic frameworks and research. During Year 3 of the Ready To Learn Initiative, WestEd conducted a series of research studies, which included: •
Review of Boston University’s Teaching of Reading – Technology
Integration Course Syllabus
WestEd conducted a focus group to gather feedback from advisors and experts around the content and structure of Boston University’s course, Teaching of Reading – Technology Integration, that includes Ready To Learn literacy content, as described in the current course syllabus. The focus group feedback focused on participants’ comments and recommendations regarding the overall structure of the course, and the content of the four modules. The focus group asked participants to share their views about whether the course would fulfill the goal of incorporating technology into a reading methods course to develop pre-­‐service teachers’ awareness of, beliefs about, and ability to select and integrate technology into their future practice. •
Formative Evaluation of PBS KIDS Transmedia Suites - Alpha Testing
WestEd conducted a formative evaluation during the spring of 2013 to pilot test 11 newly created PBS KIDS transmedia games from the Peg + Cat suite. WestEd tested alpha versions of individual PBS KIDS games with preschool teachers and their students to provide feedback on the games’ academic content and pedagogy, as well as their perspectives on the utility of the suites and how the suites can be used in the classroom environment. WestEd’s cognitive reviewer also conducted a cognitive task analysis to examine how the games’ intended learning goals aligned with the academic standards for that grade level. •
Ruff Ruffman’s Monumental Mini Golf 3D Immersive World Study
WestEd conducted a study to examine student collaboration and learning in students who play PBS KIDS’ Ruff Ruffman’s Monumental Mini-­‐Golf, a collaborative 3D digital game which is publicly available on the PBS KIDS website. The game is based on the popular PBS KIDS television show, st Fetch! with Ruff Ruffman, and is designed to support 21 Century Skills and mathematics learning in the areas of measurement, geometry, and multiplication. The goals of the research study were to WestEd Year 3 Ready To Learn Research and Evaluation Summary Report | page 2 determine: 1) the educational affordances of the game, including indicators of learning around st mathematics, and 2) Whether playing the game promotes 21 Century Skills, including effective collaboration and communication among students during gameplay. The study also documented differences in student collaboration and learning in different gameplay conditions. •
PBS KIDS Progress Tracker Parent Study
WestEd conducted a formative evaluation of how parents use the PBS KIDS Progress Tracker website, a web-­‐based reporting application that provides feedback on children’s progress on PBS KIDS games and skills to parents. When it is completed, the Progress Tracker will be linked to games that are featured on the PBS KIDS website and will be able to track the progress of children that are logged on to the site. The goal of the Progress Tracker Parent Study was to provide the developer feedback around usability, user experience, ease of use of log-­‐in procedures, overall impressions of the Progress Tracker, and parent understanding of content and involvement in their child’s gameplay and mathematics learning. •
PBS KIDS Progress Tracker Algorithm Validation Study
WestEd conducted a formative evaluation of the PBS KIDS Progress Tracker, a system that utilizes big data technology to collect and analyze data from children’s gameplay of PBS KIDS games. The goal of the research study was to measure the accuracy of the Progress Tracker’s algorithms’ output in relation to student proficiency in select mathematics skills. Specifically, the research focused on the accuracy of the Progress Tracker’s algorithms in determining students’ actual mathematics proficiency in the areas of counting and cardinality. •
PBS KIDS Mathematics Transmedia Suites in Preschool Families and
Communities Study
WestEd conducted a study of PBS KIDS mathematics transmedia suites in preschool homes. The focus of the study was on the efficacy of four PBS KIDS transmedia suites (The Cat in the Hat Knows A Lot About That, Curious George, Dinosaur Train, and Sid the Science Kid) and their accompanying parent support materials in increasing preschoolers’ mathematics skills and enhancing their parents’ ability to support their children’s mathematics learning in the home environment. WestEd Year 3 Ready To Learn Research and Evaluation Summary Report | page 3 Summary of Year 3 Studies
This section details the activities and key lessons learned from the six major research studies from Year 3 of the CPB-­‐PBS Ready To Learn initiative. Review of Boston University’s Teaching of Reading – Technology Integration
Course Syllabus
The purpose of the study was to gather feedback from advisors and experts around the content and structure of Boston University’s course, Teaching of Reading – Technology Integration, which includes Ready To Learn content, as described in the current course syllabus. A focus group of advisors and experts provided data on the overall structure of the course, and the content of the four course modules. The focus group asked participants to share their views about whether the course would fulfill the goal of incorporating technology into a reading methods course to develop pre-­‐service teachers’ awareness of, beliefs about, and ability to select and integrate technology into their future practice. Measures and Data Collection Data collection measures included: 1) a telephone focus group with three advisory board members, and 2) an expert review of the course syllabus by a WestEd pre-­‐service literacy specialist. The telephone focus group was recorded and transcribed. All qualitative data (focus group transcript and expert review) were coded using qualitative data analysis techniques. Key Findings •
Overall, the course content and structure are strong. •
Several participants w orried the course m ay be overly ambitious for busy students. Suggestions include streamlining the course’s required activities, or to have students become an expert in a certain technology or d omain and share their expertise w ith their classmates, rather than having each student m aster the u se of all the technologies and d omains. •
The technology component of the course n eeds to be better d efined in terms of objectives and goals. Suggestions include implementing and/or videotaping a technology-­‐based lesson in the classroom as one w ay to assess the pre-­‐service teachers’ understanding of technology use and its integration in a classroom environment. •
The course does not assess pre-­‐service teachers’ technological competencies. •
All participants suggested adding content around teaching reading w ith English language learners. •
Participants expressed a n eed to integrate w ays to create sustainable technology u se after the course. WestEd Year 3 Ready To Learn Research and Evaluation Summary Report | page 4 Formative Evaluation of PBS KIDS Transmedia Suites - Alpha Testing
WestEd conducted a formative evaluation to pilot test 11 newly created PBS KIDS transmedia games from the Peg + Cat suite. The Peg + Cat transmedia suite is designed to improve core numeracy skills for preschool students through activities that feature a connected narrative across multiple media platforms, such as TVs and computers. WestEd tested alpha versions of individual PBS KIDS games with preschool teachers and their students to provide feedback on the games’ academic content and pedagogy, as well as their perspectives on the utility of the suites and how the suites can be used in the classroom environment. Three preschool teachers were recruited from underserved communities in the San Francisco Bay Area to participate in the formative evaluation. Measures and Data Collection Data collection measures included written teacher feedback and a cognitive task analysis. The written teacher feedback included reviews by experienced preschool teachers who played the alpha versions of the games and reviewed the games for educational content and utility in the classroom. For the cognitive task analysis, WestEd’s cognitive reviewer played the games and examined how the games’ intended learning goals aligned with the academic standards for that grade level. The cognitive psychologist also suggested ways to improve overall gameplay and increase the potential for student learning. Key Findings •
The m ajority of the alpha versions of the games w ere considered to be useful instructional tools, w ith teachers reporting they w ould u se the games to review and practice early mathematics skills in the classroom. •
Most alpha versions of the games w ere aligned to the teachers’ preschool m athematics curriculum. •
The tone and context of all the alpha versions of the games w ere appropriate for the preschool age group. •
Some of the alpha versions of the games had usability issues and raised pedagogical concerns. Teachers suggested that the game provide feedback on w hy a student’s response m ight be wrong and to provide scaffolding on how to find the correct answer. Ruff Ruffman’s Monumental Mini Golf 3D Immersive World Study
This study examined student collaboration and learning in students who play PBS KIDS’ Ruff Ruffman’s Monumental Mini-­‐Golf, a collaborative 3D digital game which is publicly available on the PBS KIDS website. The goals of the research study were to determine: 1) the educational affordances of the game, including indicators of learning around mathematics, and 2) whether playing the game promotes 21st Century Skills, including effective collaboration and communication among students during gameplay. Ruff Ruffman’s Monumental Mini-­‐Golf is a 3D online game designed for play by students six through eight years old. The game involves moving a user-­‐created avatar through 3D problem-­‐solving spaces and working with an online partner to solve math challenges, such as measuring 3D monuments and collaboratively WestEd Year 3 Ready To Learn Research and Evaluation Summary Report | page 5 building with 3D shapes. Study Design and Methodology To address the research questions, researchers designed a two-­‐phased, mixed-­‐methods study. Phase I of the study focused on identifying the educational affordances and implementation feasibility of the 3D game. This phase involved recruiting 4 third grade classroom teachers from WestEd’s database to review the 3D game. Researchers collected data around the mathematical content of the 3D game, the alignment of the content to the regular classroom curriculum, suggestions for improvement, and utility of the 3D game in the classroom. Phase II of the study employed collaborative play sessions with pairs of students playing the Duo mode of the game in one of three randomly-­‐assigned gameplay conditions: Side by Side (SXS), No Voice (NV) and Voice Only (VO). WestEd recruited 14 third grade students, who ranged in age from eight to ten years old, to participate in the collaborative play sessions. Each session lasted between 90 and 120 minutes, with 60 minutes reserved for gameplay and the remaining time allotted for student assessment and interviews. Key Findings Educational Affordances of the 3D Game •
All teachers said that, w ith m inimal changes to the game, they w ould consider using the game with their students to support teaching and learning concepts around m easurement, geometry, and m ultiplication. •
Teacher reviewers in Phase I w ere very interested in the collaborative aspect of the game and noted that m any of the 3D tasks had high learning value. •
Overall, teachers felt the game’s instructions w ere easy for students to follow, and that each mission’s task w as clear. Gains in Student Knowledge Around Target Mathematics Skills •
Assessment data from Phase II showed a slight improvement in student performance from pre-­‐
test to post-­‐ test, w hich suggests that students gained knowledge around target skills after playing the 3D game. Utility of the 3D Game in Different Learning Environments •
Some teachers said they w ould u se the 3D game in conjunction w ith their regular classroom curriculum and activities, w hile other teachers felt the game m ight be better suited for out-­‐of-­‐
school time review or enrichment. •
Teachers expressed some concern over the amount of playtime required for students to reach the educational content in the m issions, and thought this m ight d etract from the u tility of the game in the classroom. WestEd Year 3 Ready To Learn Research and Evaluation Summary Report | page 6 Student Collaborative Play in the 3D Game •
On average, students playing the game collaborated (either via voice, action, or avatar action) while actively solving m athematics tasks in the game’s m issions over 65% of the time. •
Student avatars m ade m eaningful collaborative progress as pairs throughout the game 48% of the time in the side-­‐ by-­‐side (SXS) condition and 53% of the time in the voice only (VO) condition, but only 34% of the time in the n o voice (NV) condition. •
In the N V condition, student avatars w ere idle or performing independent actions that w ere not considered collaborative 49% of time. •
In sharp contrast, in the SXS and V O conditions, student avatars performed non-­‐collaborative, independent actions or w ere idle 26% and 25% of the time, respectively. Conclusions and Recommendations •
Students could benefit from m ore challenging m issions and m ore difficult m ath content w ithin the 3D game. •
Students could benefit from having additional m ethods of communication available to them during gameplay. PBS KIDS Progress Tracker Parent Study
The purpose of this study was to formatively evaluate the PBS KIDS Progress Tracker website, a web-­‐based reporting application that provides feedback on children’s progress on PBS KIDS games and skills to parents. The Progress Tracker is being developed as part of the CPB-­‐PBS Ready To Learn initiative. When it is completed, the Progress Tracker will be linked to games that are featured on the PBS KIDS website and will be able to track the progress of children who are logged on to the site. The goal of the Progress Tracker Parent Study is to provide the developer feedback around usability, user experience, ease of use of log-­‐in procedures, overall impressions of the Progress Tracker, and parent understanding of content and involvement in their child’s gameplay and mathematics learning. WestEd recruited parents from local preschools and elementary schools to participate in the study. Ten parents participated in “single-­‐use” sessions with the Progress Tracker website. In these one-­‐hour sessions, parents were introduced to the features of the website by researchers and were asked about what they thought of the website. In addition, two parents participated in “multi-­‐use” sessions with the Progress Tracker website. In these sessions, parents were also initially introduced to the features of the website. Then, they were encouraged to use the website at home with their children in a naturalistic manner. After using the website over a period of 1-­‐2 weeks, parents were interviewed about their experience using the website over time. Measures and Data Collection During each single-­‐use session, parents were guided to try out numerous features of the Progress Tracker website and to describe their experience of doing the following tasks: understanding the Progress Tracker’s WestEd Year 3 Ready To Learn Research and Evaluation Summary Report | page 7 home page, parent account creation, child account creation, selecting and playing games, creating playlists, the dashboard, the math report, and the progress trends report. The two parents who participated in multi-­‐
use sessions learned the aforementioned features of the Progress Tracker website with a researcher prior to using it with their children at home. Key Findings •
A m ajority of parents noted that either they or their children did have prior experience w ith the general PBS K IDS w ebsite. •
All parents generally expressed a clear understanding of the intent and purpose of the PBS KIDS Progress T racker w ebsite. •
Overall, parents expressed a clear understanding of w hy they needed an account for this website. H owever, w hen trying to create a n ew parent account on the PBS K IDS Progress Tracker w ebsite, parents consistently expressed confusion w ith particular elements of the account creation process. •
Parents generally expressed a clear u nderstanding of w hy the child account w as linked to their parent account and could d escribe the child account in terms of w hat it offered to kids and parents respectively. O verall, almost all parents considered it to b e easy to set u p a child account, though some confusion arose over elements of the kid account creation process. •
After using the site over time, both parents that participated in m ulti-­‐use sessions raised issues relating to the actual login process. •
Some parents particularly appreciated the d etail provided on individual games, such as game content, age level, and skill covered. •
Most parents w ere familiar w ith the term “playlist,” and half the parents generally interpreted the playlist as their kid’s preferred or favorite games, or games that they could select for their child. Some confusion d id arise w ith the process of setting u p a n ew playlist, although locating the “Games” tab helped clarify the process of adding games to the playlist for parents. •
Almost all parents w ere able to navigate back to the dashboard from the “My A ccount” page. Several parents had positive feedback related to tracking their student’s play time through the “Dashboard,” and viewing the star rating. H owever, parents had some m isconceptions around the “total play time” d isplayed, the “skills practiced” icon, the “trends” icon, and the grade level filter. •
Several parents expressed a preference for the style and type of information provided in the math report w hen compared to the dashboard. T he m ajority of parents expressed that the math skills list and the recommended games list w ere u seful. H owever, some confusion arose for both single-­‐use and m ulti-­‐use session parents around the m eaning of the ratings and labels included in the m ath report. •
Parents appeared to have a clear u nderstanding of the information provided u nder the “Progress T rends” tab. Some parents found the information presented to be essentially similar to w hat they had already seen in the “Dashboard” and in the “Math R eport,” w hile others w ere very impressed w ith the n ew visualizations of information in this tab. •
Following the sessions, m ost parents expressed that they w ould u se the PBS K IDS Progress Tracker w ebsite in their daily lives, at different levels of frequency. •
All parents w ho w ere probed as to w hether they trusted PBS to provide this type of progress information on their children expressed confidence and trust in PBS as an organization. WestEd Year 3 Ready To Learn Research and Evaluation Summary Report | page 8 PBS KIDS Progress Tracker Algorithm Validation Study
The PBS KIDS Progress Tracker is a system that utilizes big data technology to collect and analyze data from children’s gameplay of PBS KIDS games. Users will be able to log in to the PBS KIDS website and have the Progress Tracker track and analyze their gameplay data and report on potential learning and progress around math and literacy standards. The primary goal of the research study was to measure the accuracy of the PT algorithms’ output in relation to student proficiency in select mathematics skills. Specifically, the research focuses on the accuracy of the PT algorithms in determining students’ actual mathematics proficiency in the areas of counting and cardinality. The study was guided by the following research question: Does the Progress Tracker algorithms’ output accurately reflect students’ progress in the mathematics skill of counting & cardinality? A secondary goal of the research study was to explore the effectiveness of different methods of validating algorithms for games that will be linked to the PT. Measures and Data Collection The study used a mixed-­‐methods design. Data were collected in three phases: (1) a series of cognitive labs, (2) the collection of large data sets of game metrics, and (3) detailed observations of student gameplay at school sites. In Phase 1, WestEd recruited two preschool students and their parents to participate in cognitive labs. In Phase 2, a large set of anonymous student usage data was collected through the PT and the PBS KIDS website. For Phase 2, WestEd partnered with a public media station from a large metropolitan area to recruit 15 preschool and Kindergarten teachers and 450 students to participate in the study. In Phase 3, researchers worked with preschool students and teachers from a local preschool. In total, WestEd recruited three classroom teachers and 80 students to participate in the study. Data from all three phases were combined and analyzed to generate global findings around the accuracy of the PT algorithms’ output in relation to student progress in select mathematics skills. Key Findings •
Analyses of the data suggest there are differences in students’ patterns of play between the two types of games included in the study (solution-­‐based and exploration-­‐based games). Specifically, the data suggest that the exploration-­‐based games (which introduce counting and cardinality concepts, but do not present solution-­‐based mathematics tasks) were fairly easy for most children in the study to play through. Conversely, the data suggest that the four solution-­‐based games may have presented students with mathematics tasks that were more challenging for them. •
The descriptive results suggest that students were more likely to explore the full mathematical content of exploration-­‐based games than of solution-­‐based games. The results also suggest that exploration-­‐based games present less challenging mathematics tasks to students than the solution-­‐
based games. •
On average, researchers were more likely to observe students express some degree of disinterest while playing exploration-­‐based games compared to solution-­‐based games. •
Results indicate that the average raw proficiency (r=o.54, p<0.01) and weighted proficiency (r=o.52, p<0.01) of the eight games, when aggregated, are significantly correlated with students’ mathematics skills. WestEd Year 3 Ready To Learn Research and Evaluation Summary Report | page 9 •
When disaggregated by game type, the average raw proficiency (r=o.633, p<0.001) and weighted proficiency (r=o.631, p<0.001) for the four solution-­‐based games are also significantly correlated with students’ mathematics skills. In contrast, the PT algorithms’ output for the four exploration-­‐
based games is not significantly correlated with students’ mathematics skills. •
When considering individual games, the PT algorithms’ output for the solution-­‐based games “Blast Off” and “Meatball Launcher” shows the highest correlations with students’ mathematics skills. PBS KIDS Mathematics Transmedia Suites in Preschool Families and Communities
Study
WestEd conducted a study on the efficacy of four Ready To Learn PBS KIDS transmedia suites, The Cat in the Hat Knows A Lot About That, Curious George, Dinosaur Train and Sid the Science Kid, in increasing preschoolers’ mathematics skills and enhancing parents’ ability to support their children’s mathematics learning in the home environment. The suites focus on improving core literacy and numeracy skills for children through the well-­‐planned and coordinated use of multiple media platforms. The purpose of the study was to determine: 1) Does children’s use of the PBS KIDS transmedia suites with their parents increase children’s knowledge and skills in mathematics? and 2) Does parents’ awareness and support of their children’s mathematics learning at home increase after interacting with their children around the suites and using support materials related to the suites? An additional goal of the study was to explore the feasibility of having teachers from a Head Start community facilitate parent meetings related to the intervention. The study explored the needs of trainees and corresponding strategies and supports to meet those needs. Study Design and Methodology The study used a quasi-­‐experimental, non-­‐equivalent groups design, which randomly assigned 60 families from each of two Head Start centers to either an intervention or comparison group. The intervention was based on four PBS KIDS transmedia suites and four mathematics concepts included in those suites: numbers and operations in base ten, measurement and data, sorting and patterns, and geometry and spatial sense. The ten-­‐week program prompted intervention group parents and children to work together on PBS KIDS transmedia activities for 3o minutes per day for four days per week, and encouraged parents to attend weekly parent meetings at their child’s preschool. Participating children’s average age is 3.8 years old. All children qualified for a Free and Reduced Lunch program. To increase the community’s capacity to support children’s learning through transmedia, the pilot training sessions for facilitator trainees was designed to meet trainees’ needs. Each week, trainees participated in a two-­‐hour pre-­‐parent meeting training which focused on necessary technologies, underlying mathematics content of the transmedia games, and parent meeting logistics. Trainees also had post-­‐parent meeting reflections on their experience, strengths of the meeting’s facilitation, and areas and strategies for improvement. Depending on trainees’ readiness, trainees co-­‐facilitated and/or independently facilitated parent meetings. WestEd Year 3 Ready To Learn Research and Evaluation Summary Report | page 10 Key Findings Children’s Knowledge and Skills in Mathematics •
The intervention w as positively associated w ith gains in children’s knowledge and skills in mathematics, as m easured by the C hild M ath A ssessment (CMA) . •
Adjusted m ean differences on the post-­‐test m easure of the C MA show that the intervention group exceeded the comparison group (point estimate of .028), after accounting for d ifferences in baseline test results, parents’ support of learning at home, and participant d emographic characteristics (i.e., race/ethnicity and child age). •
Intervention students in the 3-­‐year old subgroup demonstrated significant gains in mathematics knowledge as m easured by the C MA w hen compared to their peers in the comparison group. Parents’ Awareness and Support of Their Children’s Mathematics Learning •
Treatment parents’ perceived use of resources to support learning at home surpassed comparison parents (point of estimate of 19.45, effect size = .64). •
Parents’ support of their child’s m athematics learning increased d uring the intervention. T hey were able to create a home environment that supports their child’s m athematics learning. Discussion of Pilot Training Sessions In order to explore the feasibility of having teachers from a Head Start community facilitate parent meetings related to the intervention, WestEd recruited two teachers from the Head Start community to 1
participate in pilot training sessions. Including trainees on the research staff was an informative exercise in outlining attainable goals for the trainee, understanding teachers’ needs around facilitating groups of parents in transmedia and mathematics activities, and developing corresponding strategies and supports to meet those needs. The trainee’s successful curriculum presentation in English and Spanish during Week 7 was a significant achievement for the trainee as well as the lead parent facilitator responsible for training her, and it was clear from both individuals that the experience was extremely rewarding and encouraging for the future of the train the trainer model. 1 In week two of the intervention, one of the trainees was hired for a new position at her Head Start site and was not able to continue as a trainee.
WestEd Year 3 Ready To Learn Research and Evaluation Summary Report | page 11 Cross-Findings and Reflection
In Year 3 of the Ready To Learn project, WestEd conducted a number of formative and exploratory studies. As in past years, WestEd reviewed early versions of transmedia suites. WestEd reviewed eight games from the Peg + Cat suite. These reviews found that each transmedia game was well-­‐designed and provided opportunities to learn and practice mathematics concepts. An exploratory study was conducted to explore the promise of a 3D Immersive World mathematics game in increasing students’ mathematics and 21st century skills such as collaboration and communication. Results of the study suggested that the game was highly appealing and allowed students to practice standards-­‐aligned mathematics content and collaborate with avatars and each other to solve math-­‐related problems. WestEd conducted two studies related to the PBS KIDS Progress Tracker. First, WestEd conducted a formative evaluation of how parents use the PBS KIDS Progress Tracker website, and provided the developer feedback around usability, user experience, overall impressions of the Progress Tracker, and parent understanding of content and involvement in their child’s gameplay and mathematics learning. Secondly, WestEd conducted a formative evaluation of the PBS KIDS Progress Tracker system to measure the accuracy of the Progress Tracker’s algorithms’ output in relation to student proficiency in select mathematics skills. These studies found the PBS KIDS Progress Tracker had high appeal to parents and functioned well in its purpose of using data from children’s gameplay of mathematics games to provide performance information to parents. Also, WestEd provided early feedback on plans for a pre-­‐service course that introduces new teachers to the use of technology in their instruction. Finally, in Year 3, WestEd conducted an exploratory study of PBS KIDS transmedia suites in low-­‐income homes and communities. This study found that the intervention, which included the low-­‐
income parents’ use of PBS KIDS transmedia suites, led to growth of parents’ awareness and support of their children’s mathematics learning. In addition, the children’s mathematics learning grew from pre-­‐ to post-­‐
test. These studies, taken as a whole, continue to suggest that Ready To Learn educational materials are highly effective in engaging children and parents, and in promoting learning in children ages three to eight years-­‐old.