| HOME | INFORMATION | PUBLICATIONS | CONTACT US | ||
![]() |
|
|
|
|
Rationale for the StudyEarly in 2005, the U.S. Department of Education's Institute of Education Sciences (IES) worked with Mathematica Policy Research, Inc., to convene a panel of experts in mathematics, mathematics instruction, and evaluation design to (1) discuss issues that confront a study of the effectiveness of mathematics curricula, and (2) recommend how the study could address those issues. This panel helped shape the study by identifying the early elementary grades as the most important level for the evaluation because, even before they enter elementary school, disadvantaged children fall behind their more advantaged peers in basic competencies such as number line ordering and magnitude comparison. Research also shows that learning these basic skills in early elementary grades is essential to learning fractions in later elementary grades. The expert panel also recommended an evaluation that compares different approaches to teaching early elementary mathematics, and advised that many mathematics curricula have been developed recently that are being widely implemented without evidence of their effectiveness. In addition, several basal math textbooks have been widely used over the years—also without evidence of effectiveness.
Research Questions and Evaluation DesignIn October 2005, IES began working with Mathematica to conduct the Evaluation of Mathematics Curricula. The evaluation seeks to answer three questions:
Experimental methods are being used to answer the research questions listed above. In particular, the evaluation is based on a school-level random assignment design, in which participating elementary schools in each participating district are randomly assigned to the curricula included in the study. Consider, for example, a district that has eight elementary schools participating in the study. The study team randomly selected two schools to implement curriculum A, two schools to implement curriculum B, and so on. In each school, teachers at the target grade levels receive training and both teacher and student materials free of charge for the curriculum assigned to their school. Relative effects of the curricula are estimated by comparing average math achievement of students in the various curriculum groups. The study is designed to detect an effect size as small as 0.20 between any pair of curricula. To detect the study's target effect size, the study recruited 12 districts and 111 schools to participate. Districts and schools were selected for their suitability and willingness to comply with the study design. In general, participating schools serve a large number of low-achieving students or students in poverty. A key requirement for participation is that district and school staff were willing to permit the study determine through random assignment the curriculum their schools will use. Geographic representation and urban or rural location also were factors in identifying districts and schools. Schools in the study used their assigned math curriculum in the first grade during the 2006-2007 school year, the first and second grades during the 2007-2008 school year, and the first, second, and third grades during the 2008-2009 school year.
Timeline and Data Collection PlanThe evaluation began in October 2005 and is a five-year study. The first year involved several activities that set up the study. Among these activities were (1) selecting the curricula, (2) recruiting districts and schools, (3) developing data collection forms, and (4) finalizing the evaluation design. Implementation of the curricula and data collection took place in first grade during the 2006-2007 school year, in first and second grades during the 2007-2008 school year, and the first, second, and third grades during the 2008-2009 school year. A report based on the 2006-2007 school year is available. A report based on both the 2006-2007 and 2007-2008 school years is expected in Fall 2009. A report based on the third year of implementation is expected in September 2010. The study includes several complementary data collection efforts that support answers to the study's research questions.
ConfidentialityAll information collected by the study is strictly confidential and carefully guarded to the full extent allowed by law. The study's results will be reported only for groups of students, such as “70 percent of first graders achieved mastery of rudimentary mathematics.” |