Pilot Study Results

The Research, Evaluation and Technical Assistance (RETA) project, headed by Dr. Genia Toma of the University of Kentucky, has been charged with the goal of assessing whether the AMSP’s professional development activities have contributed to decreasing the achievement gap in central Appalachia relative to non-AMSP counterparts. The RETA project is conducting a retroactive evaluation of the AMSP participant data from 2002 to the present.  Dr. Toma and her team conducted a pilot study of AMSP’s effectiveness using school level data in Kentucky.

The data from the pilot study indicate that AMSP participation has contributed to academic achievement, although the change has only been observed at the middle school and high school levels. This actually meshes well with the expected outcome because the majority of program intervention has been targeted at the middle and high school levels, rather than elementary school.. Perhaps the results are best described by Dr. Toma herself, “We found that after controlling for other factors, if the percentage of teachers participating in AMSP professional development within a school is increased by one percent, math scores of 11th graders rise by .29 points.” 

Dr. Toma’s team is conducting the statistical analysis for the AMSP data with a  quasi-experimental approach - one typically used in economics - so that the research controls for the fact that school districts and even individual teachers within those districts participate in the AMSP on a voluntary basis (nonrandom). Many other factors, such as socioeconomic status, teacher work experience, and hours of teacher professional development participation, are also taken into account in this quantitative study. This will allow for the discovery of a great deal of information as the research progresses.

Current Research Timeline

Data are being  collected on a massive scale down to the most micro-level variables about students, teachers, and the professional development programs themselves. Data are specially coded and names removed to protect the privacy of students and teachers. Eventually the research will show not only whether the AMSP has been effective but also information as complex and detailed as whether specific professional development activities contributed to an increase in achievement. The information will be able to be analyzed in many ways such that the influence of individual teacher’s contribution and student demographics on students’ achievement can be measured.

This method of analysis, once finalized, can serve as a research and analysis model for  MSP programs nationwide and ultimately for any teacher professional development programs.