Wednesday, July 31, 2013

A Control Group

We began tracking outcomes from the Manaiakalani Programme in 2007 and from time to time get asked, "Do you have a control group of students?"

The answer has always been an unequivocal "No" for a variety of reasons, including a philosophical disagreement with the thought of select young people having an innovative, engaging learning experience while others in a class next door look on.

It came as a surprise when reading through our research report to realise that - inadvertently - for the first time in 2013 we have a control group of students in one of our cohorts.  It was the transition from Year 8 in our primary schools to Year 9 at Tamaki College that enabled this. When data was collected in Term One from e-asTTle testing in Reading, Writing and Maths there was a cohort (one third approx) who had participated in the Manaiakalani programme when they attended Manaiakalani primary schools, and a cohort (approx two thirds) who came from a wide variety of primary schools from around Auckland and beyond. This second group mostly came from low decile schools also.

The difference in e-asTTle data was surprising, even to us true believers.  

The full report is available online and pages below are referenced from this document.

This commentary was provided by Prof Stuart McNaughton in the brief summary of the report:

Students entering Year 9 (2013) (into Tamaki College) from the Manaiakalani schools had higher average achievement levels than those from elsewhere, supporting evidence that the primary schools are having some success in achievement levels, compared with students from schools in like circumstances. The two thirds of students who came from outside the cluster were two sub levels below the others in reading (3B). This meant the total cohort entering Year 9 had widely spread achievement levels. 
The patterns for maths and writing were similar but at lower levels.


The Reading image is taken from page 122 of the report.
The cohort on the left are the Year 9 students who were at Manaiakalani primary schools in the previous year.
The cohort on the right came from 'Other' primary schools.

This is the most startling contrast , with the red line indicating the gap between the two groups of students' achievement in Reading at Year 9.


The Maths image is taken from page 158 of the report.

The cohort on the left are the Year 9 students who were at Manaiakalani primary schools in the previous year.
The cohort on the right came from 'Other' primary schools.

The Writing image is taken from page 177 of the report. 
The cohort on the left are the Year 9 students who were at Manaiakalani primary schools in the previous year.
The cohort on the right came from 'Other' primary schools.



The challenges our high school teachers face in effectively teaching this diverse group of students are huge. 

"The pattern (also) indicates a substantial challenge catering for the range and especially the very low group, who have reading skills and knowledge which are not sufficient to engage effectively in reading in subject areas at expected levels without intensive support. The level of the challenge is such that they need to make up four full curriculum levels by Level 2 NCEA for the school to meet the 85% pass rate." p11