At the beginning of the month the Woolf Fisher Research team of Dr Rebecca Jesson, Prof Stuart McNaughton and Aaron Wilson presented the report from their evaluation of the Manaiakalani Programme over the academic year 2012-2013. The full report (170 pages) and the executive summary (17 pages) are publicly available at this link, and the slides from their summary to educators are embedded below.
The report covers the most disruptive year a cluster could hope to have - in the old-fashioned as well as the modern usage of the term disruptive!
The 'housing situation':
The vast majority of students in our Manaiakalani schools live in rental accommodation i.e. government housing. And in 2012 the Tamaki Redevelopment Plan meant some of our families had to leave their homes (and their schools) and many lived in a state of insecurity, dreading when it would be their turn to be moved out. This meant half our schools saw increased churn in their student population. Some schools at one end of the suburb had rolls shrink, some at the other end had rolls increase to accommodate families moving around.
All in:
This was the first year that ALL students from years 5-13 in ALL our schools moved to a digital learning environment using their own netbook as their basic learning tool. The pilot years were officially over. This meant that ALL teachers were required to work in digital learning environments and the days of the research only including enthusiasts, lead teachers, pilot classes (i.e. willing volunteers) were over. It was all in! The number of uncertain or unwilling was low, but we also had a number of relievers, beginning teachers and staff changes during the year across the 8 schools included in the research data.
A couple of weeks of reflection and discussion with colleagues have gone by and today I met with the research team for a big picture conversation.
The Manaiakalani cluster has some key goals which have not changed since 2006 when the principals' group co-constructed the vision statements. Raising student achievement outcomes is absolutely at the forefront - not just because it is a government priority but simply because our students as individuals deserve this. Early screening of our new entrants shows that across our cluster our 5 year olds start school two years behind the norm academically. For our children to catch up we have to accelerate their academic progress. This means their shift inside a year has to be more than a year.
We have become used to our research data from our pilot and lead teacher classes demonstrating an accelerated shift towards the norm, and even overtaking it.
This year, with 'all in', our aggregated data showed our cluster moving at the expected rate for the rest of New Zealand i.e. in one year our kids made one years shift, but as a cohort they didn't accelerate.
So yes, those who are passionately committed to this are disappointed. But given the year we had, we have to take some satisfaction that we didn't fall back into the low-achieving mould of making less than one year's progress.
"In summary, like international studies of large scale 1:1 programmes, evidence for acceleration of student achievement is limited once 1:1 programmes are rolled out in larger sizes. While there is little evidence for acceleration and higher levels over time associated with Manaiakalani schools at an aggregated level (cluster, school) there are pockets of substantial achievement. What evidence we have suggests that Manaiakalani may be associated with important educational outcomes from these pockets; for example in the ̳pilot‘ group of NCEA Level 1 students or in the classrooms with above nationally expected gains. " p65
Lots to work on there.
The feedback from researchers spending time in the classes observing learning and teaching, and interviewing students and teachers following these observations, was more in line with the other Manaiakalani goals.
Engagement, one of the key elements of Manaiakalani 'the hook from heaven', was clearly evident and commented on
"Overall, the present data confirm predictions that the e-learning classrooms were associated with high levels of behavioural engagement, possibly sufficiently high to overcome what has previous been described as a generalised drop in engagement over the transition to secondary school." p65
"In interviews, teachers indicated that they had goals to develop students‘ independence, most often in terms of accessing their curriculum and engaging independently in learning tasks. Our observations confirmed that in general on task behaviour and 'streamlining' of classes suggests that is largely successful." p74
This is much more complex than a couple of 'sound bites' can represent. All teachers are interested in engagement inside the classroom, but this report also covers 'Active Pursuit', students who are engaged with learning and extending this beyond the classroom. We are excited to have our hunch confirmed, that students are actively engaged in learning outside of school - somewhat atypical in Decile 1 communities.
"The interviews with students suggest there is a notable group of students across the schools and at both primary and secondary who are actively pursuing school/academic related activities."p67
Our teachers are clearly implementing the Manaiakalani vision
"…our analyses of the classroom observations show that, in large measure, implementation of Manaiakalani reflects teachers‘ understanding of the goals of the programme." p64
"Teachers articulated high enthusiasm for and fidelity to the goals of Manaiakalani. They articulated digital access to resources and curriculum and engagement as goals. In large measure we saw evidence that teachers were operationalising these goals, implementing digital pedagogies and engaging their students in learning. This provides some indication that given shared vision, teachers in general have the means to implement this in their classes." p73
An innovative intervention covers three phases (R Jesson), and to use NCEA terms our current report card reads....
The Manaiakalani Programme:
- Implementation - Achieved with Excellence
- Changing Teacher practice - Achieved (in some cases with Excellence)
- Student Achievement outcomes - Achieved (in pockets)
Like 1:1 interventions internationally, Manaiakalani seems to have precipitated a shift in classroom dynamics, allowing for more extended discussion, student control, personalised feedback and instant access and infinite range of learning resources. The challenge now it to understand the conditions under which this leads to improvements, not only in the learning experience, but in outcomes for learners. p73
