Wednesday, July 31, 2013

A Control Group

We began tracking outcomes from the Manaiakalani Programme in 2007 and from time to time get asked, "Do you have a control group of students?"

The answer has always been an unequivocal "No" for a variety of reasons, including a philosophical disagreement with the thought of select young people having an innovative, engaging learning experience while others in a class next door look on.

It came as a surprise when reading through our research report to realise that - inadvertently - for the first time in 2013 we have a control group of students in one of our cohorts.  It was the transition from Year 8 in our primary schools to Year 9 at Tamaki College that enabled this. When data was collected in Term One from e-asTTle testing in Reading, Writing and Maths there was a cohort (one third approx) who had participated in the Manaiakalani programme when they attended Manaiakalani primary schools, and a cohort (approx two thirds) who came from a wide variety of primary schools from around Auckland and beyond. This second group mostly came from low decile schools also.

The difference in e-asTTle data was surprising, even to us true believers.  

The full report is available online and pages below are referenced from this document.

This commentary was provided by Prof Stuart McNaughton in the brief summary of the report:

Students entering Year 9 (2013) (into Tamaki College) from the Manaiakalani schools had higher average achievement levels than those from elsewhere, supporting evidence that the primary schools are having some success in achievement levels, compared with students from schools in like circumstances. The two thirds of students who came from outside the cluster were two sub levels below the others in reading (3B). This meant the total cohort entering Year 9 had widely spread achievement levels. 
The patterns for maths and writing were similar but at lower levels.

The Reading image is taken from page 122 of the report.
The cohort on the left are the Year 9 students who were at Manaiakalani primary schools in the previous year.
The cohort on the right came from 'Other' primary schools.

This is the most startling contrast , with the red line indicating the gap between the two groups of students' achievement in Reading at Year 9.

The Maths image is taken from page 158 of the report.

The cohort on the left are the Year 9 students who were at Manaiakalani primary schools in the previous year.
The cohort on the right came from 'Other' primary schools.

The Writing image is taken from page 177 of the report. 
The cohort on the left are the Year 9 students who were at Manaiakalani primary schools in the previous year.
The cohort on the right came from 'Other' primary schools.

The challenges our high school teachers face in effectively teaching this diverse group of students are huge. 

"The pattern (also) indicates a substantial challenge catering for the range and especially the very low group, who have reading skills and knowledge which are not sufficient to engage effectively in reading in subject areas at expected levels without intensive support. The level of the challenge is such that they need to make up four full curriculum levels by Level 2 NCEA for the school to meet the 85% pass rate." p11

Thursday, July 25, 2013

Research Report 2012 to 2013

At the beginning of the month the Woolf Fisher Research team of Dr Rebecca Jesson, Prof Stuart McNaughton and Aaron Wilson presented the report from their evaluation of the Manaiakalani Programme over the academic year 2012-2013. The full report (170 pages) and the executive summary (17 pages) are publicly available at this link, and the slides from their summary to educators are embedded below.

The report covers the most disruptive year a cluster could hope to have - in the old-fashioned as well as the modern usage of the term disruptive!

The 'housing situation':
The vast majority of students in our Manaiakalani schools live in rental accommodation i.e. government housing. And in 2012 the Tamaki Redevelopment Plan meant some of our families had to leave their homes (and their schools) and many lived in a state of insecurity, dreading when it would be their turn to be moved out. This meant half our schools saw increased churn in their student population. Some schools at one end of the suburb had rolls shrink, some at the other end had rolls increase to accommodate families moving around.

All in:
This was the first year that ALL students from years 5-13 in ALL our schools moved to a digital learning environment using their own netbook as their basic learning tool. The pilot years were officially over. This meant that ALL teachers were required to work in digital learning environments and the days of the research only including enthusiasts, lead teachers, pilot classes (i.e. willing volunteers) were over. It was all in! The number of uncertain or unwilling was low, but we also had a number of relievers, beginning teachers and staff changes during the year across the 8 schools included in the research data.

 A couple of weeks of reflection and discussion with colleagues have gone by and today I met with the research team for a big picture conversation.

The Manaiakalani cluster has some key goals which have not changed since 2006 when the principals' group co-constructed the vision statements. Raising student achievement outcomes is absolutely at the forefront - not just because it is a government priority but simply because our students as individuals deserve this. Early screening of our new entrants shows that across our cluster our 5 year olds start school two years behind the norm academically. For our children to catch up we have to accelerate their academic progress.  This means their shift inside a year has to be more than a year.  
We have become used to our research data from our pilot and lead teacher classes demonstrating an accelerated shift towards the norm, and even overtaking it.

This year, with 'all in', our aggregated data showed our cluster moving at the expected rate for the rest of New Zealand i.e. in one year our kids made one years shift, but as a cohort they didn't accelerate.
So yes, those who are passionately committed to this are disappointed. But given the year we had, we have to take some satisfaction that we didn't fall back into the low-achieving mould of making less than one year's progress.
"In summary, like international studies of large scale 1:1 programmes, evidence for acceleration of student achievement is limited once 1:1 programmes are rolled out in larger sizes. While there is little evidence for acceleration and higher levels over time associated with Manaiakalani schools at an aggregated level (cluster, school) there are pockets of substantial achievement. What evidence we have suggests that Manaiakalani may be associated with important educational outcomes from these pockets; for example in the  ̳pilot‘ group of NCEA Level 1 students or in the classrooms with above nationally expected gains. " p65
Lots to work on there.

The feedback from researchers spending time in the classes observing learning and teaching, and interviewing students and teachers following these observations, was more in line with the other Manaiakalani goals.

Engagement, one of the key elements of Manaiakalani 'the hook from heaven', was clearly evident and commented on
"Overall, the present data confirm predictions that the e-learning classrooms were associated with high levels of behavioural engagement, possibly sufficiently high to overcome what has previous been described as a generalised drop in engagement over the transition to secondary school." p65

"In interviews, teachers indicated that they had goals to develop students‘ independence, most often in terms of accessing their curriculum and engaging independently in learning tasks. Our observations confirmed that in general on task behaviour and 'streamlining' of classes suggests that is largely successful." p74

This is much more complex than a couple of 'sound bites' can represent. All teachers are interested in engagement inside the classroom, but this report also covers 'Active Pursuit', students who are engaged with learning and extending this beyond the classroom.  We are excited to have our hunch confirmed, that students are actively engaged in learning outside of school - somewhat atypical in Decile 1 communities. 
"The interviews with students suggest there is a notable group of students across the schools and at both primary and secondary who are actively pursuing school/academic related activities."p67

Our teachers are clearly implementing the Manaiakalani vision
"…our analyses of the classroom observations show that, in large measure, implementation of Manaiakalani reflects teachers‘ understanding of the goals of the programme." p64

"Teachers articulated high enthusiasm for and fidelity to the goals of Manaiakalani. They articulated digital access to resources and curriculum and engagement as goals. In large measure we saw evidence that teachers were operationalising these goals, implementing digital pedagogies and engaging their students in learning. This provides some indication that given shared vision, teachers in general have the means to implement this in their classes." p73

An innovative intervention covers three phases (R Jesson), and to use NCEA terms our current report card reads....

The Manaiakalani Programme:
  • Implementation  - Achieved with Excellence
  • Changing Teacher practice - Achieved (in some cases with Excellence)
  • Student Achievement outcomes - Achieved (in pockets)

Like 1:1 interventions internationally, Manaiakalani seems to have precipitated a shift in classroom dynamics, allowing for more extended discussion, student control, personalised feedback and instant access and infinite range of learning resources. The challenge now it to understand the conditions under which this leads to improvements, not only in the learning experience, but in outcomes for learners. p73

Tuesday, July 23, 2013

Thinking about SAMR

The SAMR model developed by Dr Ruben Puentedura has been around for some time now and three separate events have led me to take another look at it recently.

First our research report for the Manaiakalani Programme was presented to our schools at the beginning of July for the 2012 academic year. It was the first year all the children in our cluster from years 5-13 had owned their own digital device AND the first year all our teachers had been required to create a digital learning environment.  The overall picture was very positive, but in the 100+ pages of data there were also concerns. In her oral feedback Dr Rebecca Jesson pointed out that 2012 was the first year the research data included all children and all teachers, rather than volunteer and lead teachers as in previous years. It is clear that the PLD team has work to do yet, and my thinking turned to the SAMR model.

Secondly, I caught up with a friend I haven't see for a couple of years and she asked my advise about the 'hard to shift' teachers she works with, those who are not prepared to step onto the bottom rung of the SAMR ladder. A recent experience came to mind of digitising some family papers and I suggested she try using a new feature of Google Docs with them to at least entice them onto the Substitution rung. 
Did you know that if you photograph a page of text (any basic camera or phone camera will do - just get the lighting crisp) and upload the photo to your Google Drive with all the conversion options checked, you will then get a Doc that not only has the photo embedded in it but also has copied all the text and pasted it below in the Document ready to use? 

Delete the photo from the Doc and the text remains in your Document ready to process, and no scanner or complicated technology needed.
I know what you are thinking here!  A filing cabinet of worksheets about to be converted to Docs! It is not my preferred option for teaching either , but in 2013 if a teacher has not yet made an attempt to use technology this could be a hook to get started.

Thirdly, we have been investigating Modern Learning Environments and a month ago went overseas to look at some fabulous schools and classrooms.  Everyone in the group agreed that our Manaiakalani schools (most built in the 1950s) would love to have had any of them and our teachers and young people would thrive implementing our pedagogy.  We were intrigued to see that in some instances the very expensive modern buildings were being used in ways that surely hadn't been intended.  And so the SAMR model popped into my mind again.
We saw a multi million dollar building being used at the Substitution level ie young people seated in rows, facing lecturing teachers, but inside a space designed for personalisation, collaboration etc etc. We also saw Augmentation and Modification but mostly we saw Redefinition and were envious :)

Finally, I came across this image below by Susan Oxnevad and thought it was worth sharing with our team.

Wednesday, July 17, 2013

‘ReTooling School’

‘ReTooling School’: a principal's perspective

The July Nethui, held in Wellington this year, featured an education perspective on ReTooling School as one of the Keynote addresses. 

Although I didn't get to attend or watch the live stream, I was able to keep an eye on the Twitterstream and the collaborative note-taking while attending another event.

It seemed that most of the tweets from non-educators and that participants were appreciating hearing that innovation is occurring in the education sector. Blog commentators also reflected this. 

Russell has been speaking about ReTooling School for some time now and was recently interviewed for an article in the Term 2, 2013 edition of "School News" (p7). The cover featured Modern Learning Environments, so a principal with a predominantly 1950's building complex may appear an unusual contributor to the conversation. However, it is clear that significant change needs to  occur in the hearts and minds of the people in the the school community if modern structures are to have an impact on learning environments. 

ReTooling School requires:

1. A Change Pedagogy Imperative

2. Operationalising of Te Tiriti o Waitangi
3. A new partnership around Schooling
4. Technical Provision
5. Research and Development

The full text of this conversation can be read here