Digital Pedagogy

The search for digital pedagogy that raises achievement

Graphic created by Russell Burt

Dr Rebecca Jesson from the Woolf Fisher Research Centre attended our recent Manaiakalani combined Boards of Trustees forum and presented current findings and recommendations resulting from the research carried out in our schools.

Her half hour presentation was informative, to the point and made very accessible for the audience - most of whom were not educators.

The graphic on the left illustrates one of the outstanding findings shared by the researchers - that learners who spend three consecutive years in the Manaiakalani programme make more progress than the average kiwi kid.

The video below is a highly recommended watch for people wanting a progress update on the 12 Manaiakalani Schools in 2016.  

Another standout quote from the night for this group of parents who may have had questions about their children being in digital learning environments assured them that the face to face conversations teachers have with the young people can be more powerful because of the digital.

"Manaiakalani is one of the most effective interventions across the world for improving face to face conversation". 
Dr Rebecca Jesson, Nov 2016

A Control Group

We began tracking outcomes from the Manaiakalani Programme in 2007 and from time to time get asked, "Do you have a control group of students?"

The answer has always been an unequivocal "No" for a variety of reasons, including a philosophical disagreement with the thought of select young people having an innovative, engaging learning experience while others in a class next door look on.

It came as a surprise when reading through our research report to realise that - inadvertently - for the first time in 2013 we have a control group of students in one of our cohorts.  It was the transition from Year 8 in our primary schools to Year 9 at Tamaki College that enabled this. When data was collected in Term One from e-asTTle testing in Reading, Writing and Maths there was a cohort (one third approx) who had participated in the Manaiakalani programme when they attended Manaiakalani primary schools, and a cohort (approx two thirds) who came from a wide variety of primary schools from around Auckland and beyond. This second group mostly came from low decile schools also.

The difference in e-asTTle data was surprising, even to us true believers.  

The full report is available online and pages below are referenced from this document.

This commentary was provided by Prof Stuart McNaughton in the brief summary of the report:

Students entering Year 9 (2013) (into Tamaki College) from the Manaiakalani schools had higher average achievement levels than those from elsewhere, supporting evidence that the primary schools are having some success in achievement levels, compared with students from schools in like circumstances. The two thirds of students who came from outside the cluster were two sub levels below the others in reading (3B). This meant the total cohort entering Year 9 had widely spread achievement levels. 

The patterns for maths and writing were similar but at lower levels.

The Reading image is taken from page 122 of the report.
The cohort on the left are the Year 9 students who were at Manaiakalani primary schools in the previous year.
The cohort on the right came from 'Other' primary schools.

This is the most startling contrast , with the red line indicating the gap between the two groups of students' achievement in Reading at Year 9.

The Maths image is taken from page 158 of the report.

The cohort on the left are the Year 9 students who were at Manaiakalani primary schools in the previous year.
The cohort on the right came from 'Other' primary schools.

The Writing image is taken from page 177 of the report. 
The cohort on the left are the Year 9 students who were at Manaiakalani primary schools in the previous year.
The cohort on the right came from 'Other' primary schools.

The challenges our high school teachers face in effectively teaching this diverse group of students are huge. 

"The pattern (also) indicates a substantial challenge catering for the range and especially the very low group, who have reading skills and knowledge which are not sufficient to engage effectively in reading in subject areas at expected levels without intensive support. The level of the challenge is such that they need to make up four full curriculum levels by Level 2 NCEA for the school to meet the 85% pass rate." p11

Research Report 2012 to 2013

At the beginning of the month the Woolf Fisher Research team of Dr Rebecca Jesson, Prof Stuart McNaughton and Aaron Wilson presented the report from their evaluation of the Manaiakalani Programme over the academic year 2012-2013. The full report (170 pages) and the executive summary (17 pages) are publicly available at this link, and the slides from their summary to educators are embedded below.

The report covers the most disruptive year a cluster could hope to have - in the old-fashioned as well as the modern usage of the term disruptive!

The 'housing situation':
The vast majority of students in our Manaiakalani schools live in rental accommodation i.e. government housing. And in 2012 the Tamaki Redevelopment Plan meant some of our families had to leave their homes (and their schools) and many lived in a state of insecurity, dreading when it would be their turn to be moved out. This meant half our schools saw increased churn in their student population. Some schools at one end of the suburb had rolls shrink, some at the other end had rolls increase to accommodate families moving around.

All in:
This was the first year that ALL students from years 5-13 in ALL our schools moved to a digital learning environment using their own netbook as their basic learning tool. The pilot years were officially over. This meant that ALL teachers were required to work in digital learning environments and the days of the research only including enthusiasts, lead teachers, pilot classes (i.e. willing volunteers) were over. It was all in! The number of uncertain or unwilling was low, but we also had a number of relievers, beginning teachers and staff changes during the year across the 8 schools included in the research data.

 A couple of weeks of reflection and discussion with colleagues have gone by and today I met with the research team for a big picture conversation.

The Manaiakalani cluster has some key goals which have not changed since 2006 when the principals' group co-constructed the vision statements. Raising student achievement outcomes is absolutely at the forefront - not just because it is a government priority but simply because our students as individuals deserve this. Early screening of our new entrants shows that across our cluster our 5 year olds start school two years behind the norm academically. For our children to catch up we have to accelerate their academic progress.  This means their shift inside a year has to be more than a year.  
We have become used to our research data from our pilot and lead teacher classes demonstrating an accelerated shift towards the norm, and even overtaking it.

This year, with 'all in', our aggregated data showed our cluster moving at the expected rate for the rest of New Zealand i.e. in one year our kids made one years shift, but as a cohort they didn't accelerate.
So yes, those who are passionately committed to this are disappointed. But given the year we had, we have to take some satisfaction that we didn't fall back into the low-achieving mould of making less than one year's progress.

"In summary, like international studies of large scale 1:1 programmes, evidence for acceleration of student achievement is limited once 1:1 programmes are rolled out in larger sizes. While there is little evidence for acceleration and higher levels over time associated with Manaiakalani schools at an aggregated level (cluster, school) there are pockets of substantial achievement. What evidence we have suggests that Manaiakalani may be associated with important educational outcomes from these pockets; for example in the  ̳pilot‘ group of NCEA Level 1 students or in the classrooms with above nationally expected gains. " p65

Lots to work on there.

The feedback from researchers spending time in the classes observing learning and teaching, and interviewing students and teachers following these observations, was more in line with the other Manaiakalani goals.

Engagement, one of the key elements of Manaiakalani 'the hook from heaven', was clearly evident and commented on

"Overall, the present data confirm predictions that the e-learning classrooms were associated with high levels of behavioural engagement, possibly sufficiently high to overcome what has previous been described as a generalised drop in engagement over the transition to secondary school." p65

"In interviews, teachers indicated that they had goals to develop students‘ independence, most often in terms of accessing their curriculum and engaging independently in learning tasks. Our observations confirmed that in general on task behaviour and 'streamlining' of classes suggests that is largely successful." p74

This is much more complex than a couple of 'sound bites' can represent. All teachers are interested in engagement inside the classroom, but this report also covers 'Active Pursuit', students who are engaged with learning and extending this beyond the classroom.  We are excited to have our hunch confirmed, that students are actively engaged in learning outside of school - somewhat atypical in Decile 1 communities. 

"The interviews with students suggest there is a notable group of students across the schools and at both primary and secondary who are actively pursuing school/academic related activities."p67

Our teachers are clearly implementing the Manaiakalani vision

"…our analyses of the classroom observations show that, in large measure, implementation of Manaiakalani reflects teachers‘ understanding of the goals of the programme." p64

"Teachers articulated high enthusiasm for and fidelity to the goals of Manaiakalani. They articulated digital access to resources and curriculum and engagement as goals. In large measure we saw evidence that teachers were operationalising these goals, implementing digital pedagogies and engaging their students in learning. This provides some indication that given shared vision, teachers in general have the means to implement this in their classes." p73

An innovative intervention covers three phases (R Jesson), and to use NCEA terms our current report card reads....

The Manaiakalani Programme:

• Implementation  - Achieved with Excellence
• Changing Teacher practice - Achieved (in some cases with Excellence)
• Student Achievement outcomes - Achieved (in pockets)

Like 1:1 interventions internationally, Manaiakalani seems to have precipitated a shift in classroom dynamics, allowing for more extended discussion, student control, personalised feedback and instant access and infinite range of learning resources. The challenge now it to understand the conditions under which this leads to improvements, not only in the learning experience, but in outcomes for learners. p73

Research Online

We now have five years of research reports available on our cluster website.  The latest report to be published is the 2011 Evaluation report of the pilot year of 1:1 Netbook implementation in 18 Manaiakalani classes.



The 2011 evaluation was designed to answer two major questions: 


(1) How does The Manaiakalani One-to-One Device Project contribute to the teaching of literacy in one-to-one device classrooms? 
(2) How does The Manaiakalani One-to-One Device  Project impact on student learning of literacy and engagement in one-to-one device  classrooms? 


The 465 students in 18 classrooms that effectively had a full year of implementation were part of the evaluation. The evaluation used achievement measures, video records and in situ observations of classrooms; one to one interviews with teachers and students; teacher surveys, and artifacts from class and individual blogs. 


To read the executive summary, or if you are extra keen to read the whole report, follow this link.


Research reports are not universally popular, so if you prefer an anecdotal approach to following an initiative I would recommend following clusternz on Twitter. This way you will hear it from the 650 or so children whose blogs feed through this account!  They are not afraid to be honest about most aspects of (school) life!  This should come with a warning though - 25,000+ tweets (each representing a blog post) have been processed in less than three years.


Farewell Colleen:
On another note, the Manaiakalani Cluster wishes Colleen Gleeson all the best as she steps into her new role as Review and Development Officer of Schools, for the Auckland Catholic Diocese.  Colleen has served the cluster as Researcher since 2008 and we are indebted to her commitment to the students, teachers and schools of the area, and her contribution to the Manaiakalani vision. She will be greatly missed.

Manaiakalani Research: 2008 - 2010

Colleen Gleeson from "The Learning Edge" has released the eagerly awaited report from her research into the Manaiakalani Project.  The focus question was  "What is the impact of the Manaiakalani Project on literacy teaching and learning?"  She has been gathering data from students, lead teachers, principals and parents over three years, 2008-2010.  She has released interim reports at the end of each year which can be accessed here.  Her final report is available here.

The Project definitely provided a motivation for writing, an improvement in audience awareness and purpose and in presentation skills. Other school interventions also had an impact on literacy achievement; however the Project has provided a purpose and enthusiasm for literacy.

The students of Manaiakalani were provided with a “hook” (e-learning outcomes published in on-line spaces) which gave these decile 1 students a voice to be heard globally. Subsequently, participating in the Manaiakalani Project enhanced their literacy, engagement, oral language and presentation.

With the advent of netbooks in 2011, schools are starting on a new and innovative initiative that, with careful planning and implementation and adequate support and funding, could be the key to 21st century education in New Zealand.

This gives us a lot of confidence in our pedagogical foundation as we move into Phase 2:  The Manaiakalani Project provides 1:1 digital learning opportunities for all students in years 5-13, rolling out across 7 Decile 1a schools in 2011/2012.

Student Internet Use in the Holidays

After the July school holidays seemed like a good time to check up on how much access our kids have to the internet out side of school.  I wanted to get a feel for the different age groups across our seven schools and so I tried to come up with a few questions that would help them to be quite clear in their own minds about what they did online.  We have filed the report that Colleen Gleeson kindly wrote up, after analyzing the data, under 'Research', but it definitely fell into the 'snap poll' category rather than scientific research!
However, the idea was to capture a snapshot of our students and I am pleased we got that.

In the week following the July holidays, teachers were asked to survey their classes, using a show of hands for responses.

Student Internet Use in the Holidays
This is what they were asked for:

"I would like to get an idea of how many of our students had access to the internet over the holidays. The Tamaki Wireless Net is going ahead, and in time the whole community will have wireless. So it will be interesting to look back and see how many kids had access to the internet BEFORE this was established.
Please ask your class these questions and complete the form."

The questions were: (with teacher clarification in brackets)

1. Did you go on the internet in the holidays?
2. Did you go on the internet by yourself ie. You held the mouse, or did you watch other people using the internet? (Tease out who actually chose what to click themselves and who were bystanders.)
3. What did you look at most on the internet? (Group the responses and give numbers eg YouTube = 13)
4. How many children visited school or cluster blogs? (To read, comment or post - you could verify this by asking what they saw )
5. Where did you use the internet? (Try to find out how they get access to it. Group the answers and give numbers eg Home = 12, Internet cafe = 11, etc)
6. What device did you use to access the internet? (eg. Computer? laptop? Phone? PS3?)

We had six of the schools respond, so the results covered Years 1-8.  Although Mike has rightly reminded us that Decile rating is not a good description of schools, in this case it is helpful as an economic reminder.  All the students in the survey are Decile 1a, and that gives a clear indication of the lack of resources available to them out of school.  Our wider community has two excellent libraries in Glen Innes and Panmure (the Mt Wellington library) and both have a number of computers available for kids to use.

What did we find out?
29 classes responded. 25% of our students had access to the internet at home.

• Approximately 50% of the surveyed students accessed the internet over the holidays at some time. 
• Of this 50%, half of them had access to the internet at home, mainly using computers or laptops.
• The majority of all age levels used the internet at home.
• The majority of students who went online accessed the internet by themselves ie they got to hold the mouse - a significant point in internet use!
• Social networks, games and You-tube were the most popular sites for students to visit. Social networks were used by all age groups. As there are no students in this cohort at the legal age to access social network sites, this is an interesting finding.
• An average of 38% of students visited their own school’s blogs or a cluster school’s blog.
• 50% of the students who used the internet had access to the internet at home. One school stood out (School E) as not having the access at home. Incidently this is the school furtherest from the libraries or shopping centers with internet cafes, so they had very little access at all.
• The older the students, the more they accessed the internet.
• A majority of junior and middle school respondents (66% and 69%) used the internet by
  themselves. 80% of intermediate students used the internet by themselves.
• Junior students mainly visited games, then YouTube and Blogs, Middle school students
  mainly visited games, then social networks and YouTube. 42% of Intermediate students
  visited social networks, then games and YouTube.
• Junior students only used computers or laptops to access the internet. The majority of
  middle and intermediate students used computers and then laptops to access the internet. However they did use PS3s, iPod touch, cell phones and X-Box as well.
   

This wasn't scientific but it gave us some useful information leading up to the first round of netbooks arriving and some students having their own device and their own free access to wireless 24/7.  

As is always the case with surveys and research, a lot of the interesting information comes out in the discussions AFTER the process.  

• A few teachers saying they were surprised by what their students revealed - mostly in terms of what they did online and the amount of access they were able to find. They can't be having these kinds of discussions with their students.  
• The amount of under-age social networking accounts owned means we will have to review carefully how we managed our filters for the wireless infrastructure. We will be supplying it to them free to use out of school.
• A teacher from School E who blogs with her class prolifically, and gets very little comments on the blog and few international 'hits' found only one student had access to the internet over the holidays.  Her question, "Why should I bother continuing when only our class reads it?" was hard to answer.
• Conversations from the students about why they didn't use their own blogs more revealed a very obvious scenario; they only get a very limited time on the internet and when they do get a look in it's YouTube, Games and Social Networking all the way.  It will be interesting to see if the Manaiakalani project ends up reflecting overseas studies; when students have their own device 24/7 and they are not time pressured they will multitask between recreational, social and learning based activities.  Time will tell.