Questioning the Data

Educators from around the world gathered in November at The Quest for Increased Student Achievement: Educational Systems That Are Working conference in Richmond Hill.

What are their systems doing to help children learn? How do they really know if it's working?

When school systems look to each other, seek expert advice and scour the world for innovations and ideas to boost student achievement, all talk gravitates to data. Beyond who's doing what and how, they want to see the proof – quantifiable results – of student improvement.

But is the zeal for assessment data simplistic?

Are the parameters too narrow?

Do students really benefit?

Late last fall, educators from 64 Ontario school boards, Hong Kong, New Zealand, the US, New Brunswick, Alberta, Finland and Great Britain met for York Region DSB's annual Quest conference to discuss best practices and debate the best way to measure what works.

“The more narrowly we define achievement, as testing literacy and math, the less students will achieve,” said keynoter Andy Hargreaves, Thomas More Brennan Chair in Education at the Lynch School of Education in Boston.

Quoting British author David Hargreaves (no relation), he said there were four kinds of achievement: intellectual/cognitive, practical, interpersonal and motivational.

Expanding on this he noted that intellectual achievement dominates schools. It's knowledge remembered and easily tested. Practical achievement is learning how to do something or the practical application of knowledge, such as science-fair projects. Practical achievement is important, particularly for slow learners, but is less apparent in the system. Interpersonal achievement, our emotional intelligence, reflects our sense of security, tolerance and global awareness. Acknowledging motivation as a form of achievement, we can see that leadership is a form of learning.

Hargreaves said, “Happiness, essential for achievement, is itself achievement.”

“We need to create capacity to increase student achievement over time,” he said. Improved capacity, he added, can come not only from increasing supply but from decreasing demand. We can and should train, develop and provide for teachers, but we should also reduce the demands placed on teachers – such as paperwork, reinventing report cards and downloading new initiatives.

“Bureaucracy increases demand,” he said. “The most effective schools manage their demands as a community.” They do this by setting priorities, filtering and selecting.

Why does the Ontario government set long-term targets for the environment but short-term ones in public education? Hargreaves asked. Both are critical to our long-term sustainability. “If these are life and death matters, we should have some consistency.”

For example, he said, our education system has a history of rotating principals through schools. Each new school leader brings a vision, layering over what occurred before, good or bad. At the same time the system leaves teachers to manage issues of equity in the classroom. The curriculum doesn't do it.

Hargreaves said that there has to be a middle ground between the no-targets, no-testing mentality of school systems in Wales and Finland and the strict goals and tests in the rest of Great Britain and Canada.

Widely regarded as having one of the world's best education systems, Finland sent a delegation to explain its model, where teachers are respected, teaching as a career is valued, schools are free and open to all, are unstreamed and serve hot lunches to students. It was noted that Finland has a relatively homogeneous population in which 92 per cent speak Finnish and 85 per cent are Lutheran.

Conference co-facilitator Michael Fullan, author, education advisor to Ontario's premier and former dean of OISE/UT cautioned attendees not to be “one-factor Charlies,” looking for a single solution to complex problems and situations. Just because Finland doesn't have standardized testing doesn't mean that's the way to go, he said.

“We're at the early stages – so far from what needs to be done.” Conference co-facilitator Michael Fullan is education advisor to Ontario's premier, an author and former dean of OISE/UT.

Peter Hill, Secretary General of Hong Kong's Examinations and Assessment Authority and co-facilitator with Fullan, said that “data can and should be our friend.” He advocated that educators own it and gather it on all factors affecting student outcomes – “for all students, not just for those at level two being lifted to level three.”

Hill advocated realistic targets but warned, “Be careful. You need the right amount of pressure and support to achieve them or they can be counterproductive.”

The challenge, Hill said, was how to capture data one day and turn it into useful information to guide instruction the next.

“Our goal should be that learning in every class is equal to the learning in the most effective class,” he added.

Education is about designing learning systems that work for people, Hill said. “I feel that people study education like it's a phenomenon. We have to change the reward structures and get them to do the design work.” Hill pointed out that, unlike other professionals, most teachers don't have computers and telephones in their classrooms. “They should. They need them.”

Hargreaves didn't dismiss the need for standardized tests. “We do need quick wins – confidence-building lifts that show we are achieving over time.”

However, there are data of all kinds, including student-attendance and student-engagement data, that need to be considered, Hargreaves said. “The strongest systems make formal use of data and their own judgment. The weakest systems are driven exclusively by math and literacy data.”

“Happiness, essential for achievement, is itself achievement.”

Avis Glaze, Ontario's Chief Student Achievement Officer and head of the Ministry's Literacy and Numeracy Secretariat, reminded educators that “context matters.” She argued the need for testing and setting targets for student achievement.

“Do we want to turn back the clock? No,” Glaze said. “Literacy liberates. We do not want any child to leave the system without being able to read, write and comprehend.”

As a member of the 1995 Royal Commission on Learning, whose For Love of Learning report and recommendations changed learning in Ontario, Glaze heard the cries of parents across the province: “What are my children learning? By when? How will I know?”

“Public education must deliver on the promise that all students achieve,” Glaze advocated. “We must improve. The public decides how good public education is. They want higher achievement.”

Children in Ontario are nowhere near where they can be, she said. “Let us stay the course. We need to raise the bar for those who do well and help those who do not.”

The secretariat, only two years old, has funded 250 projects developed by school boards. Student achievement is on the rise, Glaze pointed out. “We need data to help with improvement.”

Education Minister Kathleen Wynne said the job of public education is to remove the barriers that keep kids from reaching their potential, such as having too many kids in primary classrooms.

Confidence in public education is up, but 75 per cent of the public doesn't know it, Wynne said. “Test scores become political tools but what's important are the resources we pour in to support that.”

To wit, Wynne announced the $25-million Ontario Focused Intervention Partnership Program to help students struggling to meet provincial standards in reading, writing and mathematics. She said that there are 800 schools where two-thirds of the kids don't meet the standards.

Our best, most advanced systems are good but not great, Fullan said. “We're at the early stages – so far from what needs to be done.”

“Our goal should be that learning in every class is equal to the learning in the most effective class.” Conference co-facilitator Peter Hill is Secretary General of Hong Kong's Examinations and Assessment Authority.

Brenda Willis, Director of Learning Support Services with the Edmonton Catholic School District, said the 85 schools there were “fiefdoms going in different directions” until the system used assessment to create cohesion and put everyone on the same path towards student achievement.

“The systems that I see that work pay attention to contextual influences,” said Alan Boyle, Director of Leannta Education Associates in London, England. “The most important is the home.”

Teachers need to go into the communities in which their students live and understand the power of peer pressure, Boyle said. “Talk to students. They're there every lesson.”

Douglas Willms, Chair in Human Development with the University of New Brunswick's Faculty of Education, said the most critical transition for students happens around age 10 when they go from learning to read to reading to learn. “Kids who don't make that transition get stuck in the system.” Willms added that schools have all the data they need when a child enters school in kindergarten to predict how they'll do. “We can intervene early. We don't need to wait for them to fail.”

Hargreaves said, “Teaching is beset by a culture of present-ism.” Teachers are caught up in the moment – wiping noses, soothing the child with the scraped knee, bringing the bully and bullied together to reconcile. Teachers work in the present. Kids make it so. It is difficult for teachers to focus on the longer term.

He believes it will take great discipline in policy, structures and leadership to push school systems to connect short-term initiatives and activities to the long term and to lever change.

He advocated a range of strategies. In the short term he suggested giving kids test-taking strategies and better nutrition – “encouraging kids to eat more bananas and drink more water” – initiating parent-teacher conferences and celebrating student accomplishments. Mid-term strategies might include teacher-mentor programs to help struggling colleagues and school training days, while long-term strategies could include organizations that restructure school leadership teams.

Hargreaves said that Ontario already has the right people – well paid teachers who are motivated and highly qualified – but the system takes the passion out of people rather than putting it in. Releasing the energy of teachers will release the energy of students. “Bring people together, peer to peer,” he said. “Teachers push each other.”

The focus, Hargreaves said, should be on student achievement of all kinds.

“When we attend to all, we will lift achievement in literacy and math.”