How Do We Know What Works?

Whether you are a school leader, department head, or main-scale teacher, there are some simple steps to working out what works.

1. Identify What Works

Professor John Hattie started the research revolution in schools, with Visible Teaching in 2009. There’s now a sequel. His basic message was this:

Everything works!

The fact that something will work is not enough to justify doing it.

So, only take on new ideas which are likely to be better than average. The value he put on this was d0.4, which was the equivalent of 5 months extra progress. An easy shorthand might be to think of it as 0.4 in Progress 8 terms.

For example, imagine 10 students had this intervention. At the end, 6 would see no change in grade, but 4 would go up a grade. (Of course, each might go up 40% of a grade, and none of these might take them over the threshold for the next grade - but I hope you get the idea).

If it isn’t going to add a decent chunk of Progress 8, don’t do it. (In this thought experiment, I’m assuming every subject counts equally in terms of Progress).

2. Estimate the Progress 8 Impact if Everyone Did It

This is gut instinct estimates, based on what you know.

So, imagine you want to make sure that all lessons end in the same way. Students packed, silent, dismissed in rows.

I often ran my room like that - I’d walk the rows making sure there was no litter, no drawing on desks. Tip top, spit spot, off you go children, up to the atmosphere, up where the sky is clear - high aspirations.

But, would I make this compulsory?

3. The Wisdom of Crowds

Almost no experts have enough information to make accurate predictions. But, if we canvas experts with a range of expertise, we are likely to get much closer to the ‘true’ picture. That’s what The Wisdom of Crowds teaches us.

So, now I would present this idea to SLT and Heads of Department. If every lesson ended this way in a student’s career, how much of a grade, or more than a grade, would the student achieve? Answers on it post-it note. One minute thinking time, then commit.

Think about this. In a five lesson day, only two of these transitions are potentially quicker than not implementing this, as three are before break, lunch and home.

Ok, now you simply take the mean of the room.

If the Progress measure is high enough to justify the work, you do it. Otherwise, do something else.

3. Estimate the Counterfactual

Now you think of what else you might implement instead, with the same degree of effort and time. That’s the counterfactual.

Let’s say it is improving your Do Now retrieval tasks so that, instead of having 5 retrieval questions, they have 20. (Why this is so desirable is another post).

You repeat the exercise with the post-its.

Compare the two.

Another simple way of doing this is to write down the names of all the teachers who you know have this end of class routine, then look at their Progress 8 results.

Then take all the teachers with positive Progress 8 who don’t have this routine. The comparison of these two groups will give you an interesting steer as to whether these routines are likely to have an impact on P8, and, best case scenario, comparing the two means will suggest by how much.

4. Estimate the Likelihood of Implementation

Still busy with those post-its!

Let’s imagine the SLT want to embed these ideas across the school. So they are going to focus on it every time they visit a lesson. They are going to try to visit the end of lessons - if it is the packing away that your team choose, or the beginning of your lesson if they focus on Do Nows.

You (the senior leader) have an idea that having a fortnightly focus, where you celebrate teachers’ success in emails and in public is going to achieve this. Next fortnight, you focus on something else.

So, you put it to the team.

They each fill in their post it notes with a percentage likelihood of success. How far do you think this approach will successfully get all teachers doing this? Scores out of a hundred.

If the mean is less than 75%, you need to ask for another method.

Research Schools

Regular readers will know that the data shows research schools, on average, add almost no extra progress to their students, year on year. One such is the Durrington Research School, and so I offer this post for free - it is normally a paid part of my school improvement role.

The Durrington Research School has a brilliant newsletter, full of excellent ideas and research.

Their newsletter about end of lesson routines prompted this post. I don’t know their context, or how this became their focus, or how they chose a fortnight focus as a model.

These may be exactly the right decisions for the school. I simply don’t know.

And, my opinions are a sample of one. Pretty useless.

But, that’s why the post-it, and those four planning questions, are so important.

If you want to know how to apply research to get amazing progress in your lessons, The Slightly Awesome Teacher shows you how to do it with fantastic work-life balance.