Next steps

Don't be a data slave

Data will help your decision making – but be wary of its limitations, says headteacher Chris Luck.

Conventional wisdom would seem to suggest that the more data available to someone making a decision, the more informed and by implication the more effective that decision will be.

Certainly leaders in education today are expected to make decisions based on analytical approaches and rigorous data analysis. Headteachers are urged to use data to analyse and evaluate. The ability of heads to analyse data has become more and more important and the analytical abilities of prospective heads are subject to close scrutiny at interviews.

And there is certainly no shortage of data for heads to analyse: data that may be internally generated by the school in terms of teacher assessment, optional SATs test or any other standardised tools, and Raiseonline. Then there is the annual achievement report that all schools are required to provide by law – often an opportunity for headteachers to show off their amazing abilities at extracting meaning from a bewildering array of data.

“Not all data is equal.”

But is it true that the more data you have the better your decisions?

The problem is that not all data is equal. Quite often heads are faced with information that might have any of the following characteristics.

Missing information

Some bits of data may actually be missing, giving at best a distorted picture of what’s going on. And of course the bit that may be missing might be the most important bit! There is much more to children’s learning and achievement than their test performance in maths and English but if this is the only information that leaders used in assessing a school and devising strategic ways forward then it may surely miss the point.

Data can only ever provide evidence for part of the total picture. Albert Einstein’s famous saying that “not everything that counts can be counted, and not everything that can be counted counts” certainly holds true for education, and focusing on aspects of school purely because they are measurable rather than important is surely the bane of our time.

Interpretation

Data on its own is useless. It needs to be interpreted in order to become useful but sometimes it can be difficult to make sense of the data that is available. This might be simply because there is too much of it or that it is presented in a way that makes it difficult or awkward to make sense of.

Manipulation

Data can always be manipulated, sometimes consciously and sometimes unconsciously. It is easy to get carried away with analysing data and perform some wonderful numerical wizardry simply because it is possible. It has become very fashionable to ‘drill down’ into assessment data. But this can lead to the situation where a significance that is totally unwarranted is given to data based on an ever-decreasing number of pupils.

Noise

Not all information is important. Sometimes there can be an overload of information that becomes a noisy fog of data, which may well hide some crucial information. More data is not necessarily better.

So it can be argued quite convincingly that extensive and laborious data analysis can actually reduce the effectiveness of decision making. Back in 1986, Roy Rowan in his book The Intuitive Manager identified what he described as “analysis paralysis”.

“If school leaders rely purely on data they will find themselves forever chasing their own tail.”

He wrote: “Constantly accumulating new information and numbers, without giving the mind a chance to percolate and come to a conclusion intuitively, can delay any important decision until the time for action expires. This immobility, caused by substituting study for courage, has been labelled analysis paralysis.”

This kind of data paralysis can be a particular problem in failing schools where ever more desperate attempts to extract meaning and purpose from an ever-expanding pool of data merely distracts the school leadership from the main tasks they need to undertake. Analysing data isn’t the answer to every school’s problems. The problem is of course that people external to the school often only have data and nothing else, so end up making assumptions and providing advice on limited and shaky foundations.

Data taken out of context can be very dangerous. Data will always need interpretation if it is to become useful information that can inform future action. There is no short cut for the experienced eye of the professionally informed headteacher in interpreting data in the light of real educational settings in his or her school.

Conventional wisdom would suggest that decision and actions should follow on from analysis of data but there must be many examples of positive school development that was initiated by a headteacher’s gut feeling that something was not right – a feeling that was only later confirmed by analysis.

A major drawback of data is that it is always historical. If school leaders rely purely on data they will find themselves forever chasing their own tail. Only a headteacher’s in-depth knowledge of a school coupled with intelligence, analysis and intuition can find a way forward that meets the unique setting for that particular establishment.

Chris Luck is headteacher of Eastfield Primary School in Enfield.