Wednesday 2 November 2011

Information inflation: school reports

It is said there's more information around today than ever before. It is taken as a given fact, though people argue over whether it is a good thing or not (information riches or information glut?). But is it necessarily true?

A few years back, before the financial downturn, people in the UK would say ‘there’s more money around’ to account for the fact that people seemed to be consuming and acquiring goods more than ever before.  You could have used the same expression: ‘there’s more money around’, about the Weimar Republic, in the 1920s & 30,  but it would have meant something entirely different

Stamps from the Weimar Republic, showing inflation (from left to right: 1923, 1926 & 1928)
So, is the apparent increase in information real or hyperinflation?

With money you can make some meaningful comparisons - I was once told that the price of a Mars Bar is a good benchmark - but how can you do that for information?

I've been pondering this for a while (and had a paper on the topic rejected!). Here's some evidence for information inflation.

School reports, when I was a child, consisted of one or two hand-written sentences on each subject.  The reports that my children have brought home are much more substantial: a printed booklet with paragraphs of, typically, between 50 and 200 words on each subject. At face value, I’m getting a lot more information about my children than my parents got about me.

School reports today are frequently put together with the aid of specialised software that reduces the workload on the teacher.  An extreme example of this is The Report King that can be used in England.  Using this software, all that a teacher needs to input is the name of the pupils and their grades for each subject, selected from a pre-defined set such as: h for higher achiever, m for average to more able achiever, l for average to less able achiever and sen for students with special educational needs.  The software then writes the whole report, drawing on the statements contained in the National Curriculum for England. For example,  entering m for ICT for one pupil (John), generated:

John has extended his knowledge of a variety of computer programs and he can log into the network without support.  He has explored a variety of features included in software for composing music and is aware that questions can be turned into search criteria when using data handling programs.  He has found information relating to his topic work from given websites on the worldwide web and explains patterns that govern a computer simulation

Entering the same grade for another pupil would generate a similar but different paragraph, because the software makes use of different wording and draws on different parts of the curriculum to ensure two different pupils don’t get the same report.

In terms of the Shannon model of communication, the message from the teacher is entirely specified by the name of the pupil and the grade.  Since the grade was a selection of one from four, the information content (assuming each of the four grades was equally probable) about the pupil’s performance is 2 bits.  That whole paragraph (73 words) is a symbol for the message m.

Comparing this to the hand-written sentence of times past, it is easy to see that a written sentence is likely to contain a lot more than two bits of information. Even the most harried teacher is likely to be selecting their sentence from a lot more than four possibilities.  For really effective communication, though, the face-to-face meeting at the parents evening is still, as it always was, better than the report, whether hand-crafted or computer generated.

There’s an argument that there is more information in the computer-generated report, in this case information about the National Curriculum, but it’s not about John and it’s not from the teacher.

In conclusion, just as to say 'there's more money around' due to hyperinflation is a misuse of the word 'money', so too - maybe - to say there is more information around today is a misuse of the word information.


The Informative said...

That´s a terrific good point.

I guess we´re witnessing similar information overloads in every other field, too, maybe even in the "hard sciences".

As to the scientific aspect of it, I have an interesting additional question:

In the example with the school reports and their (poor) informational content, Classic Shannon Theory is used to estimate this content in bits. So there´s a huge gap between the "gross" informational content the report APPEARS to have for the uninformed external recipient, and its true net content it must have, because it was the net content typed in by the sender.

The difference could in fact be seen as redundancy/ inflation.

But how could the recipient ever know about the difference, if not by having some additional "second order" information ?

This could start an infinite regress, leading to some incompleteness results ?!

David Chapman said...

I've got in mind a model whereby all parents are issued with a copy of the software when their child joins the school, then the teacher could indeed send the termly 'report' using just the two bits which instruct the software to generate the report for that child. That's the only communication needed from the teacher.

However, of course, the parent would already have the software which contains, in effect, the entire school curriculum, the entire National Curriculum for England that is. Is that information? The 'report' bits for their child extract some information from the software that is relevant to them, so maybe it is useful.

This needs teasing out a bit further, and I think it is worth doing because it sheds light on the nature of information.

Allan Jones said...

Despite your paper being turned down (which you refer to), I think the idea of information inflation is too good to drop.
It's possible, though, that there is both more information around, and that there is information inflation. The crude evidence cited by expansionists tends to be annula numbers of papers or books published. On that measure expansionists would probably say that information (or knowledge) was increasing exponentially. The subjective impression, though, as you suggest, is that 'real' information is being spread ever more thinly. It might still be increasing, but at a much lower rate than usually claimed.

A possibly irrelevant example I often find myself mulling over:

I keep hearing and reading that researchers have shown that there might be a connection between X and Y. I think 'Surely that was the position before the reasearch was conducted. Why else would you embark on the research, other than the possibility of a connection?' Unless the research establishes that there is or isn't a connection, has it done anything? Possibly it has, but not as much as seems to be supposed.

Another example. As you know, we keep publishing learning outcomes for our courses. But how much information do these contain? What do they tell you? I often feel they only tell you anything if you already know what's in the course.

David Chapman said...

You could well be right that it (whatever it is) might still be increasing, but at a much lower rate than usually claimed, but it needs testing.

I'm not sure about your comment about finding things that 'might' be, though, because I don't think we ever have certainties, only reduced uncertainties. Research results should generally come up with things like "... there is a 5% chance that the correlation between X and Y is random" which is translated in the press as "there might be a connection between X and Y". That seems to be fair enough, to me.

As to learning outcomes, I'm certainly with you there! I think learning outcomes are like mission statements. They are only of use to the people who write them. It is the process of deciding what you are teaching and having to write it in a list of learning outcomes that is useful.

Teraknor said...

And then it would seem that with inflation comes information inaccuracy and information ambiguity.

Because our student is flagged as M in given subject, what is in the mind of the teacher may differ from what exists in the report writing software repository. All I see is a series of vanilla statements that could equally apply to someone with lesser or greater skills/understanding in the same subject.

To the informed parent, this may be a concern, to those who take on trust the information provided by the school. Are they being deceived by the information being shared?

I agree, there is inflation at play here, but with years of experience in the game of writing reports for students and also on the provision from schools, colleges and universities. My instinct tells me that a key proposition in such a system is to ‘say nothing’ and ‘never be damned’.

David Chapman said...

"‘say nothing’ and ‘never be damned’" or don't put it in writing anyway? I would think teachers could be far more honest and candid face to face.