Estimation and compression

In the current (September 09) IEEE Information Theory Society Newsletter, (Available online, but the September 09 one isn't there yet), J Rissanen writing on 'Optimal Estimation':

Soon after I had studied Shannon's formal definition of information in random variables and his other remarkable performance bounds for communication, I wanted to apply them to other fields - in particular to estimation and statistics in general. After all, the central problem in statistics is to extract information from data. After having worked on data compression and introduced arithmetic coding it seemed evident that both estimation and compression have a common goal: in data compression the shortest code lenth cannot be achieved without taking advantage of the regular features in data, while in estimation it is these regular features, the underlying mechanism, that we want to learn. This led me to introduce the MDL or Minimum Description Length principle, and I thought that the job was done. [However...]

My emphasis.

Meaning, and surreal experiences

Towards my modest exploration into the relationship between information and meaning...

Surreal experiences boost brain power

Psychologists at the University of California - Santa Barbara and the University of British Columbia have found that exposure to surrealism, by say, reading a book by Franz Kafka or watching a film by director David Lynch, enhances the cognitive mechanisms that oversee the implicit learning functions in the brain.[...]

"The idea is that when you're exposed to a meaning threat - something that fundamentally does not make sense - your brain is going to respond by looking for some other kind of structure within your environment," said researcher Travis Proulx. [...]

Meaning, explains Proulx, is an expected association within one's environment. Fire, for example, is associated with extreme heat, and putting your hand in a flame and finding it icy cold would constitute a threat to that meaning.

(My emphasis.) I'm thinking that the 'expected association within one's environment' corresponds to my conversion of information into meaning in a trapezium. It is the 'decoding' at the receiver. A 'meaning threat' is when that decoding goes wrong.

In a nutshell, what technology has really helped students?

I've just written this in an email:

Personally I think that of all the advances that computer and communications technology has delivered over the past 10 or 20 years - CAL, online delivery of materials, access to additional resources, production methods, eTMAs - it is online forums that have probably delivered the most benefits to [OU] students.

It suited my argument to make this claim, but I think it's true. In a nutshell, what has really made a difference to our students is the ability to talk with each other and, to a lesser extent, tutors and lecturers.

Has OLPC failed?

One Laptop Per Child - The Dream is Over says this blog post from the UN. "It’s time to call a spade a spade. OLPC was a failure." Several very interesting comments on the post disagree, though.

One of the claims in the post is: "They abandoned the human-powered power source.". A comment from Peru explains:

We don't want our undernourished children to spend their scarce calories generating power for a computer. A well fed human being can generate 20 watt, forget about children extenuating their breakfast less bodies in running the XO's after walking 4 hours to school. It seemed attractive to people who have no idea of what poverty really is.

One of the comments comes from Negroponte himself, including:

I suggest you look more carefully at Uruguay, Peru and Rwanda. In the case of Uruguay, every child has one. That is pretty amazing. Peru is headed there. Rwanda too.

All in all, a fascinating discussion of the project.

Meaning in biology

A tweet from Richard Dawkins:

"46% of our DNA was provided in the past by invading viruses, 1.5% from primate & animal ancestors."

led me to this piece* "Friendly viruses aided human evolution" in the Irish times. I find it fascinating the way in which life intermingles, and it is an example of why the 'tree of life' model is a misleading over-simplification.

But the specific thing I started to think about arose from things like the following:

The same is the case with HIV/Aids in humans. “HIV 1 is an acute retrovirus like the viruses in our genome,” says Ryan. “Even at its worst, we know that it is changing and selecting specific genotypes that it feels most comfortable with, while killing off others.

(My emphasis.) You get this anthropomorphism all the time, imbuing meaning and intentionality to things that are 'really' purely mechanistic. (It is in there in the phrase 'selfish gene'.) I'm sure the authors would mostly dismiss it as a figure of speech, a shorthand. But one thing I'm arguing with my layered ideas of information, is that at each layer boundary we see information/data acquire meaning. Thus high and low voltages mean 1s and 0s, a particular pattern of 1s and 0s mean the letter 'a' etc. The thing is, is the meaning I'm talking about here the same as semantic meaning? I think the usual response would be: no, it isn't. But I'm wondering, do the layers go all the way up (and down)? Is there a point where there is a qualitative change to semantics?

I think it relates to the age-old problem of how you get conscious thought through physical mechanisms - mind-matter dualism. I've no answers, of course, as ever, just flagging something of interest.

---

*John Holden reporting on the work of Frank Ryan. "Frank Ryan will be publishing his third paper on viral symbiosis in the Journal of the Royal Society of Medicine later this month".