Regularity

[Gerry Wolff]

Dear Sergio,

Thanks very much for your thoughts. I have made some comments, below, on
some parts of what you have said.

Sergio Navega wrote:

> From: Gerry Wolff <gerry at sees.bangor.ac.uk>
> >> [snip]
> >> Why is it so difficult to catch the nature of information? Because of
> this
> >> process duality of information. When we reduce information to 'dead
> data',
> >> we stop the machine to understand how it runs.
> >
> >I don't know what you mean by 'process duality of information'.
> >Information is simply anything which is observable and which contains
> >some variation.
> >[snip]
> 
> This last sentence contains important points, IMO. I couldn't
> resist exploring some related ideas.
> 
> Information is something that can be captured by one's senses
> (observable) and which contains some kind of variations. I'd like
> to suggest some additional ideas about what I think are the
> important aspects of this 'variation'.
> 
> First, I propose to reduce all the aspects related to 'data'
> in the universe as belonging to one of three categories:
> 
> a) The totally randomic data
> b) The totally regular data
> c) All the remainder
> 
> (obs: item b, in reality, is something that does not exist; we
> can't find nothing that is totally regular, we can find only
> things that are regular due to the precision of our senses
> or accuracy of our capturing instrumentation, but quantum
> variations prevent two really identical instances of an event).

Yes, normally - for analogue data - we have to set some minimum to the
discriminations that we can make. In effect, this is an analogue to
digital conversion at the finest grain that we can discriminate.

> 
> The main idea of my message can be summarized in this assertion:
> I would say that information may be derived only from item c).
> 
> Examples of item a) are white noise: set your radiotelescope to an
> unpopulated area of the sky (difficult to find!) and you'll listen
> to that characteristic 'sssshhhhh'. Whether due to thermal noise
> of the electronic circuits or to the random signals picked up by
> the antenna (often both), this signal, *taken in isolation*, is
> able to inform us of nothing.
> 
> Examples of item b) are the regular signals of a rotating pulsar
> or a binary star: you have that characteristic 'rat  rat  rat'
> signal with extremely regular intervals. Again, taken in isolation
> this regularity is unable to carry any information.
> 
> Anything in between (item c) is what we can find when a previously
> random source suddenly becomes regular OR when a previously regular
> signal becomes random (or does not repeat in the same rhythm).
> It is just during these transitions that information can be extracted.

You are, of course, entitled to define 'information' in any way you
like. But what you have just said is *different* from the concept of
information in Shannon's information theory or Algorithmic Information
Theory (which, together, we can call 'engineering' concepts of
information).

People often assume that random data contains very little engineering
information. This is probably because, for humans, random data is not
interesting. But a given body of random data actually contains the
maximum possible engineering information for a given symbol set. Turning
to data which is very regular, we need to be careful to distinguish the
raw data from the same data which has been encoded in a compressed form.
A body of regular data which is the same size (in bits) as a body of
random data contains the same amount of information as the random data
if we ignore the regularities. But if we use the regularities to recode
and compress the data then it is likely to be smaller than the body of
random data.

If random data is truly random, it is not possible to compress it. In
Algorithmic Information Theory, randomness is *defined* as
incompressibility!

....

> 
> This is also what I think explains our curiosity. We're easily
> bored by "known" circumstances. But when we listen to an unusual
> sound, or when our computer does something unexpected, or when
> someone shows us an 'yellow apple', then our attention is grabbed.
> We become 'desperate' to find a way to look at the event from a
> point of view that cancels the unexpectedness.
> 
> As a simple example, recall when you move to a new house. Usually,
> this house "clicks and squeaks" in very specific ways. During
> the first night in which you're at that house, every click will
> raise your attention. This occurs until some days latter, when
> these clicks will be "stored" in our perceptual mechanisms
> associated with regular and "explainable" things (the click
> of the wood of the door; the click of the aluminum window, etc)
> and this transforms it into a "known" pattern, unable to raise
> our attention anymore (we finally sleep unconcerned). Moreover,
> the perceptual work of deciphering future instances of these
> clicks goes from the conscious to unconscious, denoting the
> its lack of importance . Only "new" clicks will jump from
> the unconscious and "wake" the conscious mind, indicating a
> potentially important information on its way.
> 
> Much of what I think is learning (in human terms) appears to be
> related to this mechanism. We're bored by regularity, but we're
> constantly trying to produce it. Learning is, in my way to see it,
> the result of our efforts of transforming unknown things into
> regular and predictable things.
> 

I don't think you need a new concept of information to account for the
brain's tendency to look for regularity in the world. Everything you
have said above fits in with standard engineering concepts of
information. If we are shown random data, we may see ghostly patterns in
it but our ability to detect regularities in the world comes into its
own if we are shown data which contains regularities. For many kinds of
regularity, we can 'see' them very easily but there are some which are
difficult to detect. An example of the latter is the sequence of digits
following the decimal point in a calculation of pi. Given only the
sequence of digits, it would be very hard for most people to realise
that they are the output of a simple formula.

Well, that's all I have to say. Really, my basic point is that it is not
necessary to invent a new concept of information. Standard engineering
concepts of information will take us a very long way.

Best wishes,

Gerry