Philosophical Foundations

[Sergio Navega]

Phil Goetz wrote:
>
> Sergio Navega wrote:
>
>> >Yes.  Roughly: Intelligence is recognizing regularities.
>> >Compression is recognizing regularities.  So, cognition = compression.
>> >
>> >Phil goetz at zoesis.com
>>
>> Phil, although I agree with the overall tone of your equation, on the
>> specific level I would find it too reductionistic. Someone who is not
>> familiar with Gerry's work could be tempted to take your equation and
>> derive this one:
>>
>> Artificial Intelligence = WinZip
>
>I'm not familiar with Gerry's work, and I could be tempted to derive that.
>A nervous system quickly maps inputs from sensory neurons to outputs to
motor
>neurons.  Period.  The space is big and sparse, so you have to compress
>it to a smaller representation that contains the significant distinctions.
>Then you decompress it to motor actions.


Phil, thanks for the comments, it lead me to think a while.
That's a nice way to see things but I don't agree entirely with it.
You seem to equate motor actions to decompressed sensory inputs. I don't
find many cases in which this is true. There's roughly two kinds of
organisms on Earth: those who learn and those who react, with all kinds
of degrees in between. In the case of those who react, like a cockroach
for example, sensory inputs are almost directly mapped to motor actions
but this mapping *was not* built by a learning procedure of the organism:
this mapping was refined through evolution, that selected those
specimens who were more agile in escaping predators.

On the other side of the spectrum, we have humans. Most (if not all)
motor actions of humans is learned, not inherited. A human baby is
defenseless, he must learn how to coordinate arms and legs (among
a dozen of other things). But instead of compressing sensory inputs
and mapping these directly to motor actions, the baby develops
world models (that are, one could say, compressed versions of the
real one).

The great difference among this and a conventional data compressor
is that while WinZip concerns itself in reducing as much as possible
all input it gets, the baby is concerned in developing structures that
represent, in a useful manner, as much of the world around it as is
possible. In several cases, this is the same as compression, but
often it means the *opposite* (creation of data). I'll return to
this point below.

>WinZip is just not efficient enough to be intelligent.
>I suppose the fact that you are decompressing to a different language
>than you compressed from means that "compression/decompression"
>is not a sufficient description.
>
>> a) Recognizing regularities
>> (perception of reocurring patterns)
>
>Which is key to compression.
>

It is key to compression but it is not compression, it is just something
that compressors must be good at. Intelligent agents also must be good
at it, but there's something *fundamentally* different between the two.
Compressors look for the smallest way of representing data. Intelligent
agents look for the greatest pattern recognition ability, even if this
process increases the size of the data.

To appreciate the subtle difference, it's necessary to look at the
main effect of learning. This is not a usual way of looking at it.
Learning is, in my opinion, the ability an agent have of refining
and improving its pattern detectors. Just after being born, babies
have a very limited set of pattern recognition mechanisms. They
are unable to see any differences in, for instance, an orange and
a plastic toy car. Some cognitive psychology experiments with
babies revealed that they don't seem to be surprised if you
put two oranges behind a box and then retrieve an orange and
a plastic car.

But the babies *notice* when you put two and retrieve *three*. This
seems to indicate that they are innately sensitive to numbers
but not to shape or color (which is amazing, giving that the latter
seems more "visible" to us).

With time, babies progressively develops those recognition
mechanisms  (involving, among other things, a lot of specialization
in the visual cortex), which allows he to better recognize and
classify objects and events. As a better pattern classifier the
baby will also be a better compressor. But the intent seems to
be oriented toward categorization:

>> b) Grouping of regularities according to several similarity criteria
>> (conceptualization and categorization)
>
>Which is the same thing.

Categorization may be seen as compression, when we look from a certain
angle. But it is the *opposite* of it, from another angle. Take an apple,
for instance. An apple belongs to the category of fruits. So, "fruit"
is something that compresses apples, oranges, etc. But this effect
of compression disappears when we remember that "fruit" is a concept
*in addition* to apple, orange, etc. It does not replace them. It is
something more. Then, instead of compressing, categorization acts
in the reverse side: it augments the number of "meanings" an agent
must keep in memory.

Categorization is difficult to do (and is important to AI) because it
can easily lead to intractability: an apple is a fruit, belong to the
category of edible stuff, is part of the vegetal kingdom. But it is
also an exemplar of round objects, solid objects, textured surfaces,
rotting things, etc. I could find thousands of "irrelevant" categories
in which an apple would fit. This "explosion" of categories, if not
taken care properly, would fill the memory of an agent rapidly,
exactly the opposite of what compression is supposed to do. But then,
why we go into the trouble of categorizing things? Because of
causal models and theories:

>
>> c) Development of causal models
>> (rules, theories, formalization)
>
>Which is a type of categorization.

Causal models depend on categorization, and also can be seen as
a way of compressing data. The fire below a pan causes water to
boil, and this is something that subsumes most instances of
fire below water. Similarly to categorization, this also involves
the "storage" of more information, so it can be seen as something
that does not reduce the size of data. On the other hand, this
avoids the memorization of all instances of fire below water:
thousands of possibilities are condensated with a single
causal model. But this is not acting here just as a compression of
*incoming* signals: it is acting as a *prediction* of unseen cases
(sort of compressing "future" data), and this prediction is exactly
what leads us to theories (which I see as a knotted, interrelated
set of causal models, one leading to the other).

We can make entire virtual worlds in our minds with the use of
theories and this is, obviously, the greatest evolutionary
advantage we have over other animals.

>
>> d) Exploring uncharted territories
>> (creativity)
>
>Which is necessary to do really good compression.

Here I would say exactly the opposite: creativity is generation
of new data. We explore new ideas to generate more information
(sometimes this generation is useful to us :-).

>I could suppose that creative leaps of thought are basically analogies,
>generalizations, and specializations of other thoughts, and that although
>the application of any specific instance (such as writing a symphony)
>might not seem to be serving that end, the mechanisms that allow it
>serve that end.
>

Your comments made me think that creativity could be seen as a way of
using our "learned decompressing routines" to produce information.
What could be the purpose of this production? Your example of a
symphony seems to fit the idea: to circulate our emotions, an
often negleted part of our psyche.

Regards,
Sergio Navega.