Philosophical foundations

[Detlef Morgenstern]

Christmas Monologue on Inductive Compression
============================================

If I were a critical reader of my "Compressing Philosophy" article
(http://www.wco.com/~sanna/casc/archive/frm00056.html) I would make some
objections or have at least some questions. I will take the role of this
critical reader now and, at the same time, reply to some of the recent
postings (I will be off-line now for a couple of days, the last posting I
found in my mailbox before leaving office for Christmas was
http://www.wco.com/~sanna/casc/archive/frm00074.html). I will sort all this
thematically and quote minimally. All the recent contributions to the CasC
mail list, of course, influenced this thinking.

[When posting this contribution I found two more replies (most recent:
http://www.wco.com/~sanna/casc/archive/frm00076.html). I will take these
(including the Doctoral Thesis mentioned there) home and read at leisure. No
comments at the moment.]

-- Recursive Definition
It is scientific tradition to ban recursion from the definition of terms. My
structure definition in Compressing Philosophy is 100% recursive: Structure
is defined recursively by itself. I hold that this recursion (and similar
ones) must not be avoided, if we want to get nearer to the basics of
cognition. Our models of the world are the more efficient, the nearer they
are to reality. And the reality of "structure" is 100% recursive.

We can keep our distance from recursion in all the "applied" sciences which
are at the "differential equation stage" far from "the basics of the world".
Manoeuvring at a safe distance around recursion is not a good idea in
philosophy because it holds us at even this distance from insight. I am an
engineer and only amateur philosopher. But what we do here must have some
philosophical touch because we are investigating the basics of the basics.
To be a good engineer, one should be a good physicist. And to be a good
physicist, one should be a good philosopher.

There is one more argument in favour of recursion: We are trapped in
recursion already by the fact that we search the principle of operation of
some mystic structure applying just this structure's principle. This posting
on Inductive Compression is a result of this inductive compression. So why
not confess to this type of recursion?

-- Atomic Structure
Supposed, we accept recursion as a means of definition and simulation, the
recursive structure definition leads to this type of question: Is there some
"atomic" structure of which all the others are composed but which cannot be
dissociated further (or - if it will be dissociated - will lose the
properties of "structure")? A simple trick could be applying recursion
again: Atomic structure is such that if dissociate it we get atomic
structure again. In digital ("computer") models of "structure" we can find
the same phenomenon: There is a minimal complexity level below which we
can't go. And we can "simulate" atoms at this level of complexity applying
just these atoms. If we would let the fission go on, we could dissociate
further (down to the NAND gate), but "critical" properties would get lost.
It is because of this that I think "Bit" is already beneath the critical
threshold. Atomic binary structure must be something more complex to be able
to "simulate itself". If you allow "Bit" to be the atomic structure, you
must "interpret" it by something much more complex than "Bit". I suggest
introducing this as a measure: "Does it work without an interpreter which is
more complex then itself?"

No question, any universal model of computing must be able to simulate all
the others. Our (still) fictitious hardware can be (software) simulated by
whatever universal machine. The point is: There is a critical complexity
level of programming beyond which a machine gets unmanageable (the software
crisis). And the innovation we are looking for is a universal machine which
allows a "photographic" mapping of the world's (cross-link) structures into
the machine's program (see also below: "Structure Object").

-- (Re)Active Structure
Much of the CasC discussion seems to focus on compressing "passive"
structure. This what is subject to compression, seems to be some "dead"
matter which is re-arranged one way or the other. I hold that this must lead
to weak models. We are surrounded be a very (re)active world. At each moment
of time at each point in space, each instance of structure reacts to the
impact of all the other structure. If our model of the world tries to map
this picture in a large "simulator and its database" (the simulator being
the active instance and the database being the (passive) model of the
world), we will fail. The complexity of the simulator ("software") must be
higher than the world itself, because all of the world's potential diversity
must be programmed in this simulator beforehand. And the complexity of the
brain of the programmer of this simulator must be even more complex, and so
on. I see only one way out of this vicious circle. Make your peace with
recursion. See structure as something (re)active. Build models which allow
emerging structures.

I think, part of the connectionist success against the symbolic approach
comes from connectionists connecting in their models **active** instances of
structure - (artificial) neurons, e.g., whereas the symbolic way operates on
dead symbols, which must be "interpreted" by some "higher intellect" which -
if modelled the symbolic way - must be of even higher complexity, and
modelled by a "higher intellect" and so on until an infinite complexity of
GOD. Sure, NNs cannot be the ultimate solution (see also below: "Cross-link
Categorisation"). They introduce unneeded structural complexity (structure
hierarchy cross links) what makes them hazardously unpredictable and
uninspectable. And what is the absolute KO criterion: You cannot program
them if you wish to do so. You must train them adjusting cross links
("weights") in a way which promises least possible "crosstalk" between
structures squeezed into these NNs. (Try to train a given NN to completely
"orthogonal" evidence!). But what we can learn from the connectionist point
of view is that we must accept (re)activity of structure to model it
efficiently! Neurons and our hypothetical Atomic Structure (see above) must
be kindred spirits. Evolution's local solution (the neuron) need not be the
globally optimal solution...

-- Structure Object
I suggest working on a class definition (in the OOP sense) for an object
"Structure" (or "Pattern") with properties ("integrated" data and methods)
discussed here and in "Compressing Philosophy". This will be a piece on its
own, and is worth to have its own thread later. To say one thing in advance:
OOP languages carry the same heavy backpack loaded with the massive wish to
not be recursive. You can instantiate objects at run-time in each of the
object oriented languages, but none of those languages I know allow you to
run-time create new classes ("structures") with emerging properties (data
and methods). New language concepts are needed which provide just this
creativity  which is so inherent to the world around us.

I wish to say I love running machines (and I made a lot of them). But from
time to time there seems to be some need for introspection far from
integrated circuits and the soldering-iron. I hope, the clearer and simpler
we define the philosophical basics for a new generation of machines, the
easier and quicker we will design them.

-- Inductive Compression
I hereby withdraw the term "Functional Compression" (see my "Compressing
Philosophy" article). "Inductive Compression" fits better, and is just the
way I see induction: It compresses observation evidence in a functionally
transparent way (maintaining functionality). I stick to the opinion that
this inductive compression will, in most cases, be an irreversible
compression. It is in a natural opposition to reversible compression, which
seeks to pack passive stuff in an way that it uses least space (losing
functionality) and can be unpacked later without "loss of structure".

We can, of course, store the before-compressive-induction-image of structure
to be able to "backtrack" if new evidence makes clear our "rule" (theory)
does not fit all known instances (Popper's beloved falsification). But I
would not make a whole theory of falsification of it. Our world is so full
of natural destruction that we are obliged to nurse even the weakest sign of
creation.

I see this relation between compression and induction: Induction is
accompanied by a compressing effect. This gives a weak hold in the search of
induction: "If it is compression, it might be induction". It must be made
clear that the nugget of inductive compression is buried in a heap of
non-inductive compression. What we have: Large heaps of compression gravel.
What must be done: Find a procedure which helps panning for (inductive)
compression nuggets.

-- Highly (Re)Active Input
I am glad I found this forum for getting the mental stimuli a consciousness
needs to move from the established frame of thought (Thanks to Chuck for
being our host!). For me, reading a book or following a discussion is not
merely getting new "raw" information. I gain most from information which is
tuned to my framework of thinking but partly contradicts to some of my
established assumptions ("falsification"). This type of "highly (re)active"
information triggers cascades of dialogue between me and myself. This is, I
think, due to the sensory network each human builds to catch the information
he needs most. (Re)Activity of information must always be measured against
the sensory network (filter, comparator) which is tuned to be triggered by
it. "Good" information works like a catalyst. I want to say this also in
favour of novels and all kind of fiction. It is not that one could gather
"real world" information from it. But a good novel (or drama) gives mental
stimuli comparable to this discussion forum. A lot of WHAT-IF is packed in
it, and the reader adjusts his framework of seeing the world by solving the
conflicts in himself. I do not deny fiction stories or drama. The point
which made me (like Sergio) read and watch less of them is, that
contemporary authors and producers seem to have forgotten what this real
mission of good fiction shall be. Or - the laws of competition dictate a
style which is far from what Shakespeare wanted to affect in his audience.

Sergio, I have read less fiction for the last decade, and my wife blames me
for doing so. Fiction is a good thing for people who do not have the chance
of "socially experimenting" with others' destinies. There must be a chance
for some WHAT-IF experiments for all the people. And good fiction can
prevent us from all these experiments being carried out in real life. Same
as for us: This forum prevents us from carrying out unneeded cognitive
investigations in real life. This transits me to the next paragraph.

-- Communication protocol
In a zest for creation, I welcome each type of activating input which moves
me from the deadlock I am (and I dare say most of AI) are caught in at the
moment. This may violate some rules of the century-old scientific
communication protocol. I remember a key article of Gerry Wolff. This
article vitalised me by its refreshing non-conformity in its search of
simplicity (G. Wolff: Computing, Cognition and Information Compression.
AICOM Vol. 6 Nr. 2 June 1993, p. 107 ff.). There seems to be some
aristocratic chic in being complex, and at lot of scientific work is
dedicated to elaborate complex creations. The inquisition struck
immediately. Gerry's article was criticised heavily for just this
non-conformity one issue later. I wish the CasC community be open-minded in
a way which does not suppress ideas for simply not fitting to the
century-old prototype of dispute. I will be grateful for every mental
stimulus - even on a level of (unproved or unprovable) vision or prophecy.
There will be enough time later to cut all the raw diamonds to brilliants.
First we must excavate them. One human life seems to be too short to get
acquainted with all the "established" cognition research results, find out
what is wrong and what is needed most urgently at the moment and, finally,
introspect oneself for adequate findings. Not to speak about language and
cultural barriers. These contributions of mine are meant to catalyse first
signs of solutions buried in one's sub-consciousness. It's not merely
playing abstract verbal games. If this type of communication is redundant to
this list, please, give me a hint.

-- Inductor daemon
When I go to bed I see to have a writing pad and pencil within reach. It was
frequently the case I woke up with a striking idea early in the morning.
When I went to sleep again because of the early hour, the only thing I could
remember when I woke up later at a "civilised" hour, was that I had had a
striking idea some hours earlier. But which one??? To me, from this follows,
our sub-consciousness daemon inductively compresses the conscious evidence
to new insight. This daemon is driven by search goals (tuneable filters?) we
define consciously, and this compression daemon works continuously, but at
low priority. When it finds some "matching compression" it signals about it
(software interrupt). The fact that it has its "good time" mostly at idle
time at night (consciousness logged off) can be traced back to the fact,
that in our conscious life there are much higher prioritised jobs than this
compression daemon (I would prefer the analogy "log-on time" (instead of
"up-time")). I could even imagine there are multiple instances of this
inductor daemon running simultaneously - each processing an "image" of the
person's knowledge. These daemon instances can be driven by different goals
(if different problems must be solved simultaneously), or several of them
can be given an identical goal (when a problem must be solved as quickly as
possible). It is even imaginable, there is some communication between them
so that they can deliver constraints to each other in heavily cross-linked
knowledge (see below: "Cross-link categorisation"). Maybe, IQ has something
to do with the number of these daemons running. And too many of them will
make one mad. ("Garbage collection" seems to be not the best analogy,
because GC's main goal is collecting "dropped structure". This is not what
we are primarily interested in.)

I see no problem with the compression daemon going to make me inoperative
during "log-on time" (consciousness being "logged on"). The inductor daemon
simply makes it's "image" copy of my conscious knowledge and processes this
image. I think this can be proven by the fact that idle time at day (waiting
for boarding time at the airport, summer holiday, Christmas??) is just the
same way creatively productive (I can physically sense bursts of insight at
these rare hours).

We know the cliché of the absent-minded professor. This happens, when the
priority of conscious jobs is purposely lowered to get the CPU free for
inductor daemons at log-on time.

Visionary creativity is killed by too many too highly prioritised "MUSTs".
It makes me sad to see how much human %CPU (including mine) is wasted on
artificially (commercially) highly prioritised jobs as, e.g.,
troubleshooting computer networks or teaching superfluous user interface
functionality. A far-sighted society must support some of their philosophers
for being "idle" most of the time so that the inductor daemons can do their
job. (Sponsors: Think about this.)

-- Cross-link Categorisation
In "Compressing Philosophy" I said that looking at a given (sub-)structure
of a given (super-)structure, we can hold two different views:
(a) The given (sub-)structure contributes to the given
    (super-)structure, and it contributes to only this one.
(b) The given (sub-)structure contributes to the given
    (super-)structure, but it may - at the same time - as well
    contribute to another (super-)structure.
View (a) simplifies the real world in a search of manageable models of the
world. Resulting models are easy to handle but far from reality. View (b) is
more realistic but difficult to handle in models. It results in heavily
cross-linked structure hierarchies. Induction in such a cross-linked
structure is confronted with a nearly infinite number of constraints: While
inductively compressing along one structure hierarchy one must not disorder
the functionality along all the other hierarchies. Or even harder: One must
simultaneously compress along several (or all) structure hierarchies in a
way that none of the functionalities will be disturbed. What seems to be a
significant problem is a great chance on the other hand: These "constraints"
strictly guide inductive compression. If productively applied, they
significantly narrow the (potentially/combinatorially) very large search
space.

The phenomenon that we can find various categorisations for one and the same
structure is related to the world's cross-link structure. Depending on which
of the numerous associations of a given (sub-)structure you focus, you will
see it being "member" of another (super-)structure (category). The apple
becomes an apple for us as soon as it is "inserted" in our model of the
world. This insertion procedure is complex as it affects several structure
(categorisation) hierarchies in the above sense. Some of the potential
cross-links must be dropped to keep the model manageable at all. Mostly,
those links will be dropped which seem to not significantly improve the
survival probability of the individual or the species. This leads to a very
biased model of the world which optimally helps finding food and comfort,
escaping from danger, protecting the young. If we want to understand more of
the world me must force ourselves to "forget" all these categories of
survival and comfort (at least for some time).

[If one consequently follows this style of thinking one will arrive at two
consequences:
(a) There are no "objective" structures in this world.
(b) Forget the comfortable life. Become a monk.
I am not sure this is what I want to experience for the rest of my life. I
am open to some compromise, "pseudo-objective" knowledge and distracting
("redundant") joy.]

Let us inspect Neural Networks once again, this time from the "structure
hierarchy cross-link" point of view. Some of the artificial NNs' problems
seem to stem from a misinterpretation of what mind maps into the neuronal
structure which can be seen when dissecting the brain. What we see **are**
heavily cross-linked structure hierarchies. To me, this suggests, that a
heavily cross-linked world is "photographically" mapped into this neuronal
structure. If one reduces this picture to NNs being simply "trainable
associative storage with categorisation abilities", one will throttle the
real power of these networks. (This is, of course, highly speculative, but I
hold it "looks nice".)

Let me repeat this: Connectionism is good in connecting (re)active
structural instances (it is significantly more powerful in this then the
symbolic approach). Connectionism emerged from the attempt to mimic neural
structures' behaviour. We need not restrict ourselves by sticking to these
original goals. Other structural atoms may be connected; they may be
connected in a way which is different from NNs, and the structural
philosophy of such nets needs not restrict itself to "tuning weights for
minimal cross-talk". To not forget the main goal: The structural atoms to be
found must comply with inductive compression. They fit the better, the
easier they can be dissociated, re-arranged (maintaining functionality),
re-associated. If I can see an automatism for inductive compression, it is
driven by the constraints reflecting structure hierarchy cross-links.

Is it redundant to point out that this, of course, will be embedded in a
natural parallelism (which we need so urgently)? A new generation of
intelligent hardware will be massively parallel, but - differently from
existing artificial NNs - provide scaleable inspectability. And, differently
from NNs, it can inductively compress even "orthogonal" structures
completely without crosstalk or loss. If we do not want to delegate a
certain cognitive job to induction, we can program it explicitly (see above:
"Structure Object"). And we can inspect (and verify!) the machine's software
at an arbitrary granularity level. The machine provides the same natural
coexistence of quick "sensomotoric" reactivity and complex cognition as the
brain does.

Pure fantasy? I think we can achieve this in the near future.
I wish the CasC community an insightful year 1999.
Do resist temptation of complexity!

Dresden, Christmas 1998