Fundamental Compressionist Philosophy.

[Brendan Macmillan]

> > > Usually, we can present data in a format that reveals its
> > > hilliness - but then we ourselves have gone and done the 
> > > difficult bit.
> > OK, it is possible to call these things 'prior knowledge' and if
> > the term is used in that way, you are right.


> You must know nearly everything about the domain which you want to
> search by hill climbing etc.
> 
> Altering one bit (not knowing what it 'means') may beam you to the
> opposite side of the galaxy.
> 
> It must work completely context-free, or it won't work at all.

Nice examples, and very well said!


> I see heuristics work only on such global (off-context) parameters as
> -- amount of occupied resources
>   ("4G units is worse than 2.5G units")
> -- training set (rule) reproduction fidelity
>   ("failed at 2 samples" is better than "failed at 200 samples")
> or the like.
A very nice perspective on the problem!  I realize these are just examples, but
another such one (similar to your first) is entropy.

> I accept that this is little - compared to the search space volume.

Sadly, yes  :.(



Cheers,
Brendan
PS: but to me the great riddle is how come we humans can do it, and how come
two heads (literally, two perspectives - eyes occupying a particular point in
space) *so* much better than one.

And then, the wonderful realization that no matter what common sense or wisdom
or authority holds, there can be fantastic insights and discovers right under
our noses...  I think the history of substring matching algorithms
is a great example of this effect.  They get really clever, in ways that are
really obvious (in hindsight).  Boyer Moore, Pratt etc (sorry, forgotten all
the names).
-- 
e:  bren at csse.monash.edu.au                         v:  +61 (3)  9905 1502
Email is checked daily                              Phone is rarely attended