Wednesday, December 19, 2012

Five Principles of Developmental Robotics... Matches of Todor Arnaudov's works from 2003-2004 to a 2006/2009 academic paper ... Yet another one :)

This post is regarding the paper: 

Five Basic Principles of Developmental Robotics, 2006,

posted Sep 2, 2008 10:30 AM by Brian Tanner   [ updated Sep 2, 2008 10:48 AM ]
http://prw06.rl-community.org/posters/five-basic-principles-of-developmental-robotics by Alexander Stoytchev, Department of Computer Science, Iowa State University

Also in a 2009's IEEE edition, with extensions based on Stoychev's PhD thesis:

" ... Some Basic Principles of Developmental Robotics
Stoytchev, A.
Page(s): 122-130
Digital Object Identifier 10.1109/TAMD.2009.2029989

Abstract: This paper formulates five basic principles of developmental robotics. These principles are formulated based on some of the recurring themes in the developmental learning literature and in the author's own research. The five principles follow logically from the verification principle (postulated by Richard Sutton) which is assumed to be self-evident. This paper also gives an example of how these principles can be applied to the problem of autonomous tool use in robot "
...

Yet another academic paper that I found recently, which is published years after the works of mine and is based on a bunch of other works,  and in the academic eyes those are "new contributions".
Sure, they are novel - in the community to which they are presented and their subculture, and in the specific way they are presented.

Not that I claim plagiarism or something, the case is rather "great minds think alike" ;)), I recommend the paper.

The differences are in the social position and status, background, resources, support - $$$, peers, access to literature and conferences, - and experience needed to made those claims.


Those seem significantly in my favor, as a poor teenager in high school... :)) 

Of course it doesn't sound plausible that an institutionalized researcher, who's well fit into the mainstream system to go read "crank's" works on the Internet - "who are they" - if a paper was not published in a conference or a journal (costs $$$ to go to a conference etc.), it's like it didn't exist. I discovered this back as a freshman... That particular paper is grounded on other papers from the "system", including author's own. Also who's gonna read "crank's" (high-school students') works published in Bulgarian - non-Bulgarians are highly unlikely to have ever known of my existence up to later years.

Well, in this particular case, though, there's a chance that the author of that paper has known about my works anyway, because he's a fellow Bulgarian, and there was a forum where a few of us, "the cranks" and other enthusiasts gathered for a while in 2004 - the forum of the so called "Project Kibertron" for a generally intelligent humanoid robot

Let's get to the point:

Autonmous Mental Development


I found this in the "subculture" called "Autonmous Mental Development (AMD) - or Developmental Robotics (certainly the more popular term). That reminded me of those "split brain" academia, there exist subcultures, groups, which don't know each other good enough and may produce similar results, or results which are of help to the others. For example... Well - later about that. ;P
...
  • The Verification Principle (credited Sutton 2001)
  • The Principle of Embodiment
  • The Principle of Subjectivity
  • The Principle of Grounding
  • The Principle of Gradual Exploration
Those principles are explicitly stated in one way or another in "points" or claims about how a universal mind/human mind is supposed to work according to my "Teenage Theory of Mind and Universe", with its pique in the works in 2003 - early 2004.

Some translations of a part of my classical works

http://research.twenkid.com/agi_english/

Other not translated:
http://eim.hit.bg/razum
http://www.oocities.org/eimworld/  (Windows-1251)

If I am to state the precise points and matches, I'd write a more formal paper later, but let me give just a few short examples:

For example, the Sutton's "verification principle" and all the rest are obvious for a sensori-motor generalizing self-improving system, one of the explicit statements in my works is the "match", the way "truth" is defined and found.  (I'm "late" to Sutton here, but I haven't heard of him, as that community haven't heard of me.)

From Universe and Mind 3 (2003), Universe and Mind 4 (2004):
...
50. The truth is a match – if the knowledge (or
confidence, belief, persuasion [, desire])
matches something that is perceived somewhere
else later, then the new one is true, compared to
the old one; on the other hand, if the new one is
different, it's “a lie” (false) or it becomes truth
and the old truth turns into full or partial false,
depending on how the new truth is different
from the old one. The more the newly evaluated
for “truth” input piece of knowledge [pattern]
matches a piece of knowledge [pattern] from the
memories of mind, the more it's “truth” and
“actual”, according to mind. Therefore,
determining a “truth” is a determination of
difference between past and wanted present.
(“Wanted” was missing in the Part 3 writing,
added here in Part 4).
[“Truth” in Bulgarian is “Istina” (истина)]
Interestingly, in Serbian “isto” means “same” -
it has morphological association to “same”,
because the statement that a given feature is
“truth” means also that:

TRUTH: The feature [specifics, detail] that is
being evaluated matches the pattern/template - it
is the same as in the pattern, at a given
resolution of perception. (*That's a definition of
mine.)

Stoychev mentions the philosophy school of the logical neopositivists as an origin of the "verification" principle. My opponent and co-author of my 2002 epistolary work "Theory of Universal Predetermination II"* (Universe and Mind, Part 2) - Angel Grancharov - who was a professional philosopher having also University level teaching experience and an author of many books, was "insulting me" for being a "positivist" and explaining me how "flawed" that philosophy is, and that it's not really a philosophy. I didn't know that there's such a school, I heard about that school for the first time in those emails.

The quote above is about the subjectivity, embodiment, grounding and gradual exploration, all in one sentence: "...The more the newly evaluated for “truth” input piece of knowledge [pattern] matches a piece of knowledge [pattern] from the memories of mind, the more it's “truth” and “actual”, according to mind", "at a given resolution of perception".

"Grounding" is related also to the notion of "reality" as the "lowest level of virtual universe", and statements that for any system there's one lowest level, from where generalization starts.

Etc.

...The series will continue with a few explicit marks/comparisons of the matches between my classical works to Jeff Hawkins's "On Intelligence" and the HTM, which are published after mine.

See also: http://artificial-mind.blogspot.com/2012/12/compression-and-beauty-matches-between.html

Monday, December 10, 2012

AGI Researchers: Unite! Thinking Machines are Coming Really Soon.. Act NOW!

In the late years, and increasingly with time, I fail to see any meaningful conceptual or technological reason for thinking machines not to exist already... I've claimed it before and I claim it again.*

I dare to say that one of the reason why AGI has not been implemented yet are my own distractions and other problems which prevented me to work. The time is ticking away, though, I see crucial breakthroughs next year or in the worst case - in the next few years, and I'm switching to working mode, so look forward to see them...

We must act, unite, incorporate and work hard NOW if we are to be one of the pioneers.

Otherwise - the rich and heavy armored batteries of the rich institutionalized researchers, funded by our taxes, DARPA, and big rich companies will push it and we will stay marginalized and get forgotten, even though we were ahead of them (see my previous post about my hypotheses from 2003 and a 2011 academic work)

Do you know IIT?

Just look how big they are:

http://www.iit.it/en/research/departments/robotics-brain-and-cognitive-sciences.html

http://www.iit.it/en/rbcs-labs/action-and-perception-lab/perceptual-and-motor-bases-of-prediction.html

Italy is really pushing this field. I've marked the "danger" from the EU IM-CLEVER project some time ago, a colleague once told me how clumsy it's it in the real live, 24-blade server? or something, but they got correct research direction.

IIT is another big "danger", I think they are the creators of the iCube robot, they have a huge exhaustive research program and correct general direction which will reach to the goal. And one even stronger reason to "be afraid" if you don't join their army :) - they are apparently equipped with the best technology and have plenty of human power.

There are at least several other very solid and rich institutes, which are touching the "right directions" in the UK and the USA, I'll mark them later (have to re-find the links, but I got "frightened" from their power .. :) )

Thinking Machines Thinkers - Unite!



Monday, December 3, 2012

Compression and Beauty: matches between a work by Todor Arnaudov from year 2003, and 2010-2011 new acаdеmic contributions... - comments on "Musical Beauty and Information Compression: Complex to the Ear, but Simple to the Mind" - Part I

I wrote this as a letter to the author several days ago, but it's not personal anyway, and what I've published the original reflections of the topics I mention here 10 years ago. I haven't got any answer so far [as of 9/12/2012], not that I am optimistic about getting such by the author, but I will answer with a more solid response.

[I got a response on 17/12, the author has been busy.]

I'm warming up for a new iteration over the topic and already have insights and challenges to pose [writing in progress], on second read, I see the paper has some profoundly confused concepts. In general my view matches Schmidhuber's view for cognitive beauty as compression/compression progress and balance of predictability, but there are some crucial concepts and parameters that need to be specified and are obvious in the case of music. I've shared some general thoughts in discussions on the AGIRI AGI list also. It will be continued more formally and technically in a paper, with some details that I deliberately omitted while writing this email.


(...)

I'm Todor Arnaudov, I'd say a veteran AGI/SIGI* interdisciplinary researcher,  an extreme polymath and universal artist, I do constantly progress my art, knowledge and skills in all kinds of sciences (soft- and hard-), technical fields and arts - both creative and performance, covering all sensori-motor domains. 

I'm writing you regarding your work:  "Musical beauty and information compression: Complex to the ear but simple to the mind?"that I encountered recently and enjoyed reading. I have heard earlier some short mentions of the topic from pop-sci news feeds, perhaps related to your paper (but it might have been about Schmidhuber's, I don't remember), but I've found and read your paper as late as very recently.

First of all, congratulations for presenting those ideas together!

I agree with the many claims and hypotheses, with some comments, which however I may present in a more formal form, regarding for example measurements and deviations, and interpretation of the results in regard to neuroscience and sensori-motor generalizing hierarchies (may view on general intelligence).

I write you, because I happen to have made related, similar or in cases the same hypotheses and speculations regarding general intelligence (all domains) and creativity starting in the early 2000s, mostly between mid-2002 and early 2004, which represented my Theory of intelligence and creativity (originality); it's also metaphysical and digital-physics/philosophy.

My theory is about prediction/compression with increasing range or precision, the balance (too predictable is boring, too unpredictable is "random") and also compression progress (progress of predictions' range and precision) as basic cognitive and aesthetic drives in all domains; the cognitive "uglyness" as cognitive overload and beauty/intelligence as finding simple matches/transformations.

I do also discriminate between cognitive and physical beauty (physical one is actually just pleasure), what you call  "2) stimulating the receiver through historical association" in my view is again related to physical beauty, experienced in the past, and associated with cognitive stimuli, which makes the cognitive stimuli to invoke those physical memories. Both types are often confused due to the mess in the brain reward systems (dopamine-and-other neurotransmitters and the intrinsic cognitive reward or prediction/compression), and people use to call "beautiful" stimuli which are just pleasant, i.e. produce a release of dopamine, oxytocin, endorphines etc.

I also have claimed and held that the science and arts are "the same thing"** beginning from those a decade old works, to me it has been obvious, because I've been always doing both. I don't mean Schmidhuber's "low complexity art", but "high complexity" art, i.e. normal classical art, good old fashioned drawing, artistic photography, writing, music, acting, filmmaking - all kinds of stuff, genres, forms, lengths.
I'll save the more detailed comparison and explanation of my claims and theory of general intelligence and creativity and why my works are not known or acknowledged in the mainstream academic media so far in a dedicated paper (for example, one reason is probably because I was 18-19 years old, they were published in Bulgarian in non-academic media). 

I don't know do you care at all about that, but I'll cite the following short examples of some matches:

...This appreciation "....rests on our ability to discern patterns in the notes and rhythms and use them to make predictions about what will come next. When our anticipations are violated, we experience tension; when the expectation is met, we have a pleasurable sense of release[4]. ... from
Ball P: Harmonious minds: the hunt for universal music.
New Scientist 2010., 2759: OpenURL

I've written a very similar expression [for example] in my 2003 interdisciplinary philosophical-metaphysical-AGI-creativity... work, originally called "Conception of Universal Predetermination, Part 3", originally published in an e-zine called "The Sacred Computer", written in Bulgarian, in the middle 2003. The site had a mirror on Geocities, which is now "frozen" in several copies and can be verified, for example:http://www.oocities.org/eimworld/3/25/pred-3.htm  (in Bulgarian, Windows-1251 encoding, though) 

In English it sounds like that:  (Complete work, translated in English: http://research.twenkid.com/agi_english/Teenage_Theory_of_Universe_and_Mind_3.pdf , in this copy in the email I've done some additional minor stylistic/language corrections of mistakes in the translation, that I spotted now )

[BEGIN]
45. (...)
Why do we like to dance?

Let me try to make a speculation - maybe rough, but I guess – probable.*

Dancing is a rhythmical motion of the body  -
output stream of information to muscles -
which is in some kind of harmony
(synchrony) with an input stream – music.

Changes on the input – hearing – impact the
output – movements. 

Rhythmical means predictable. After you know
the rhythm – time period after which particular
changes in sound would happen (will hear
drums, a guitar; particular tone) – you can
predict the changes [the events] in music, you
can predict the future of the music you are
listening to.

Every muscle [muscular] action is a consequence of an
operation in brain, executed earlier. The output
is in the same time an input, because in order to
flex or to relax a muscle, mind makes particular
neurons in brain to “oscillate” (to send pulses) in
a particular fashion, i.e. the output is also an
input.

The better the synchrony/harmony between
input and output pulses, the more the total
level of the pulses gets amplified and makes
us feel pleasure.  (...)

** The rhythm should be within the mental
capabilities of mind to perceive and discriminate
it, for example we would hardly feel a rhythm of
3000 beats per minute. [250 – 300 is already
really fast]

(...)



And for example this one, it's about images, but my theory is general and the claim goes for any kind of inputs:
64. (...) 
Abstract artists' pictures, where there are no

pieces of knowledge, except the most primitive

for an image do not contain any meaningful

information [or "information for mind", i.e. data

that have patterns and can be compressed].

(Such most primitive pieces/patterns are e.g. a

point, spot, line, seasow line; polygons which
mind does not associate with known objects;
without 3-rd dimension).

There is meaningful information in images
where mind can recognize and call particular
objects, parts of objects and their
coordinates/relations to each other.

The meaningful idea ("разумна представа") is
described/defined with much less information
[data] than the image from which it was
deduced. [Compression and selection.]

...

E.g. this image has a little knowledge, because it
can be easily defined with a cycle in mind,
where the changes of the bar of the building are
small.
Photographers use to call such pictures "boring"
- they bring a little amount of information,
which is unpredictable using the precedent
information [from the picture or whatever
artifact being analyzed].  [This image is very
highly compressible.]
Well done artistic photograph must be balanced: it
must bring enough information about the subject
on the picture, be clear enough – it should be
easy to understand what's on the picture, - to
have a "complete" idea; also, the picture
shouldn't be overloaded with details which are
distracting attention (are "irritating") and make
our sight to roam around the details, searching
for a meaning in the picture.
Artistic photography is based on abstraction and
emphasizing the essential [information]. It's
like that also in classical visual art, especially in
graphics and caricature.

A person creating any kind of art is displaying
how the image of what he's creating is stored in
his memory and what features of reality are
memorized. For example the earliest children
drawings use just lines - therefore [perhaps]


images are stored as "lines"*; lines bring the
biggest possible [amoung of] information, because they are
abrupt change between the colour of the paper
and the pencil or whatever tool. The youngest
mind remembers [maybe implies: recognizes]
only the very essential parts of the perceived
reality.

Humans don't remember lots of the details in the
images. Maybe that's why pictures with lots of
details are "irritating". Human mind usually
removes the most of the minute details, when it
remembers the image as a description of pieces
of knowledge, in order to make remembering
easier.

[* Lines [curves] are also the easiest to draw, poor motorskills are also involved.]

It's easier to fill memory using a piece of
information [a pattern] to be multiplied with a
gradual change, known from the beginning; and
to be able to predict, by parts of the image, with a
higher probability, what's on other parts of the
image. 

[That's easier, than to hold a bitmap copy.]

Human hardly can remember and perceive too
"plentiful information servings" such as noise:

NOISE: A sequence of characters/symbols,
where every following has equal probability to
appear and the value of the following characters
is fully unpredictable for the one who's trying to
predict it [i.e. the *generator* may actually
*know* all the symbols and they might be
pseudo-random for him, but the *evaluator*
doesn't know them and can't predict them].

Mind works with big amount of redundancy in
information, in order to predict the future inputs
by the past easier and with higher certainty
[Aesthetics is not that simple to define in a few sentences and is subjective. There's "organic/smooth" patterns that seem to make perceptions generally more pleasant than highly compressible high contrast images; also there are high level associations, that pictures bring to us and may make us judge them emotionally etc.; "animate" objects and conflicts like in drama may engage mind to think/predict (interestingness)...]

That parallel part in the original, below the BW picture with the list of words, which I've missed to translate in that edition says:
This picture [of the old house] from the town of Kalofer, ( © Tosh 1994)
occupies just 10% more space on my hard drive* 
[than the picture of the other building], however it brings much more meaningful information.
One can recognize:
- a grass field
- bushes - branches, leaves
- a tree - stem, branches, leaves
- house - roof with tiles; second floor with a terrace, having a rail; other windows on the second floor (each of them is a "piece of knowledge in the picture)
- pillars
- windows on the first floor
- pavement
- reflections on the pavement
- water covering the pavement [causing the reflections]
- that water implies rain
- children (bottom left)
- a bench (in the middle) built of two supports and a board [actually two benches]

[END]

Discussion:
We can apply the same explicit labeling to the picture of the rectangular building (my high-school), but then it would be very short:

- a
 sky- a rectangular building- windows (the same type)- several floors- repeating bar of windows, put in perspective, but it looks "flat" because is on the same plane, there are no big perpendicular planes like in the other picture, which looks in 3D
It's more "boring" than the old house, because one bar can be multiplied with application of perspective throughout the picture. In other words, many parts of the image are "linearly dependent", and the compression progress ceases to 0 very quickly - for mind, the compression is much higher than for jpeg, it's too high. 
Now I see that in fact this 10% covers approximately the difference of the pictures' resolutions (the BW pictures is slightly bigger), which emphasizes the 
differences in "meaningful knowledge".

Well, that all is inspiring me to do another, more elaborate and detailed iteration.

Best Regards, Todor "Tosh" Arnaudov 
*SIGI - Self-Improving General Intelligence, my term for "Seed-AI"
** A T-shirt caption, designed by me, that says "Science = Art + Fun" :)
http://artificial-mind.blogspot.com/2009/07/blog-post.html