Saturday, April 24, 2010

Learned or Innate? Nature or Nurture? Speculations of how a mind can grasp on its own: animate/inanimate objects, face recognition, language...

Some possible explanations of mine. Български - като го преведа... :)


Babies early can make a difference between animate and inanimate objects etc...


They can, because the difference is easy to extract from the raw sensory inputs:

Inanimate objects are:

- Static. E.g. places which seem to drive "place cells" in mammal's brain are simpler stimuli than stimuli caused by "animate" objects.
- Dynamic, but mind can predict their future with a precision that it believes is "high enough". E.g. when one throws a ball, he doesn't know where exactly it will fall, but the ball wouldn't fly at all if an animate object didn't throw it.
- They change in parallel with the will, in pair with the motor commands.
- Linear (generally).
- Places are immovable inanimate objects. 

An object is a set of correlated stimuli.

"Animate" objects or subjects with "free will":

-  Their future cannot be predicted with high enough precision and one cannot predict precisely enough their behavior based only on observations. Sometimes they are totally unpredictable.
- They can start to do a sequence of actions on their own, a behavior. 
- They can react to your action, without being physically affected/touched.
- Mind can assume that there are internal states, which are unobservable. This is the inanimate object's "free will" or  its "state of mind", opinion etc., as opposed to predictable entities, whose state is observable.
- Non linear.
- It's hard to synchronize your will with the will of another "animate objects", or to do so you need to do a complex sequence of voluntary actions. 

Where "high enough" precision is a value that mind decides. 

Recognition of faces

A baby can easily spot that this is an entity that changes the most in his first perceptions, especially the eyes and mouth. I think the eyes - two spots that move in the same direction and  same speed can show to a developing mind that they are correlated. This also goes for nose and mouth, which all do translate in parallel. Entities which are correlated can be grouped together, this is "a face", i.e. this is a set of dynamic correlated elements. 

Actually, this can be one of the first coherent and stable set of visual correlations/patterns that the mind understands, and may serve as a basis for the future ones, which may explain existence of "face cells" or so. Faces are seen everywhere afterwards, and we use to see faces in inanimate objects.

Besides, these stimuli are early and further strongly conditioned with the baby's/one's own feelings, behavior and other agents' behaviors - these functions are exercised a lot.

Why baby's cry irritates us?  

Nativists: "Evolutionary encoded" etc., to call his mother to deal with it... 


Actually a very simple explanation of this is: because it reminds us our own cry, and these sound patterns were conditioned with our own early unpleasant experiences.

Why "mama" means a mother in so many languages?

This is also a simple one to explain - my theory is that "mama" requires one of the simplest possible articulations, if not the simplest - just the mouth opens and closes twice + breathing.* The sounds is recognizable even without a breathing, and even dogs can be trained to say "mama".

I think it is important that there are two open-close operations, because this may serve as a confirmation of the first syllable.

So I believe that the babies themselves  have coined the word for their mothers, then their parents started to use it themselves. 

*BTW, I've been speech synthesis designer and developer and have some projects, but not the time....


- formation of long term memory
- navigation
- head direction cells
- spatial view cells
- place cells

At least several or even all of these can be generalized. Places and navigation goes together. Places are long-term memories of static immovable inanimate objects (the agent has not experiences that these entities move).

Navigation, head-direction, spatial-view, place-cells - they all are a set of correlations found between motor and sensory information, and long-term memories, which are invoked by the ongoing motor and sensory patterns.

The static immovable inanimate objects (places)  change - they translate/rotate etc. - most rapidly in a correlation with head direction (position) and head movements. 

Navigation and spatial view are derived from all. 


This one needs work, but in short I would say that:

The language is a hierarchical redirection/abstraction/generalization/compression of sequences of sensory inputs and motor outputs, and records and predictions for both.

Chimps can communicate using sign languages, such as: 

Mirror neurons

They were found experimentally in rhesus monkeys (it's well known that monkeys use to imitate people) . Monkeys can imitate facial expressions and operations with hands such as picking.

There is a famous research of Miltzoff proving that newborn human babies at age of a few weeks are capable to imitate some facial expressions, like sticking out a tongue. They've never seen themselves and other experiments show that babies at that age are not capable to recognize different faces or to find a difference between a face without a nose or mouth or 2D vs 3D etc. (it smiles when see fake faces, by Fantz, R., 1966 trough R. Stamatov "Child Psychology")

This is interesting, I don't know how far the experiments have gone in the aspect of showing to the babies stimuli which are similar to faces.

This capability may require inborn links between basic image analysis (to find contrast/color change etc.)  and motor commands to facial and tongue muscles, I suspect that part of this may be done out of the neocortex, it can be of a very low resolution. Or there can be some pre-wired mirror neurons at the neocortex, related to this.

Other researches state that a one month baby vision has about 1/30 of the adult's resolution, which grows rapidly to about 1/4  at 8 months.

Regarding imitation of manual operations by monkeys, though,  I believe that this can be learned by mapping of similarities. Human arms and hands are visually similar to monkey's ones, they are "sticks" and "planes" which translate, rotate etc.

Unlike with the face, a monkey can see its own hands and other hands and can make a map between both. I don't know how it's done, but I think it can be done even without a pre-wired map.

I would quote my own teenage Theory of Mind and Universe (only Bulgarian yet:, where I concluded that mind is a Universal hierarchical simulator and predictor of virtual universes, where these virtual universes are derived from sensory inputs.

Further reading:

Google keywords and expressions: "mirror neurons", "Miltzoff", "chimps sign language" etc....

Keywords: Artificial General Intelligence, Seed AI, Development, Cognition, Developmental psychology, Neuroscience

Thursday, April 8, 2010

Universal Artificial Intelligence (Artificial General Intelligence/AGI) - Course Program, Plovdiv University | Универсален Изкуствен Разум - Програма

Todor Arnaudov (aka Tosh, The Twenkid)
researcher, software engineer, versatile artist, (wanna-be) entrepreneur

Age: 25. M.S. Software Engineering 2008 (Second average grade in alumni), B.S. Computer Science 2007 (Best average grade in the alumni) in Plovdiv University. Intern in RIILP Wolverhampton – Computational Linguistics/Natural Language Processing. A polymath, independent researcher and versatile artist, working on establishment of a private company for R&D in Artificial General Intelligence and other fields, as well as a professional film and multimedia studio.

More Info:                  Български  
An interview in a popular-science magazine:
Twenkid Research:
Todor Arnaudov's Researches Blog: http://artificial-mind.blogspot.comTwenkid Studio (mostly in Bulgarian yet):

Universal Artificial Intelligence(Artificial General Intelligence/AGI)

Program as of April 8-th 2010

It will be taught in Faculty of Mathematics and Informatics of Plovdiv University, Bulgaria, starting from 9-th of April.

The course is appropriate for undergraduate students who want to work in the very advanced interdisciplinary field of the Artificial General Intelligence/Universal Artificial Intelligence, also known as Strong AI. It would be interesting also for those who just want to learn what are the current theories of intelligence and how the brains and mind work, according to current theories and speculations. The course tries to give the direction of how to design a thinking machine, eventually self-improving, and gives the directions for deeper study and research.

This course is original and it seems it's one of the first worldwide, if not the first* as systematic and interdisciplinary one, including:

- Futurology (Singularity, Transhumanism)
- Neuroscience  - How Brain works.
- Developmental Psychology
- Behaviorism/Reinforcement learning in living beings/humans and machines

Even Love as a form of reinforcement learning “bug”. ;) The course gives insights and theories of intelligence/Universe and meta evolution – higher level principles that cause and drive Universe evolution - which seem not to be popular yet – Boris Kazachenko's and mine. 

I will give students also some ideas of how and why to do practical research and engineering and high-tech entrepreneurship, something I'm trying to start-up myself and have some experience as an engineer (employee) in a fabless semiconductor start-up. The course is appropriate for all undergraduate students, because the special preliminary knowledge is taught in the course. A graduate student barely have an  advantage over a second year student.

Lectures will be in Bulgarian, but knowledge of English is highly recommended, especially if you wish to deepen your knowledge in the field. The exam will be a test.

Probably; eventually I will translate the course to English. Maybe teaching it abroad if reach to that point.

Universal Artificial Intelligence

  1. Introduction to the course. Why should one does research, especially so brave? PhD – Pro & Con. Is it possible to do a private research/science and scientific business? What is a Start-up? True stories and advices.

  2. What is Artificial Intelligence? Is it possible? Criticism and scepticists – Searle's Chinese room. Summary of the classical AI and why it failed. Review of the results and successes in current Weak AI (specialized/narrow AI) – why do they have limitations. Cognitive Science and cognitive architectures.

  3. What is Universal Artificial Intelligence (UAI, AGI, Strong AI). Technological Singularity and Singularity Institute. Transhumanism. Expected computing power of human brains. Attempts for literal simulation of brains. Universality paradox of the brains. Ethical issues, related to AGI.

  4. Complexity and Theory of Information. Probability Theory – statistical (empirical) probability. Chaos Theory. Systems Theory. Emergent functions and behavior. Universe as a computer – digital physics. Algorithmic Probability. Kolmogorov's Complexity and Minimum Message Length. Occam's Razor.

  5. Design of Mammalian and Human brains.

  6. What drives mammals and humans? Behaviorism – Reinforcement Learning as a Universal learning method. Supervised learning and imitation.

  7. Developmental Psychology (Child Psychology). Stages in cognitive development. First language acquisition.

  8. Machine Learning. Markov's Chains. Hidden Markov Models (HMM). Bayes Networks. Hierarchical Bayes' Networks and Hierarchical HMM. Principles of the algorithms of Viterbi and Baum-Welch (Expectation-Maximization).

  9. Tests for human and machine intelligence. Definition of a reinforcement learning Intelligent agent, according to Marcus Hutter. Beauty according to Jurgen Schmidhuber and Todor Arnaudov. Schmidhuber's Godel Machine.

  10. Theory of Mind and Universe by Todor Arnaudov – mind as a hierarchical system of interacting universal simulators of virtual universes. Universe as a computer and trends in Universe evolution. 

  11. Theory of Intelligence by Jeff Hawkins. How the neocortex in mammals and human works. Memory-Prediction Framework and Hierarchical Temporal Memory.

  12. Theory of Intelligence and Universe Meta-evolution by Boris Kazachenko. Cognition: hierarchically selective pattern recognition & projection. Мeta-evolution as Abstraction of a conserved core from its environment, via mediation of impacts & responses by increasingly differentiated adaptive interface hierarchy. Boris' prize for ideas.

  13. Summary of the principles of intelligence according to Todor Arnaudov, Marcus Hutter, Jeff Hawkins, Boris Kazachenko and others. Prediction, hierarchy, bottom-up development, scalability, complexity growth. Seed AI. Sample architecture of an AGI/UAI agent.

  14. Ben Goertzel's cognitive architecture Novamente.

  15. Other researchers and private companies in the field of AGI. Bibliography, references, links and directions. Discussion and Questions.

  16. Exam.

    See you in class! ;)

    * I know about the summer school of Ben Goertzel, Hugo De Garis and their colleagues in Xiamen 2009, but according to their program, it lacks significant parts, e.g. Jeff Hawkins is not even mentioned.
    ** I spotted one interesting interdisciplinary course, called “Psychology, Anthropology, Neuroscience, and Zoology 619” or “Biology of Mind” by Deric Bownds. It's taught to graduate biology students, so it's too much into Biology and lacks technical/mathematical parts.

    Универсален Изкуствен Разум - програма на курса на Тодор Арнаудов - Тош във ФМИ на ПУ "Паисий Хилендарски" | AGI - course program (see link)

    Information about the course in English  (Contains more details) 
    Интервю за сп. „Обекти“ 10/2009:

    Универсален изкуствен разум
    Програма към 8/4/2010

    Курсът е авторски и оригинален, ще се води във ФМИ на ПУ „Паисий Хилендарски“ от 9-ти април 2010 и е предназначен за студенти, които искат в бъдеще да се занимават с авангардната област Универсален изкуствен разум (в миналото известно като Силното направление на Изкуствения интелект) или да разберат за най-новите теории на разума и търсенията на отговор на въпроса как да се създаде самоусъвършенстваща се разумна машина. Курсът е един от първите по рода си в света, а така систематизирано изложение на теориите на разума, заедно с необходимите основополагащи знания е оригинално и може би е първото и най-изчерпателно до момента (виж статията за курса на англ. какви са подобията English and some more about the course).

    Курсът е интердисциплинарен и включва освен информатични и математически въпроси и супер авангардната материя - Теории на разума и мета-еволюция на Вселената, - още Футурология (Технологична сингулярност, Трансхуманизъм), Функционална невроанатомия на мозъка на бозайниците и човека, Психология на развитието (Детска психология),  Когнитивна психология; Бихевиоризъм (учене с подсилване), като тези знания се включват в контекста на теориите на разума. Например какво са емоциите и любовта като когнитивни и поведенчески процеси. 

    Ще се дадат още информация и идеи за извършване на практически насочени научни изследвания и за високотехнологично предприемачество.
    Курсът е подходящ за всички, защото специалните предварителни знания се преподават в самия курс. Лекциите са на български, но е препоръчително ползването на английски език, особено ако имате намерение да задълбочите познанията си. Изпитът е тест. 

    Универсален изкуствен разум

    1. Въведение в курса. Защо човек да се занимава с научни изследвания, още повече смели? Докторантура – за и против. Може ли да се прави частна наука и научен бизнес? Какво е „start-up”? Истински истории и съвети.

    2. Какво е изкуствен разум? Възможен ли е? Критици – Китайската стая на Сийръл. Обзор на класическия изкуствен интелект (AI) и защо се провали. Преглед на резултатите на съвременното слабо направление в ИИ (специализиран ИИ) - защо имат ограничена перспектива. Когнитивна наука и когнитивни архитектури.

    3. Какво е Универсален изкуствен разум? (AGI, Силно направление) Технологична сингулярност (Singularity). Сингулярен институт (Singularity Institute). Трансхуманизъм. Предполагаема изчислителна мощ на човешкия мозък и парадоксалната универсалност на мозъка. Опити за буквална симулация на мозъка. Етични въпроси, свързани с УИР.

    4. Сложност и теория на информацията. Теория на вероятностите - статистическа (емпирична) вероятност. Теория на хаоса. Теория на системите (Systems Theory). Възникваща функционалност и поведение. Вселената като компютър – цифрова физика. Алгоритмична вероятност. Сложност на Колмогоров и минимална дължина на съобщението. Бръснач на Окам.

    5. Архитектура и работа на мозъка на бозайниците и човека.

    6. Двигатели на човешкото поведение и поведението на бозайниците. Бихевиоризъм - учене с подсилване (reinforcement learning) – универсален метод за обучение. Учене с учител и подражаване.

    7. Психология на развитието (детска психология). Етапи на когнитивното развитие. Научаване на първия език (language acquisition).

    8. Машинно обучение. Вериги на Марков. Скрити модели на Марков. Мрежи на Бейс. Йерархични скрити модели на Марков. Принципи на алгоритмите на Витерби и Баум-Уелч (Expectation-Maximization).

    9. Тестове за човешка и машинна интелигентност. Дефиниция на интелигентен агент на Маркус Хутер, основана на учене с подсилване и взаимодействие със средата. Изследванията на Юрген Шмидхубер за интересността и красотата и машината на Гьодел. Красотата в теорията на Т. Арнаудов.

    10. Теория за разума и Вселената на Тодор Арнаудов - разумът като йерархична система от взаимодействащи си универсални симулатори на въображаеми вселени. Вселената компютър.

    11. Теория на разума на Джеф Хокинс. Механизъм на работа на кората на мозъка на бозайниците. Предсказване, основано на паметта - Memory-Prediction Framework. Йерархична темпорална памет - Hierarchical Temporal Memory.

    12. Теория на разума и метаеволюцията на Вселената на Борис Казаченко.

    13. Обобщение на принципите на разума според Тодор Арнаудов, Маркус Хутер, Джеф Хокинс, Борис Казаченко и др. Предсказване на бъдещето, йерархичност, развитие отдолу-нагоре, мащабируемост, натрупване на сложност. Зародиши на разум. Примерна архитектура на универсален ИР.

    14. Когнитивната архитектура „Новаменте“ на Бен Гьорцел.

    15. Други учени. Фирми, които се занимават с УИР. Библиография - насоки. Дискусия, въпроси.

    16. Тест