Monday, May 24, 2010

Sunday, May 23, 2010

The NLP Researchers cannot understand language. Computers could. Speech recognition plateau, or What's wrong with Natural Language Processing? Part 3

Thanks to my friend O.G.I, for sharing the link below, about the plateau in Speech Recognition software.

Rest in Peas: The Unrecognized Death of Speech Recognition

If you check this out (specialized NLP training is not needed to see)

What's wrong with Natural Language Processing? Part 2. Static, Specific, High-level, Not-evolving...

And this simple generalisation: Language is a hierarchical redirection/abstraction/generalization/compression of sequences of [multi-modal] sensory inputs and motor outputs, and records and predictions for both.  (me) 

You'll get what causes the plateau - why NLP, parsing, speech recognition or whatever would stay at their dead-end forever if it doesn't change radically.

I especially enjoy this one:

"...To some, these developments are no surprise. In 1986, Terry Winograd and Fernando Flores audaciously concluded that “computers cannot understand language.” In their book, Understanding Computers and Cognition, the authors argued from biology and philosophy rather than producing a proof like Einstein’s demonstration that nothing can travel faster than light...."

So silly.  The same goes for any similar sentence of retired AI-niks, because they're actually saying this:

Computers cannot understand language [or think], because computers do exactly what they, those retired AI researchers, program them to do. Besides, machines lack free-will, also you know - Goedel incompleteness, quantum-mechanical blah-blah-blah etc. 

However, this implies that computers just execute their programmers' instructions,  therefore it is not the computers who cannot understand language, it is their incapable programmers.


It is the Programmers and old-fashioned AI-niks doing NLP who cannot understand language, not the computers.

NLP programs are playing with words in dictionaries, while mind is playing with multi-modal pre-processed raw sensory inputs. An AGI is needed to make speech recognition correct, this is what should be worked on.

"...So not everyone agreed. Bill Gates described it as “a complete horseshit book” shortly after it appeared, but acknowledged that “it has to be read,” a wise amendment given the balance of evidence from the last quarter century."

Hmmm, Bill is cool!  :) 

http://www.sadanduseless.com/2010/05/steve-jobs-vs-bill-gates/

Saturday, May 22, 2010

Юрген Шмидхубер - красотата, любопитството, машината на Гьодел. Курс по Универсален изкуствен разум, Други учени и насоки за изследванe.(Част 2 от ..)

Юрген Шмидхубер е един от водещите учени в УИР, твърди че неговата амбиция е да създаде съвършения учен - машината на Гьодел, която решава всякакви задачи. Занимава се с рекурентни невронни мрежи, т.нар. LSTM - Long Short Term Memory, които имат някои от възможностите на HTM като предсказване и йерархичност. Пише за универсални алгоритми за самообучение; за красотата и изкуството и връзката им с принципите на универсалния разум, подобно на това което си говорихме в седмата седмица, но в математически издържана форма. Може би ще поговорим още малко за красотата, интересността и любопиството в осмата (на 28-ми май).

В сайта на Ю. Шмидхубер: http://www.idsia.ch/~juergen/ може да откриете много материал за четене, препоръчвам ви да гледате презентацията му на Singularity Summit 2009. Освен интересна, е много забавна и смешна.  :)
Алгоритмични принцип на любопитството, интересността и красотата (на английски)

Jürgen Schmidhuber at Singularity Summit 2009 - Compression Progress: The Algorithmic Principle Behind Curiosity and Creativity from Singularity Institute on Vimeo.




Keywords: Jurgen, Juergen, Schmidhuber, Creativity, Beauty, AGI, UAI,

Tuesday, May 18, 2010

Курс по Универсален изкуствен разум, Други учени и насоки за изследванe. (Част 1 от ...)

Здравейте, колеги,

Записах малко допълнителни материали за по-любознателните от вас, за пропуснатите часове. До края на курса или след него може да има още.

Видеолекциите от мен тук са около 24 - 25 минути.

Ключови думи: Хуго Де Гарис, Маркус Хутер, Шмидхубер, Йордан Янков, СФЕРА, Кибертрон, Лятно училище по УИР в Зямен (Xiamen), Бен Гьорцел, FPGA, УИР, AGI

1.avi
6:05 Хуго де Гарис, Хутер, Де Гарис, Лятно училище в Зямен 1, AGIRI, Adaptive AI - SmartAction, ...

Тук може да чуете впечатляващите записи на реални диалози между хора и агента на Adaptive AI "Smart Action": Демота на Smart Action

Забележете как клиентите прекъсват агента и говорят през него, и той ги чува и се поправя. Показва разпознаване на свързана реч, разбиране на естествен език и разговорът е на човешко ниво, но в ограничена област - поръчки.

2.avi
3:26 Кибертрон, Йордан Янков; аналогов разум; Бен Гьорцел, Хаотична логика; блогове

3.avi
1:20 Лятно училище по УИР, Китай

4.avi
3:30 СФЕРА, Йордан Янков, Кибертрон

5.avi
9:45 McGovern Institute, FPGA

...

Вижте и видео от DARPA Urban Challenge - състезанието за коли-роботи, които се управляват съвсем сами в градски условия. :) В свързаните видеа има от по-ранен етап на проекта - DARPA Grand Challenge, където колите карат в извънградски условия и без трафик.





Един линк от колегата Красимир Василев. Забавно видео за способността на Азимо освен да ходи, да посяга към предмети като бебе, да ги разпознава, да прави прости обобщения и малко да разпознава реч.





На любознателните препоръчвам канала на Тим Тайлър в Ютюб:

http://www.youtube.com/user/tmtyler

Отшелническият му външен вид е леко притеснителен :) и говорът малко монотонен (някои записи имат и текстова версия), но Тайлър е сериозен и задълбочен мислител и разглежда най-различни интересни въпроси за бъдещето, живота и разума, а във видео записите включва интересни откъси от презентации на големите учени от универсалния изкуствен разум.




Saturday, May 1, 2010

Thalamic Nuclei - primary causes for Mirror Neurons? |Human Face - Important Aspect of Evolution | Cingulate Cortex | Nature or Nurture


a face in action

Regarding mirror neurons and early facial expression imitation, another hypothesis. Continues from:  Learned or Innate? Nature or Nurture? Speculations of how a mind can grasp on its own: animate/inanimate objects, face recognition, language...



Emotional facial expressions are innate, I remember a many decades old research on understanding facial expressions around the world, which shows that we're compatible at emotions expression level, no matter of culture and the level of society development. I think this was proven thousands of times by Cinema, and it matches with the hypothesis that emotions are driven by older parts of the brain: Cingulate Cortex --> Fornix --> Thalamus --> Hippocampus --> Neocortex.

The Neocortex eventually learns to control face as well, and good actors do it well, however, being an actor myself, I would say that good acting involves feeling the emotions of the character, "living a part", which goes below neocortex. Indeed, I believe this is easier, because it happens partially subconsciously and automatically.

Perhaps neocortex "calls" or better "recalls" and "reruns" complex functions from the lower parts - memories of already felt emotions or better - situations when emotions were felt. On the other hand, bad actors probably do not feel, because cannot recall emotional memories so well. Bad actors are trying consciously /mechanically to pull-up/pull-down facial muscles which makes their faces to look unrealistic and unconvincing (like if their characters felt nothing or something inappropriate). Perhaps, because conscious cannot do control precisely in parallel so many muscles, and maybe because these muscles are too strongly linked to their thalamic nuclei.

I would suggest here: it's better not to smile if you're not happy, rather than do a "false smile".



Facial Hardware

Our face has a dedicated cranial nerve like the other sensory and motor head-"interfaces", and like the rest, except olfaction, but including optic nerve, facial nerve passes through a thalamic nuclei in the thalamus:







Going up, Thalamus has projections to the neocortex and back, so eventually it can turn into "magic neurons" up there.

Boris Kazachenko recently suggested in a comment here: Meta Evolution: fitness beyond reproduction

BK: "I think there have been minor genetic changes in humans that produced a major increase in intelligence, mostly through the growth & folding of neocortex"

I believe that the face might be one of the important aspects of human evolution, as well.  
 
Making a digital networks analogy (see Physical layer

Face - physical level interface

If brains and intelligence have evolved to cope with more complex social interactions in the high level of processing, face should have taken a part of the evolution of the "Physical Level Interface" and the thalamic nuclei are a middle layer, doing primary decoding of physical layer signals, sorting, redirecting. I suspect they might/should be correlated at DNA level, because, growing one thing requires growing a counter-part, all muscles or receptors etc. need to have appropriate nerves that eventually reach the Central Nervous System, and I guess that evolving parts in pathways inside the CNS may also involve all subregions, if they develop together prenatally. (I'm not yet that much deep into embryology, though.)

Flexible physical layer interface allows telling others clearly and visually  how do you feel, so they can react appropriately and/or learn using it as a sign/conditioned stimulus. I guess this should be related also to the grow of the importance of vision in human senses.

Basic Emotions and Mammals 

All mammals have the basic emotions, lower animals also should have at least part of theirs (such as: Fear, Panic, Lust, Search; even the Octopus seems to have an elaborated brain and "Play" system on its own).  There's a funny research showing that rats laugh when being tickled...

However, most mammals, e.g.  cats or dogs display just a portion of their emotions using face. Expression of Rage (human also growls and clenches its teeth) and Fear/Panic? (eyes wide open)  seem to be similar, and I think they both are amongst the evolutionary older emotions.

When we do say that a cat or a dog has "a sad face", though I suspect it's rather a blind visual similarity we spot (like finding faces in a fish or insect), a form of anthropomorphizing, than real. Cats and dogs faces are not that flexible and expressive as ours - as Alice says in Wonderland: "Cats can't smile". Well, I don't know.  :)


Whitney was a sad little kitty, starring in a photo story of mine.
Photos - (C) Todor Arnaudov, 2006 - "The Ghost" photo story

Big Apes 




I haven't studied well great apes facial expressions to speak seriously. Of course they do use faces for expressing emotions, but I suspect their face is not as versatile as human's, the visual contrast between features and background is worse and they lack eyebrows, which are clues for human emotions.

However mirror neurons and facial expressions imitation were found as early as rhesus monkeys, so this line may started long ago.

Conclusion

Overall, I guess that mirror neurons might be projections or related to projections from facial and optic thalamic nuclei, maybe related to a primary integration of both. I suspect that part of the integration between face and optic, dealing with imitation of facial expressions, can be done as early as the thalamus itself, by sort of low resolution processing; I haven't studied internal thalamus anatomy yet, though (is it studied/understood?). 

I would conclude, that if one day I could do study myself how exactly a baby imitates faces, I would try to find spatial/contrast etc. resolution thresholds when these reactions first appear. E.g. a baby boy may put his tongue out if he sees a quick-enough change in contrast in a wide spot of the visual field, not only in a mouth - it might be a high-contrast line moving quickly vertically anywhere etc. I don't know if such tests had been done already.
 

Appendix 1: Talents
I think there's no doubt that there are innate predispositions/talents for arts - drawing, music, dancing, also acting. So far I thought that it's related to details about  personal cortical architecture, maybe differences in the speed of learning - ease/speed/durability of synaptogenesis/neuroplasticity. Now I have an additional guess: Thalamic Nuclei may also play a significant part in talents. E.g. people who can't feel the rhythm can barely learn dancing, while the others start dancing even without being taught. At a low level dancing seems to be related to basic prediction of sound patterns, linked with syncrhonized motions. Cingulate Cortex, related to emotions, has projections into motor-pathway as well, which may add to explanation why dancing is an emotional activity. I suspect that acting talent may have some of its roots in Cingulate Cortex as well. 

Finally, good dancers and any talented people may have an advantage in that lower-level preprocessing part. The cerebellum of course should be also a "suspect" whenever there is fine motor-coordination, and it has much more neurons than the neocortex and supposingly vast computing power. I can't speak about cerebellum with supporting data yet, but my first guess is that some people have faster "processors" - cerebellum has longer-lasting and faster neuroplasticy capabilities, maybe wider range connections.

Appendix 2: Lower parts of the brain and prediction - prediction mechanisms below neocortex 

Many if not all AGI researchers would agree that prediction and compression are amongst the main basic  keywords when defining the substance of intelligence. I assume that classical conditioning, which seems to exist even in fish*, can be assumed as a primitive form of prediction of one stimuli, associated with another. The emotions or the basic behavioral drives are also forms of prediction,  predicted behavioral patterns, appropriate for particular situations. This is there in all animals, in insects as well - genetically predicted/precomputed patterns.

I think a nice example is the ants "conditioning" to care for their "farms" of greenflies, which ants keep to milk juice. Ants are not eating the greenflies, but are just milking them, and even further - they are protecting their animals from enemies such as lady-birds...

It is known that thalamus does make primary decisions about what sensory information should pass up to neocortex and what should be processed faster through lower pathways.

I suspect (it may be a well known fact), that Thalamus and Cingulate Cortex  do have capabilities to predict or drive complex motor patterns like the Neocortex, but of course their models are much vague, short and specific - emotional patterns, face expressions, body language(?).

I suspect that it may be connected with part of the body-language as well, which seems to be uniform in many cultures. (Not all gestures, though - e.g.  the middle finger evidently passes through a visual analogy...  ;)  )


...To be continued:

Beauty, music and more.