Showing posts sorted by relevance for query What's wrong with natural language processing. Sort by date Show all posts
Showing posts sorted by relevance for query What's wrong with natural language processing. Sort by date Show all posts

Tuesday, January 7, 2014

// // Leave a Comment

What's Wrong with NLP, Part 4 - an Opinion from a Paper at IJCAI-13 and a Comment by Todor

Why can't computer understand me.. 

On our best behaviour - Hector J. Levesque

 http://www.cs.toronto.edu/~hector/Papers/ijcai-13-paper.pdf  -

I'd generalize what the author tries to say: yes, the machine does need *imagination* (sensory memories and spaces, see many, for example [7]) and to incrementally learn and play with it. That subsystem is required!* I do agree also that many AI-ers, especially in NLP community didn't understand or accept that, this paper suggests that there's no common progress in the acceptance.

Todor:

Some of the reasons are simply the academic and people's urge to produce papers and "results" quick, to demonstrate that you "do something". Papers, papers, papers - who cares about the real progress.

I guess some have understood that very well, the "common sense" problem is cited from decades, it is obvious, yet in the NLP they've been insisting to push a bunch of words with no relation to the real world and then ask "why it can't do proper word-sense disambiguation, given only a corpus?"

Also, IMO it's  the researchers in NLP etc. who don't understand language understanding [4],  rather than the poor computers, which, as many AI-use to say "do what they are told to".

Language teaching of a machine should be done incrementally with sensory-motor feedback and interaction, like teaching a child and not in sensory-less batch mode with a huge corpus with zero interaction, no coordinate space, just a bunch of words, because:

Todor: Natural language is a hierarchical redirection/abstraction/generalization/compression of sequences of multi-modal sensory inputs and motor outputs, and records and predictions for both.

A mere corpus has no imagination, even if it's 1 Terrawords. A small corpus, built by systematic interaction and mapped to sensory-motor system is intelligent and can explore and learn further on its own. Such as - the toddlers.

I support also the criticism of the Turing Test. Right, it is a test for fooling people - not a test of intelligence, and as I myself claimed back in 2001 (see [5] "Човекът и мислещата машина..." below) - a true, honest and too smart or quick AGI will quickly FAIL the test, because its output is too complex, too quick, too deep, she doesn't have some required memories (shuld lie for her childhood etc.) etc.

Finally, I will save some of my thoughts from myself...

Credits to Aaron for sharing the link!

* More about that when I complete the first milestones of mine...

See also: (And many others...)


[1] Faults in Turing Test and Lovelace Test. Introduction of Educational Test (2007)

[2] What's wrong with Natural Language Processing? Part 2. Static, Specific, High-level, Not-evolving... (2009)

[3] What's wrong with Natural Language Processing? Part I (shorter) (2009)

[4] Part 3: The NLP Researchers cannot understand language. Computers could. Speech recognition plateau, or What's wrong with Natural Language Processing? (2010)

[5] Анализ на смисъла на изречение въз основа на базата знания на действаща мислеща машина. Мисли за смисъла и изкуствената мисъл (2004)

[6] Човекът и мислещата машина. Анализ на възможността (...), 2001 г.

[7] Embodiment is just coordinate spaces, interactivity and modalities - not a mystery (2011)



Read More

Wednesday, July 3, 2019

// // Leave a Comment

Cognitive Science's Failure to Become an Interdisciplinary Field - the Multi-Interdisciplinary Blindness

Discussion of mine regarding the paper:

"Perspective | What happened to cognitive science?,  10 June 2019


https://www.nature.com/articles/s41562-019-0626-2?fbclid=IwAR2mMzO4qzIKINXT2BUcMZAS1ZI4PONd04SHQeF7FLgvH5VTc1pwR4DcLow

Available on Github: https://github.com/rdgao/WH2CogSci/blob/master/nunezetal_final.pdf?fbclid=IwAR17HIosUS-7EKdFT4a--SesVuwb3aPp-1a0yE4Rbk_q8io8w5S6nFVrWvY

https://www.facebook.com/groups/RealAGI/permalink/1230556097152988/

IMO a big share of the problem lays in the whole researchers's and overall intellectual direction in the academic circles (and power-and-profit driven societies, modern slavery). It is narrow knowledge and world view, specialization is promoted, ones who obey and execute instructions of their superiors grow the ladder and become leaders, doing the same. The creative, wide-minded and really original ones are not leaders of the research.*

That is related to multi-interdisciplinary blindness, related to insufficient working memory capacity and faculties for understanding and representing the inputs generally enough so that one can encompass the concepts from different domains and contexts and think of them together.

BK calls it too simply "depth of structure".



* In the past there were exceptions, such as Alan Kay
** That survey paper reminds me of my cycle "What's wrong with Natural Language Processing" some 10 years ago, because to me NLP/Computational Linguistics should have been a part of the AGI, not what they were.

** Sorry for the sick formatting, I had to write it in external editor etc., this one is annoying, but not now.

See elaborate related discussions:

#1

Circa 2009-2010 - series of 3 "perspective" articles



What's wrong with NLP, part I:

http://artificial-mind.blogspot.com/2009/02/whats-wrong-with-natural-language.html

Monday, March 23, 2009


http://artificial-mind.blogspot.com/2009/03/whats-wrong-with-natural-language.html



Note: now in NLP there are impressive results in NLG (generation), BERT etc. with such "mindless" vector representations, using current methods of machine learning, convolutions, "transformers" etc. however  it probably more or less emulates virtual sensory-motor interactions - by traversing and comparing huge corpora and how different texts/segments (mappings of sensory records) map and relate to each other, what's reasonable in what context. It is more advanced than as it was in the earlier simple frequency-based representations and inverse-frequency - frequency/probability of a token in current document, compared to average in the other documents etc.

#2 Friday, January 1, 2010

I will Create a Thinking Machine that will Self-Improve 

An Interview with Todor, "Obekty" magazine, issue November-December 2009

http://artificial-mind.blogspot.com/2010/01/i-will-create-thinking-machine-that.html   


- Where does the researchers' efforts should be focused in order to achieve Artificial General Intelligence (AGI)?

First of all, research should be lead by interdisciplinary scientists, who are seeing the big picture. You need to have a grasp of Cognitive Science, Neuroscience, Mathematics, Computer Science, Philosophy etc. Also, creation of an AGI is not just a scientific task, this is an enormous engineering enterprise – from the beginning you should think of the global architecture and for universal methods at low-level which would lead to accumulation of intelligence during the operation of the system. Neuroscience gives us some clues, neocortex is “the star” in this field. For example, it's known that the neurons are arranged in sort of unified modules – cortical columns. They are built by 6 layers of neurons, different layers have some specific types of neurons. All the neurons in one column are tightly connected vertically, between layers, and are processing a piece of sensory information together, as a whole. All types of sensory information – visual, auditory, touch etc. is processed by the interaction between unified modules, which are often called “the building blocks of intelligence”.

- If you believe that it is possible for us to build an AGI [Since you do believe], why we didn't manage to do it yet? What are the obstacles?

I believe that the biggest obstacle today is time. There are different forecasts, 10-20-50 years to enhance and specify current theoretical models before they actually run, or before computers get fast and powerful enough. I am an optimist that we can go there in less than 10 years, at least to basic models, and I'm sure that once we understand how to make it, the available computing power would be enough. One of the big obstacles in the past maybe was the research direction – top-down instead of bottom-up, but this was inevitable due to the limited computing
(...)


#3

Tuesday, August 27, 2013

Issues on the AGIRI AGI email list and the AGI community in general - an Analysis

https://artificial-mind.blogspot.com/2013/08/issues-on-agiri-agi-email-list-and-agi.html
"- Multi-intra-inter-domain blindness/insufficiency [see other posts from Todor on the list] - people claim they are working on understanding "general" intelligence, but they clearly do not display traits of general/multi-inter-disciplinary interests and skills.


[ Note: Cognitive science, psychology, AI, NLP/Computational Linguistics, Mathematics, Robotics … – sorry, that's not general! General is being adept, fluent and talented in music, dance, visual arts (all), acting, story-telling, and all kinds of arts; in sociology, philosophy; sports … (…) ... + all of the typical ones + as many as possible other hard sciences and soft sciences and languages, and that is supposed to come from fluency in learning and mastering anything. That's something typical researchers definitely lack, which impedes their thinking about general intelligence. ](...) "

Discussion:
http://artificial-mind.blogspot.com/2014_08_05_archive.html



Tuesday, August 5, 2014


#4

The Super Science of Philosophy and Some Confusions About it - continuation of the discussion on the "Strong Artificial Intelligence" thread at G+

Read More

Wednesday, February 4, 2009

// // Leave a Comment

What's wrong with Natural Language Processing?

A short philosophical essay...

NLP?
Do you know about Machine Translation - either rule-based or Statistical, the Lexical Databases like WordNet? Statistical Parsers, POS-Taggers, Parallel Corpora with 1 billion words. Machine Learning, N-grams, Hidden-Markov-Models. And Blah-blah-blah...

What's wrong with NLP?

The main issue I find in the paradigm of NLP today is, I think, embedded in the mindset of the researchers in general, and in the research tradition. IMHO, usually researchers are mathematicians and too nerdy persons.

Usually researchers are mathematicians - not artists, not creative enough and not brave enough to dive into too deep imaginative directions.

Science do also pushes the typical researcher not to invent too much. If he does use his imagination too much and does create "imaginary structures", he might be unable to prove their creation and existence "scientifically" enough to the other members of the sect. He would not be acknowledged etc.

Too nerdy, too mathematical, too obvious and directly "provable" by the raw output data.

As an example of this I would mention Statistical MT.

These distributions work to a certain degree, but this is a trick. It is not really original. There are pure mathematical parameters, which are pretty obvious.

Sorry, but isn't Statistical Machine Translation a mathematical trick?
What are scientific basis of it?

Flat parameters, which can be derived by the distributions of words.

- Take a problem.
- Divide it it into "items" with which you can do something.
- Take the items which you can do something with, and find the combinations which make a difference.
- Take the items and do combinations in order to see what is the difference.

OK, research is a process of exhausting anyway.

Research is exhausting anyway, but if you do not invent structures which are outside and above the obvious ones, you cannot reach too far. However, the relations and combinations which are obvious or easily derived by the raw data without auxiliary "unreal" structures are easier to prove and to be acknowledged in the sect; pardon, I mean science.

This reminds me Quantum Mechanics and the hidden variables. I don't now about physics, but in NLP definitely hidden variables do exist.

Sorry, but in NLP there are hidden variables, out of the text.


I would mention also, that for any creative writer or poet, words are much more than distributions. Writing of a novel is an imaginative process, you are building a world with actors, with laws etc.. Then you simulate that world and record in text what happens.

Creative writing is imagination and simulation, not probability distribution of words.


Actually your dreams about the world you describe are very much more detailed, because the mind is not based on words.

And those Statistical techniques (at least that I know) are flat, because they lack imagination and will, they do not simulate worlds.

Do standard statistical techniques do simulate worlds? Don't think so.

Mind needs to build a virtual world and fit the text to the virtual world, which is simulated during understanding. And I believe that the simulated world in mind is not built by NP, VP etc. NP, VP etc. can be mapped to some aspects of the "simulator" or to modify that simulator in a particular way, but I don't think they are the simulator.

CON: There is no simulator, brain is very slow etc.

(Mostly) memory-based reconstruction is also a simulation. Those 100-level or so of neurons in one "pass" might be pretty enough, if the task is subdivided in a smart way.

Does the "simulator" in mind is built by NP, VP etc.? I don't think so.*

Rather mind can fit its simulator to work with these structures if it needs/wishes.

A prove: babies do not know about words, and in early age when they know something about words, their mind is not based on words anyway. They are pretty intelligent without NP-VP etc., later on they fit NP-VP to something else. I guess so.



To be continued:

- Reflect on explanation of the reason Statistical translation give useful results. ?
- Finally start doing some kind of "simulators" yourself and stop just talking philosophycally! :))


Continues:

What's wrong with Natural Language Processing? Part 2. Static, Specific, High-level, Not-evolving...


* Edit, 1.2.2024: "Do the "simulator"... --> Does the ...; Continue: ... --> Continues

Read More

Saturday, March 20, 2010

// // 1 comment

NLP stands for Natural Language Processing and not Neuro-Linguistic Programming | НЛП означава Обработка на естествен език, а не ...

The more popular NLP is a half-scientific or pseudoscientific psychological methodology for the so called "social engineering".

In my opinion this is an immoral methodology of  how to manipulate and deceive people, especially about making sex or selling something. There are courses about how to pick-up girls and make them believe you love them while you just want sex; how to make people buy your shit and persuade them how brilliant it is etc.

This kind of pseudo NLP sucks, and people who try to use this "methodology" are hard to be trusted. I've met several ones, such as a seller of stolen goods and an abusive network marketing distributor.

Not that the scientific NLP is fine yet - What's wrong with Natural Language Processing? Part 2. Static, Specific, High-level, Not-evolving...

But at least it can be taken as a science.  :)

...

НЛП означава Обработка на естествен език. Има и друго - "невролингвистично програмиране", което според мен представлява неморална методология за манипулация и заблуда - може да си платите за курс на който да ви научат как да заблуждавате хората, да ги накарате да си мислят, че сте им приятел (а да имате единствено користни цели) и т.н.. Веднъж усетиш ли такъв човек, трудно можеш да му имаш доверие след това...

Не че откриват кой знае какво, според мен концентрират добре известни и "работещи" методи на свалячи, ласкатели и други. Ако сте достатъчно умен човек и имате усет за хората, тези "мъдрости" на които ще ви научат, ще си ги знаете от собствен опит. 

Не че Обработката на естествен език е цвете - What's wrong with Natural Language Processing? Part 2. Static, Specific, High-level, Not-evolving... (Къде греши Обработката на естествен език), но това е друга история.


Read More

Sunday, May 23, 2010

// // Leave a Comment

The NLP Researchers cannot understand language. Computers could. Speech recognition plateau, or What's wrong with Natural Language Processing? Part 3

Thanks to my friend O.G.I, for sharing the link below, about the plateau in Speech Recognition software.

Rest in Peas: The Unrecognized Death of Speech Recognition

If you check this out (specialized NLP training is not needed to see)

What's wrong with Natural Language Processing? Part 2. Static, Specific, High-level, Not-evolving...

And this simple generalisation: Language is a hierarchical redirection/abstraction/generalization/compression of sequences of [multi-modal] sensory inputs and motor outputs, and records and predictions for both.  (me) 

You'll get what causes the plateau - why NLP, parsing, speech recognition or whatever would stay at their dead-end forever if it doesn't change radically.

I especially enjoy this one:

"...To some, these developments are no surprise. In 1986, Terry Winograd and Fernando Flores audaciously concluded that “computers cannot understand language.” In their book, Understanding Computers and Cognition, the authors argued from biology and philosophy rather than producing a proof like Einstein’s demonstration that nothing can travel faster than light...."

So silly.  The same goes for any similar sentence of retired AI-niks, because they're actually saying this:

Computers cannot understand language [or think], because computers do exactly what they, those retired AI researchers, program them to do. Besides, machines lack free-will, also you know - Goedel incompleteness, quantum-mechanical blah-blah-blah etc. 

However, this implies that computers just execute their programmers' instructions,  therefore it is not the computers who cannot understand language, it is their incapable programmers.


It is the Programmers and old-fashioned AI-niks doing NLP who cannot understand language, not the computers.

NLP programs are playing with words in dictionaries, while mind is playing with multi-modal pre-processed raw sensory inputs. An AGI is needed to make speech recognition correct, this is what should be worked on.

"...So not everyone agreed. Bill Gates described it as “a complete horseshit book” shortly after it appeared, but acknowledged that “it has to be read,” a wise amendment given the balance of evidence from the last quarter century."

Hmmm, Bill is cool!  :) 

http://www.sadanduseless.com/2010/05/steve-jobs-vs-bill-gates/

Read More

Monday, March 23, 2009

// // Leave a Comment

What's wrong with Natural Language Processing? Part 2. Static, Specific, High-level, Not-evolving...

By Todor Arnaudov

Independent Researcher - Twenkid Research
Independent Filmmaker - Twenkid Studio
ASIC Engineer - (as of March 2009)

MS Software Engineering, Plovdiv University, 2008
BS Computer Science, Plovdiv University, 2007
Internship in RIILP, University of Wolverhampton, 2007



What's wrong with Natural Language Processing?
Part 2. Static, Specific, High-level and many more!



In brief: Static, Specific, High-level, "Short-chain of intelligent operations", Lack of evolution of the systems, Not enough experimental approach and research for new paradigms

Part I: http://artificial-mind.blogspot.com/2009/02/whats-wrong-with-natural-language.html


Overall, I'm disappointed by the state-of-the-art of NLP. I think the approaches are too shallow, too obvious, too static... And many more...


[Critics] Who the hell are you to be disappointed from "the state-of-the-art"? Who cares? When did you finish school?!


And I think after taking into account all of the "wrong" parts, it is not surprizing that the progress is slow.

We are walking on almost the same road.

The road does go to somewhere, but I think - not right there where me and other are aiming to. OK, call us yet "dreamers".

[Critics] Crazy mad "scientists"!

If the approach doesn't change, I can't see how the "normal" researchers would go there . The same stuff which is not going to evolution is being done over and over again...

...

In the beginning, let me note that I believe that any research or research direction, including mainstream NLP research is supposed to be a mirror, indication, derivation, reduction, output or whatever of operation of mind. This implies that there is *something right* in the paradigm, the paradigm maps some aspects of researchers' mind and is supposed to solve some problems. Science proves this "scientifically" - OK, no doubt that, what I call "mainstream NLP" does solve many problems...

However, one of the basic mistakes I see, is that the abstractions of these problems are at so high-level and so dispersed, that it is impossible to "ignite" the engine to run on its own.
Say, it is not an engine, but just a tool. A tool for another mind to "push buttons" and see the output, not an engine or an organism to be born and to develop.


*** What mainstream NLP is doing wrongly, in my opinion ***


-- Trying to do a reverse engineering of language, starting from words, text and very abstract and not based on "good physics" linguistic constructs.

[Critics] What the hell is "Good physics?"

"Good physics" is a basis that allows you to build an "engine", "ignite" it and make it running on its own. :) (Arnaudov, 2009)

I believe text is one of the reductions of the operation of mind, which is dynamic, based on multiple inputs simulation of multiple virtual universes at different levels of details/abstraction. Some of them are at very low level (say sets of images, sets of "video", sets of relations between sensory inputs). Words are pointers to very low-level models in mind.

Language, viewed as a bunch of static text, could not contain enough information to rebuild-back intelligence in the low level. Low level of mind is massively reduced and "cleaned", when converting to text, mind is reconstructing the missing part using its internal rich models.

-- The focus of the models is output - models are based too closely on text itself and on structures which are derived from the text itself and the output words.

Again - too obvious and cheap. Text is a reduction of mind in action. Language is not just a flat bunch of words with tags and a boring set of numbers for distribution and frequency.

Mind operates with images, relations and dynamics of virtual universes/systems that it simulates, and then reduces representation of this simulation into text, which actually is a system of pointers to the items and rules, which represent the real structures in mind. The structures in mind are dynamic, they are not 1 billion word corpora.

[Critics:] And how those "dynamic models" supposed to be modeled? You're stupid theorist! You don't define concepts you use! You're doing nothing!

I did define some of them, but years ago and OK, in pieces yet only in Bulgarian.
Anyway this reminds me a "practical" AI professor of mine I spoke with a year ago. He didn't make a difference between McCarthy and Minsky (who cares anyway?) and haven't even heard of Numenta or other advanced research directions, like simulation of neocortex columns...

I am a theorist, because I've been busy doing stuff for a living, besides theorizing. In fact I haven't been really theorizing since my teenage years. I'm tired of being only theorist, so be prepared!


-- Lack of freedom in researcher's imagination and lack of will to test more imaginative, complex, growing and dynamic models than obvious flat static relations between "scientifically linguistically proven" structures.

Researchers are walking on the same old "paved" road. Freedom? Imagination? Quotations rule this world - not freedom of imagination.

You want to do a radically new experiment? Get lost! If your research was not based on your supervizor's, on the best known researcher in the field so far etc. - then you're not a scientist and your research is not scientific.

Young researchers are trained that way. Fine - knowledge, history, respect, methodology, etc...
That's good. However, then they become experienced researchers, they quote their own papers, which are quoting the previous ones, which are accepted to be in the right direction.

Of course, revolutionary research is hard to be done in such conditions.

[Critics:] What revolutionary research are you talking about? You stupid dreamer! Come down and step on Earth! Learn the real NLP! Join the mainstream and you will be forgiven!

Who told you that I don't learn it?Thanks. I may take this option, but let me first try to do it differently.

-- Systems are not general, they are created to solve specific abstract problems, defined in terms of words or other very abstract concepts. That's like dealing with the symptoms, not with the cause of the "desease".

-- Models are not only specific, but static.

Machine learning, Naive bayesian etc. - they seem to be models in development. But what are they actually learning?

Probabilities between some set of "symbols" inside a set.

That's fine, but what is done with those probabilities between symbols later?
What those models want to do later with these probabilities?
Can they want to do, and do anything at all?

This is too flat. Words are pointless without doing something else with them - humans use words to make somebody do, imagine or feel something.

The purpose of "probabilities" in real natural language is to cause something different than words.

-- Lack of will and intentions in models. Lack of effectors. Lack of general feedback loops for self-improvement.

[Critics] Will and intentions? "Desire is irrelevant. They are machines!" And Computational Linguistics is not exactly Artificial Intelligence! Don't mix the fields!

Mind needs will and effectors. Otherwise it is not mind, but a mere number cruncher. And a pure number-cruncher architecture would hardly have capabilities of mind.

[Critics] Oh... Intelligent Agents. Bravo! You reinvented the wheel!

Thanks! You're so sweet!

Here we are - another weak part of mainstream research.

All that dividing of everything, instead of integration.

This division of everything is connected with the tendency of mainstream researchers to solve specific dispersed abstract problems, but not to search a solution for general problems which can solve many specific problems in an elegant way. I suggest you check out Boris Kazachenko's site.

What are the pieces and the mechanisms that can build intelligence up? The general mechanisms and evolved system will be capable to solve all anaphora-resolutions, word-sense-disambiguation, multiword expression recognition and whatever...

Let's search for an engine, not for tools.



-- Lack of continuous development and accumulation of experience. Lack of evolution.

Of course. Models are so much hand-crafted and specific, like tricks. Meet some of my unfortunate disappointments in Computational Creativity:

MEXICA: A Computer Model of Creativity in Writing - "Creativity" Disappointment again

Faults in Turing Test and Lovelace Test. Introduction of Educational Test. (Arnaudov, 2007; suggestion of educational test and analysis of works by Bringsjord, S., Ferrucci, D)


These systems (MEXICA and BRUTUS.1) may seem very good at first sight, but after you look under the hood, you will see how much they are based on word-by-word direction and how weak are they in creative generation of text.

These systems really are not "computationally creative", it is implied by the simplicity of the models.

A nice model is growing on its own by communicating with intelligent environment. You shouldn't be capable to understand it in details after it grow. If you are capable to understand the details and follow them - your model is too simple, it is too "young" or both.

If you use to code everything line by line and direct it... If you can predict everything by hand or in an obvious way... Sorry, but this is - at least - very boring!

[Critics] Theorist!

Thank you! :))


-- The following generation of researchers base their work on the work of the previous ones.

Again... Sure, this is science. It should be like that. Of course the state-of-the-art should be known. And one should use the knowledge, accumulated in the past.

However, I think the efforts spent on this is should be dosed.

Instead of imaging and testing new approaches, most of the time typical NLP researchers do study bibles with models which are proven to lead to very painful and slow progress.

Or the bibles consist of solutions which researchers are supposed to implement.


Or researchers are spending long-long time, building hand-crafted tools and databases, which cannot evolve on their own, later on.

The same path for so many years...


[Critics] Slow progress? Parsing, "Marsing", Syntax 45.4%, 67.4%, POS-Tagging: 96.4%, ...

So...? This progress doesn't lead to intelligent machines.
Those numbers do not map to a genuine general intelligence, but to production of tools.

Hand-crafted tricks with text... If you call this "Natural language processing" - OK, it's great.
This is useful to a certain degree and for particular class of problems.

Yes, mainstream NLP at the moment:

- Is useful.
- Solve some abstract specific problems by heuristics.
- It works to some degree for "intelligent" tasks, because of course language do maps mind.

However, the mainstream still does not lead to a chain of intelligent operations, there are not loops and cumulative development.


-- The lenght of the chain of inter-related intelligent operations in NLP today is very short. This is related to the lack of will and general goals of the systems. These systems are "push-the-button-and-fetch-the-result".

-- Swallowing of a huge corpus of 1 billion of words or so and a computation of statistical dependencies between tokens is not the way mind works.

!!! Mind learns step by step, modeling simpler constructs/situations/dynamics/models before reaching to more complex.
!!! Temporal relations of the input with different complexity is important.
!!! Mind usually uses many sensory inputs while learning. Very important.
!!! Mind has will, uses feedback and can actively and evolutionary test and improve correctness and effectiveness of its operation, including natural-language-related.


I suggest:

1. Hollistic approach - the goal is building an operational mind with long chain of intelligent operations, not completion of a table with values 94.55% 96.5% 90.4% and a long list with quotes in the end of a paper.

2. System must have will and effectors and must evolve. And saying "to evolve", I am not talking about "genetic algorithms", I'm talking about increasing of complexity by fetching "complexity" from the environment and pushing it into the system.

In other words:

Methodology for building very complex systems:

-- Don't do everything by hand, design something which is capable to design parts of it on its own
.

3. If doing reverse engineering - let it be reverse engineering in the beginning of mind development and reverse engineering of the evolution of mind. Not reverse engineering of text.

4. Straight Engineering. Experimental engineering. Experimenting with designs of systems which evolve and fetch complexity.


[Final Critics] Who the hell are you, crazy ignorant stupid kid? "50 years of research of the brightest, talented, etc!" And you think you will change the world! Crazy!

If you are walking on the wrong way, you can't reach the right place, even if you were "the brightests" etc. persons. They couldn't find the right way.

I think one of the important issues with NLP research is that it had been lacking persons with the appropriate combination of talents, mindset and personality to go to a different path than those 50-years old one.

It is not easy to state: "I think this is a wrong approach, let's find another one!", especially if you are young.

Most researchers accept "this is correct, because - quote prof. A, prof. B... They are from University C, which has the most publication in journals D, E and F, which are ... (Oops, there are no Nobel prizes in NLP).

Anyway - therefore, this is the best, because it is quoted there and has 89.95% in this measure, which is accepted by.... Also, the paper suggests 94.34% in the test of "interrelated multipart tagging of coverage structures" etc., so this is real!." and so on.

Or they just want to have their PhD now, and the easiest and fastest way is to fetch a topic from the mainstream and do it the way it is done - these topics are... In Bulgarian it's called "Dissertabilni" - acceptable for a PhD. But mainstream is supposed to be behind the cutting edge.


So I'll say it again:

The reason why so many researchers are doing the same research and progressing so slowly is that they do assume that the others with higher status are right, and base their "original" research too much on it. They do not imagine wildly enough.

The same trivial, not original, not really inventive research, dealing with the old obvious parameters and items, supposed to be "the right ones"....


Conclusion: The paradigm of NLP is wrong.


THE END


To be continued...


Best Regards
Todor Arnaudov


Suggested reading (google): Boris Kazachenko, Jeff Hawkins, Todor Arnaudov (български - http://eim.hit.bg/razum)
Read More

Monday, June 29, 2020

// // Leave a Comment

Is Natural Language Recursion or Dereferencing? - Comment about the article "Recursive language and modern language were acquired simultaneously ..."

https://phys.org/news/2019-08-recursive-language-modern-simultaneously-years.html?fbclid=IwAR04SqHIJKrbZR52gvllBiM2qJp3-xhZ8aJ0vQXW1-hwEcplsjTw8ZccglE

And the discussion of B.K. from CogAlg: https://www.facebook.com/boris.kazachenko.5/posts/10216534326967246
"a snake on the boulder to the left of the tall tree that is behind the hill,"  
"Prefrontal Synthesis (PFS)."  
" Similarly, nested explanations, such as "a snake on the boulder to the left of the tall tree that is behind the hill,"  force listeners to use PFS to combine objects (a snake, the boulder, the tree, and the hill) into a novel scene. Flexible object combination and nesting (otherwise known as recursion) are characteristic features of all human languages. For this reason, linguists refer to modern languages as recursive languages."

The bold part (bold - mine) is a general characteristic of codes.

I didn't see the capacity of the "working memory" mentioned, while it is crucial and general - both as the definition of  "7+2" and in more general sense of complexity of the representations one could productively work with, which correlates with the complexity of the matters ones could deal with.

Nested and recursive are not synonyms by default IMO. You could call them so, but IMO they *could* be, but are not obliged to be. Recursion is about self-reference, while the example IMO is not "self" reference, unless assuming such "function call" mechanism. It's *chaining* and redirection of relations (a list, graph, network), some of the parts could be not processed the same way, thus not "recursion".

( Sure, one can call "recursion" whatever she wants; e.g. regarding CogAlg there was a time, I don't know if it is still valid, where "recursion" was called "iteration" interchangeably - they are opposite methods in programming. The general of these is "a stage of processing". )

As of redirection, a famous quote in Computer Science says, that "we can solve everything with one more level of redirection". The original uses another word:

Indirection | Dereferencing | Redirection

"We can solve any problem by introducing an extra level of indirection."  
Indirection, dereferencing: https://en.wikipedia.org/wiki/Indirection
https://en.wikipedia.org/wiki/Fundamental_theorem_of_software_engineering

Processor's Indirection

It starts with the CPUs with more advanced methods for addressing, even for simple CPUs like read the value at the address, pointed by the current content of a register, offset by the content of selected index register (read the content of the index register, add the value to the already read value, multiplied by the selected word-size - byte, word, doubleword, quadword, - where those selections are encoded in the opcodes of the instructions - and store the read value in the same source register.

Say:  A - Accumulator, X - IndeX register

MOV @[A+X]

It could be to point to an address, which is used as another address of a table, add to it offset from another register, add something else etc.

This is constantly done also when parsing and expanding high level code and data structures, first load one table with addresses with a key (identifier), search, find where it redirects, get another table etc.

It is similar to the linguistic dereferences like:

"This is the book of the girl that lives on the third floor in your house".

All that is easier to be expressed in code and by graphs/flowcharts with arrows, rather than by text, because it's naturally "spatial" and connected.

...

Regarding the example from the article:


It can be traversed and processed recursively, but it could be done also iteratively, up to particular "depth", length, and since in human case that length is quite limited anyway, and it doesn't grow infinitely, unlike the claims in the artificial syntactic examples, most people very quickly get lost in the relations betweeen words - probably like in this sentence.

That has a simple general interpretation, though and exposure to nested input can't help it: too limited working memory (resources) do not allow to spawn and to hold enough objects/patterns and relations in order to understand their relations or to combine them into new; while just activation/recall is cheaper, so a system may be able to just recall memories and reconstruct already visited patterns, which may require less activations in the higher levels (PFC), but couldn't have an even higher level which use many of these into new imagined entities, combined by other parts. The relations are patterns and they seem to be expensive ones for humans.

Even if you had that skill of "PFS" within the cognitive repertoir and you can generate sequences in principle, if you had that "modern imagination", if the available resources allow just to manage one level of certain complexity, the result will be the same.

This is exemplified by the complexity of the used sentences and language, which is supposed to grow during language acquisition, but it has a limit. That goes also for the complexity of programming code expressions for developers' skills.

The math "word problems" given to students are also tests for the working memory capacity:

John has 5 apples, Mary has 3 apples more, but she gave 2 to Kate who had 1. How many apples Mary has now?

These problems require correspondingly big enough working memory for such patterns in order to hold the elements and not just in principle to be able to do "recursion" (or nesting or chaining).

...

Mapping to vocalisation

I agree about with Boris that a mapping to vocalisation exists ( to recording spoken words in working memory as well), I've measured sometimes the lenght of text which I can remember and write correctly while listening to a talk, a show etc, see also my old compressed definition of natural language, given for example in the "What's Wrong With the NLP" series:

Todor: Natural language is a hierarchical redirection/abstraction/generalization/compression of sequences of multi-modal sensory inputs and motor outputs, and records and predictions for both.
When we learn the language in a multi-modal net which includes motor commands to the vocal tract and sensory from the records of other people talking, our own sounds and utterances, expectations of what sound would be produced when we do this or that sequence of motor commands to the vocal system, starting with current state etc., then it would recall memories of "vocalization" etc.

https://en.wikipedia.org/wiki/Vocal_tract


Declarative Memory, Hippocampus, Consciousness, Creativity, Sequences

I also agree that there is a correlation/relation between declarative memory capacity/clarity/skills and general intelligence and creativity (and have at least myself as an example for that phenomenon), so if Hippocampus is required and important for the former, then it would be for the general hierarchical sequence generations and analysis/re-syntehsies as well, which is in short what producing and cognition of creative works is, either "hard" as code or scientific theories, or more artistic as creative writing, music, films. Also "declarative memory" is required for navigation as "memory" - you must be able to record and compare what have you visited and what's "history", in what sequence, what was prior and what follows etc.

It is known by research that the capacity of the working memory strongly correlates with the G-factor - general intelligene, - which suggests both/either the existence of general processing within "mind" and that this memory is somewhat distributed, and/or something that Schopenhauer has suggested in 19-th century if I remember correctly - that the difference between genius and average/mediocre is actually quantative, not qualitative, the latter just runs "out of memory" too quickly and can't reach to the required complexity and length to understand, discover or produce something which is "more meaningful" or original than the expected*.


* If I'm not mistaken that was in "Parerga and Paralipomena"

Prepositions

That reminds me of insights I had about "prepositions" in language and how they related to the Cognitive Algorithm theory of Boris as I knew and understood it back in the end of 2015.

I may revisit the discussion:

To be continued...


* Regarding the "prepositions", mentioned in the article, as material syntactical elements, they could be virtual, implied in the word forms, but also, technically, as someone mentions in the comments section, too, many ancient languages usually are with cases (падежи), the prepositions are scarce or auxilliary or still require cases - such as the European languages: Latin, Greek, German (thus Proto-English), Slavic - except for Bulgarian which gradually lost the cases (except a few informal and in some expressions, and we generally understand some of the cases in kin languages because they use morphemes/suffixes which we use and understand, such as - "у", "ов", "му" - etc. and there are archaisms and Old Bulgarian ("Old Church-Slavonic") which are known such as "Православному българскому народу", "Моли се Богу", и съвременното: "У нас" (At my home, at our home) и пр.
Read More

Thursday, January 4, 2018

// // Leave a Comment

The lack of operational hierarchical structure in the Deep Learning ANN neural networks


A survey paper on the issues of Deep Learning by Gray Marcus: https://arxiv.org/ftp/arxiv/papers/1801/1801.00631.pdf

The author has a valuable mix of expertise both in the ANN development and in linguistics, developmental psychology, cognitive psychology.

There are good points on the lack of real hierarchical structure in (current/regular) DL/ANN, accenting that they are actually "flat", even though there are "layers" which gives a confusing impression.

"To a linguist like Noam Chomsky, the troubles Jia and Liang documented would be
unsurprising. Fundamentally, most current deep-learning based language models
represent sentences as mere sequences of words, whereas Chomsky has long argued that
language has a hierarchical structure, in which larger structures are recursively
constructed out of smaller components"

G.Marcus p.9

See also Jeff Hawkins point since 2004 "On Intelligence"/Numenta, Dileep George's Vicarious, Boris Kazachenko's "Cognitive Algorithm"; the old Hierarchical Markov Models; probably many other researchers, also myself since my early 2000s writings where even I as a teenager have realized that human general intelligence faculty is a Hierarchical simulator and predictor of virtual universes.

The ANNs (without being put in another system/organization) lack operational structure.

Good survey and discussion of areas where DL fails and emphasis of the lack of transfer of learning, i.e. that the networks are not general intelligence and don't "understand" the concepts (the "overattribution" for DeepMind's Atari-player discovery of "tunnels", see p.9)



* However I don't like the pretentiousness in some parts of the article while discussing trivialities and proposing alternatives with 15+ years(?) delay with a pinch of academic glamour or so.

E.g. unsupervised learning (not that common boring classification of fixed images) and self-organization, incremental complexity/"self-improvement" - "Seed AI"; hierarchical operational structure, "symbol grounding" - emergence of generalizaions/"symbols", "abstract thought" from the sensory processing; different levels of abstraction - including "symbolic"; causality understanding (prediction, simulation of "virtual universes"); general/universal game playing; application of general educational tests/measures... (Since AGI is about that; the term "human level (general) (artificial) intelligence" was used in the past) etc. (Not just "pattern matching" of synthetic static tests.)

The above is what AI was always supposed to be about - AGI, - at least as some talented teenagers and others realized and shouted it to the world in the early 2000s and dismissed the poisoned term "AI". Everything was called "AI" back then - somewhat similar today, AI is ubiquitous, yet not general and lacking a personal wholeness.

These suggestions and conclusions would be informative for hard-core AI-er, though (programmers-mathematicians type), it seems the "general"-... part has still a way to go as a concept for the "mainstream" developers community with its "Narrow AI" attitudes**.

...

** Narrow AI - another forgotten term, which on second thought is still actual. Current DL is in fact "narrow AI", each network is trained for a specific class of problems ("classification") and as well explained in the paper can't generalize concepts and transfer the knowledge to different domains.

*** I "don't like" my own pretentiousness, too, but I consider it funny and ironic, rather than serious like in the paper. :P

**** Thanks to Preslav Nakov for sharing the link!

...

Compare the educational test proposals with one of the first articles in this blog, a decade ago:

Wednesday, November 14, 2007
Faults in Turing Test and Lovelace Test. Introduction of Educational Test.
https://artificial-mind.blogspot.bg/2007/11/faults-in-turing-test-and-lovelace-test.html

I didn't explicitly defined the exact kinds of tests, because they were already given in details in the appropriate textbooks about the set of the respective expected skills and knowledge for the respective age or educational level.

...

The article reminds me of the series of articles "What's wrong with Natural language processing?", starting from the year 2009:

https://artificial-mind.blogspot.bg/search?q=what%27s+wrong+with

(...)

Vicarious' demo video summarizing ANN reinforcement learning faults, and their Schema Networks:





Read More

Tuesday, October 18, 2011

// // 15 comments

Rationalization and Confusions Caused by High Level Generalizations and the Feedforward-Feedback Imbalance in Brain and Generalization Hierarchies

Rationalization and Confusions Caused by High Level Generalizations and the Feedforward-Feedback Imbalance in Brain and Generalization Hierarchies

Continues from Frontal Lobe Activation Patterns in Pessimistic & Optimistic Brains, and in Infant Brain Before and After Understanding of Object Permanence


1. Higher-to-Lower level feedback is less efficient than Lower-To-Higher level feed-forward generalization.

Example: Image/Object Recognition vs Image/Object Rendering, performed by humans.

Every healthy child can recognize human faces, understand emotions and react accordingly, but it's not that easy when intentions are involved - feedback/output to act over environment. Realistic drawing or painting of faces or good acting in a film require time, talent and practice.*

(*I guess I can get the critics that regarding painting/drawing it's just "precision vs scope", some autistic people are great in copying inputs. However drawing by memory and creative drawing of imaginary subjects require both scope and precision and capability to keep consistent mapping between all the levels, from the highest to the lowest.)

2. Generalization out of specifics (rich sensory input) is simpler/more efficient than specification down from generalizations ("decompression")

Generalization is selective lossy compression. Decompression requires reconstruction of the lost data, or requires that data is preserved or the knowledge how to extract it from external memory and incorporate it is kept. Here is the point of "precision vs depth", however I'll emphasize that sometimes a lot of both is needed.

Besides, I suspect depth has more severe limitations than precision for the lower levels. The cognitive hierarchy can add generalization levels at the expense of wider scope of input data and/or lower detail, but these both are problematic - if the scope is extended at the expense of detail (to keep computational complexity under control), then generalization levels will run out shortly, because there won't be meaningful details remaining. On the other hand, if the scope is extended with more modest detail lost, then learning will go computationally out of control.

We don't know where a machine can go in generalization levels with more computing power, but I think brain is very limited.

3. Higher level sensory inputs have less of impact over the cognitive hierarchy, because the feedback is less efficient than feed-forward.

That's the reason why captions "Smoking kills" or "Speeding kills" usually have no effect to make people stop smoking or stop speeding and why first-hand experience - to see with your own eyes, to hear with your own ears and to touch with your own fingers - have more dramatic effect in transmitting any message, than relayed experience of others.

Seeing your friend smoker dying of lung cancer or seeing your friend smashed in his car because he drove drunk - that's a pretty different sensory input - and not only because of your personal involvement with the sufferers.

Text is too abstract and distant - the low level physical representation of text is meaningless, it serves only to encode a higher level representation - that's where the input starts having a meaningful impact to the brain, and the message has to go down in the hierarchy to have actual impact on the behavior.

Another example is acting and film. Film as media demands providing motion pictures and rich sound - the physics of the action at the lowest level possible with a lot of details/high resolution of the input. If there's too much of a dialogue and too much of self-explanations and declarations by the characters, especially of obvious things - then brain is fed with high level generalized input, it can't activate the lower levels from the top-down, and they're idle or "bored". On the other hand if the lowest sensory input is rich, brain can induce up generalizations and engage the entire hierarchy. (Besides the effects of the balance of unpredictability etc., see Schmidhuber's works on Creativity)

4. Rationalization is playing with high-level patterns to explain lower level patterns, which the higher level cannot access, because of the fact that feedback is worse than feedforward, or because lower level patterns are unknown

Higher levels in the cognitive hierarchy are derivatives of the lower levels - the lower levels induce the higher ones, not vice verse. In a sense (a bit simplified), higher level patterns in the cognitive hierarchy are a delayed expressions of the lower level ones. However once a higher level emerges out of the stable regularities in the lower level, it starts to mess with the lower level business - adjusting lower level input, selecting data to keep attention on; adjusting coordinates, resolution, location; and the higher level does in order to maximize its own "success" - match, prediction, reward.

Higher levels usually cannot explain and trace back how they are created, what their lower level patterns are and what are the lower level drives.

Similar situation is with bad philosophy and other fields* where lower level conceptualization, patterns and input are wanted for a conceptual progress, but practitioners deny it and keep blah-blah-ing with concepts which are too high a level, too general, too unrelated to the problem they're trying to solve.

That's also one of the reason for researchers such as Boris Kazachenko and myself to suggest bottom-up approach - it allows for the maximum possible abstraction, while keeping maximum possible resolution and keeping the traces of the abstraction.

*Search the blog with "What's Wrong with Natural Language Processing

5. There are Two Reward Systems which are Messed Up

There's another issue - two reward systems run in parallel in brain. A cognitive and a physical. Cognitive system aims at maximizing predicted match of pure data, while the physical system aims at maximizing desired match - input sensations must match hardwired target sensations loaded with value - food, warmth, water, sex etc. The physical is way more primitive and crude, it relies a lot on the more primitive brain areas and on dopamine and other neurotransmitters/neuromodulators/hormones, while cognitive system is based on finer processing, even though the former participate also. Both systems interact and overlap, the physical system can override the cognitive and make it a slave - for example higher level cognition of drug addicts is a slave of the primitive need to take the drug. Generally these systems are messed up and tangled, so it's hard to trace where starts which in real behavioral record.

6. Rationalization is also explaining physical motivation with cognitive means

Ask somebody why she loves her boyfriend. She's likely to tell you "because he's smart, funny, kind, blah, blah and because he'so soooo blah!", while the real reason is much simpler - the way he makes her feel. That's why she loves him, where "to love" has also more basic meaning than the societal one - it's about quantities of neurotransmitters and about "imprinting" of addiction-cycles of generating such neurotransmitters by physical-cognitive conditioning, inter-association.

It's true that the abstract reasons do play certain role in creating the inter-associations between physical and cognitive sensations - everyone has some preferences and favorites, - however this can be reduced to:
- I love him because he's the type I wanted him to be!
- I love him, because he's my perfect match!
- He's the best match I could find so far...

This is a match between desired and input, which is the type of match of the physical reward system - apparently this selection is driven by the physical system, overriding the cognitive.

That's why I think the abstract reasons of "smart, funny..." are "rationalizations", but rationalization is not strange at all. Everybody would say "yeah, socially acceptable explanations", but that's a cheap answer.

There's one more appropriate reason for rationalization - it's the cognitive system which is asked the question (it's asked in a natural language); the highest levels in the cognitive hierarchy are ruling this area, and yes - the society has taught this higher cognitive system how it should act in such situations etc.

If you ask the question in a lower level language, the answer is different - that's her body language and her behavior when she's with her boyfriend alone, when they're kissing and making love. The answers are also in the amount of oxytocin, dopamine and other chemicals in her brain, the release of which is conditioned with the certain cognitive patterns initially generated by perceiving her beloved one.

In general, for love and attraction it's true by the definition of "emotions" that the physical reward system kicks in. One may be "just a friend" with someone because he's "smart, funny, blah-blah", but even then if one is a human being with a brain in tact (not a sociopath/psychopath), his friendship relations would be messed up with the physical reward systems - emotions, crude emission of certain chemicals and activations of primitive brain areas, which is associated/recorded/conditioned with cognitive patterns, and both are inter-twined.

Purely cognitive "friendship" is business and if it's such, it's not really a friendship.

*Higher/Lower level drives - Passionate feelings demand for "lower" drives, where "lower" in this context has different meaning than lower in the cognitive hierarchy. The meaning here is driven by evolutionary and physically "lower" brain modules, which maps to areas different than the neocortex, archicortex (hippocampus) and thalamus. However physical reward system is not in the same hierarchy as the cognitive system and the levels regarding tracing back, cognitive and physical reward systems are entangled and the physical system can manipulate and "short circuit" all levels of the cognitive hierarchy.


Continues... - On the apparent inconsistency of the goals of a system with cognitive hierarchy and More on rationalization and the confusions of the higher levels in the cognitive hierarchy.

Suggested reading:

Analysis of the meaning of a sentence, based on the knowledge base of an operational thinking machine. Reflections about the meaning and artificial intelligence - T.A. 2004

http://knol.google.com/k/cognitive-focus-generalist-vs-specialist-bias

http://knol.google.com/k/boris-kazachenko/executive-attention/27zxw65mxxlt7/11#

http://knol.google.com/k/intelligence-as-a-cognitive-algorithm

http://research.twenkid.com/agi_english/

Slides on T. Arnaudov's "Teenage Theory of Universe and Mind"


(C) T. Arnaudov 2011
Read More

Monday, February 5, 2018

// // 2 comments

Sensori-Motor Grounding of Surface Properties - an Exercise Trace of the Thoughts and Tree of Questions by Todor from 2011 in AGI List

After the selected emails from 2012 where I discussed generalization and the real meaning of "invariance" in 3D, I'm sharing another selected letter from the AGI list on general intelligence and sensory-motor grounding and its connection to symbolic/abstract representation. The "glue" of this to a real system is the specific processing environment, which applies the sensori-motor mapping and gradually traverses the possible actions ("affordances") within a specific input space (a universe, an environment) and maps them to the sensori data hierarchically with incremental complexity. It should gradually increase for example the number and the range - e.g. involving more modalities of input and output (action), wider range in space and time - of the parameters defining a particular current "highest complexity" conception, which in the example below are eventually represented as words ("a house", "a surface", ...).

The system's "motor" should be "ignited" to explore and the exploration should generate the higher level representations out of the simple sensory inputs like the ones explained below.

Note that the learning - the inductive, discovery - process starts from the end of this "trace of the thoughts". The reasoning was to show that it is possible and even easy/obvious to introspectively trace it from the general conceptions down to the specific and how "low complexity" these abstractions actually were.


See also:

Todor's: "Chairs, Buildings, Caricatures, 3D-Reconstruction..." and that semantic analysis exercise back from March 2004 Semantic analysis ...

Kazachenko's "Cognitive Algorithm" which claims to incrementally add "new variables".


from Todor Arnaudov twenkid @ ...
to agi@listbox.com
date Sun, Sep 11, 2011 at 3:12 AM
subject Re: [agi] The NLP Researchers cannot understand language. Computers could. Speech recognition plateau, or What's wrong with Natural Language Processing? Part 3
mailed-by gmail.com (....)

IMHO sensorimotor approach has definitely more *general* input and output - just "raw" numbers in a coordinate system, the minimum overloaded semantics.

[Compared to "purely symbolic". Note that sensori-motor doesn't exclude symbolic - this is where it converges after building a sufficiently high or long hierarchy (inter-modal, many processing stages, building an explicit discrete dictionary of patterns/symbols) and when it aims at "generality", "compression" or partially arbitrary point of view of the evaluator who's deciding whether something is "symbolic". The way sensori-motor data is processed may also be represented "symbolically", "mathematically" (all code in a computer is supposed to be). The "not symbolic" sense is that it's aimed to be capable of mapping the structure of the emerging conceptions, correlations, "patterns" ("symbols"...) to a chain or a network, or a system of discoveries and correlations within a spatio-temporal domain in the "general" "raw sensory input" from the environment, or one that can be mapped to such input. On the other hand the "purely symbolic" combinations have no explicit connection to that kind of "most general" "raw input". Note, 7.1.2018]
That way the system has higher resolution of perception and causality/control (my terms), which is how close the output/input can be recovered to the lowest laws of physics/properties of the environment where the system acts/interacts.

I think "fluidity"/"smoothness" that Mike talks about is related to the gradual steps in resolution of generalization and detail of patterns which is possible if your start with the highest available sensory resolution and gradually abstract details while keeping relevant details at relevant levels of abstraction, and using them on demand when you need them to maximize match/precision/generality. When system starts from too high an abstraction, most of the details are gone.

[However, that's not that bad by default, because what remains is the most relevant - the spaces of the affordances are minimal and easily searchable in full, even introspectively. See below. Note, 5.2.2018]

BTW, I did this little exercise to trace what really some concepts mean:


[Starting randomly from some perception or ideas, thoughts and then the "Trace of the thoughts" process should converge down to the basic sensori-motor records and interactions from which the linguistic and abstract concepts have emerged and how.]...

What is a house?
- has (door, windows, chairs, ... ) /

What is a door?

has(...)... //I am lazy here, skip to few lines below...

is (wood, metal, ...)

What is wood?

is(material, ...)

What is material?

What is surface?

What are material properties?

-- Visual, tactile; weight (force); size (visual, tactile-temporal, ...)

has(surface, ...)

is(smooth, rough, sharp; polished...)


What are surface properties? //An exercise on the fly

- Tactile sensory input records (not generalized, exact records)

- Visual sensory input -- texture, reflection (that's more generalized, complex transformations from environmental images)

- Visual sensory input in time -- water spilled on the surface is being absorbed (visual changes), or it forms pools

-- How absorption is learned at first?

---- Records of inputs, when water [was] spilled, the appearance of the surface changes, color gets darker (e.g. wool)

-- How not absorbing surface is discovered?

---- Records of inputs, when water spilled, appearance of the surface changes; pool forms
------  [pools are] changes in brightness, new edges in the visual data [which are] marking the end of the pools

-- How is [it] learnt that the edges of the pools are edges of water?
---- [By] Records of tactile inputs -- sliding a finger on the surface until it touches the edge, the finger gets wet

-- What is "wet"?

---- Tactile/Thermal/Proprioception/Temporal records of sensory input:

---- changing coordinates of the finger

---- finger was "dry"

---- when touching the edge:

------ decrease in temperature of the finger [is] detected

-- when [the "wet"] finger touches another finger, ... or other location, thermal sensor indicates decrease of other's temperature as well

-- when [the] finger slides on the surface when wet, it feels "smoother" than when "dry"

[What is "smoother"?]

-- "Smoother" is - Temporal (speed), proprioception + others

-- The same force applied yields to faster spatial speed [that maps to "lower friction"]

[What is "faster [higher] speed"?]

-- "Faster"[higher] speed is:

---- [When] The ratio of spatial differences between two series of two adjacent samples is in favor of the faster.

-- The friction is lower than before touching the edge of the pool.

[What is "friction"?]

-- Friction is:

-- Intensity of pressure of receptor, located in the finger.

Compared to the pressure recorded from other fingers, the finger which is being
 sliding measures higher pressure than the other fingers

...

So yes, it seems we can define it with NL [Natural Language], but eventually it all goes back down to parameters of the receptors -- which is how it got up first. Also we do understand NL definitions, because we *got* general intelligence.


An AGI baby doesn't need to have defined "wet" as "lower temperature" etc. -- it just touches, slides a finger etc. keep the record, and generalize on it.

Then it associates it with the word "wet" which "adults" w (....)

--- THE END ---

Read More

Sunday, July 7, 2019

// // Leave a Comment

MIT's Interdisciplinary Billion Dollar Computing College - 9-10 years after the interdisciplinary program of Todor at Plovdiv University Etc.

Comments regarding a recent talk from Lex Fridman's AI podcast with Jeff Hawkins from Numenta:

https://youtu.be/-EVqrDlAqYo
Conceptually it seems Hawkins'  approach and ideas still match many of the insights and direction in my "Theory of Universe and Mind"  works from the early 2000s (published before "On Intelligence" in "Sacred Computer" - "Свещеният сметач") and afterwards.

In addition to:  not following the mainstream research which is doing minimal changes which lead  to minimal progress of some benchmarks which is assumed good enough by the mainstream researchers, published in journals and conferences etc. Rather radical jumps are needed and recently "even the godfathers in the field agreed"...

The building of deep structures, the play of resolution of perception and control, coordinate spaces as a basis for AGI  ("reference frames"), attention traversal of different scales ("time scales"  - resolution of perception and causation within the time dimension), introspection as a legitimate method for AGI research, these are my "behavintrospective" studies; that there is no separate training and inference stage as in current NN, it's supposed to be a part of one process. See CogAlg.

One difference though - he dismisses the interdisciplinary research as helpful (although I think they actually do such research). "Human-centered AI" is disliked, because it suggests study of emotions and other human traits which are not needed for the AI, "let's just study the brain etc.".

IMO the interdisciplinary minds see and understand easier shortcuts while others could eventually find these paths by laborious digging and wandering in seas of empirical data and brute-force search.

https://youtu.be/-EVqrDlAqYo?t=5141

~ 1:25 h

"The new steps should be orthogonal ..."  (no little changes "1.1% progress" on standard benchmarks - see Todor's "What's wrong with Natural Language Processing" series)

1:25:41

The Billion dollar computing college of MIT, interdisciplinary, from this fall:
http://news.mit.edu/2018/mit-reshapes-itself-stephen-schwarzman-college-of-computing-1015

https://fortune.com/2018/10/16/mit-college-computing-artificial-intelligence-billion-dollars/


Well: 9-10 years after the course/research direction program that I announced in late 2009, presented in spring 2010 at Plovdiv University, with practically zero funding, besides the opportunity to present it at my University (thanks to Mancho Manev and Plovdiv University).





Read More

Monday, February 20, 2012

// // Leave a Comment

Philosophical and Interdisciplinary Discussion on General Intelligence, AGI and Superintelligence Safety and Human Moral | Cognitive Origins of the Concepts of Human Soul and its Immortality | Free Will and How it Originates Cognitively | Animate Being and Soul and the Cognitive Reason for the Believe that "Thinking Machines can't have a Soul and Consciousness" | Technology Making us more Humane | The Egoism of Humanity | And more

By Todor "Twenkid/Tosh" Arnaudov
http://research.twenkid.com/wp/

For the followers of the blog - that's not the paper announced in the previous post.


The expression and sharing of the following reflections and speculations is inspired by a publication for a position offered at the FACULTY OF PHILOSOPHY, Oxford University, published on the AGI List (thanks to Sean O'Heigeartaigh).

http://www.fhi.ox.ac.uk/get_involved/future_tech_vacancies/futuretech

See the pdf document, the research foci on p. 2, section 5:

http://www.fhi.ox.ac.uk/__data/assets/pdf_file/0014/25034/Futuretech_Tamas_RF_230112.pdf 

5. This is a research position focused on topics related to the long-term future of machine intelligence and AI. Relevant research foci include: 
- Studying paths towards strong AI: Monitor the current state of progress in the field; identify milestones, roadmaps, etc. 
- Machine intelligence: Analyze fundamental concepts—e.g. how to define and measure general intelligence in artificial systems; how to distinguish different kinds of goal-seeking systems.
- Self-modifying systems: What can be proved about the performance and capabilities of different kinds of recursively self-modifying programs? Can a framework be developed in which demonstrably safe, recursively self-improving AIs could be constructed, with stable and human-friendly goal systems? 
- The role of big data: What impacts and applications will arise from the increasing availability of enormous data sets? What are the fundamental tradeoffs between processing power and data? Is there a minimum amount of data that an arbitrarily powerful intelligence would need in order to effectively deal with various task domains? Can one quantify how much information the human brain contains at birth to enable it to develop general intelligence? Can one quantify how “difficult” it was for such a system to evolve? 
- Philosophy of computer science and of AI: What does it mean to implement a computation? Would a “Boltzmann Brain” or a “Swampman” be conscious? 
- The control problem: How could one ensure that an artificial superintelligence would be safe and beneficial? 
- Can one analyze classes of possible utility functions and construct general statements about the behavior of expected utility-maximizing superintelligence based upon utility functions of a particular class?

Sounds familiar - I have already worked on and have been studied the topics listed in section 5 in the pdf document since the "teenage theory" era, my ultimate aim is at understanding general intelligence and building an eventually developing an AGI system, thus my intent when working on related philosophical issues is to put the  speculations and explanations in coherent computational/theoretical form aligned to a theory of GI to the general/cybernetic/systems theory trends in Universe evolution as I see them (a sort of cybernetic metaphysics).

I also aim to match my theoretical speculations and insights with all accessible kinds of neuroscientific, evolutionary, cognitive science, behavioral etc. evidences I'm aware of at the time. Often I discover such evidences later. For example ~10 years ago, while not being familiar with neuroscience as now, appropriate reflections and a basic understanding in developmental psychology allowed me to induce the existence and functional distinction of hippocampus and the neocortex using only behavioral evidences. See: http://artificial-mind.blogspot.com/2010/06/teenage-theory-of-mind-and-universe.html
Teenage Theory of Mind and Universe (Excerpts, Part 1) - Theoretical Induction of Neocortex and Hippocampus using Consciousness; Compression 

Regarding the philosophical problems, some of my oldest work include analyses of questions, for example related to:
-- What is a "soul" really and why people believe they have such a thing and are more likely to intuitively believe higher animals have and machine shouldn't have?
-- Why a mind believes the soul is immortal and should exist forever?
-- How death is actually perceived from a thinking machine and human perspective and what humans are really afraid of?
-- What actually humans (intuitively) mean with "free will" and why humans insist they have and machines would not?
-- What actually the concepts of hell and heaven illustrate about human mind operation, why so many religions have such concepts, how a cognitive system invents them, why it's so strong?
-- Discussions about consciousness, what is it from different perspectives.
-- General metaphysical universe principles and trends displayed in society, computers, intelligence
-- Core human drives, such as egoism vs altruism, and why love is socially/pragmatically praised?
-- What is creativity, what is to be original, why humans believe computers can't be creative?
-- What is actually implied by "rational". Can an intentional agent (and human) be really "irrational" or rather the model of the agent, used by the evaluator of rationality, is too coarse grained and wrong?


Regarding those sample directions in section 5., I'll use the opportunity to give some contrary statements, from the point of view of the thinking machines, that would be smart enough to unveil our biases and self-praising believes about humans "undisputed" moral superiority.


Oxford: Machine intelligence: Analyze fundamental concepts—e.g. how to define and measure general intelligence in artificial systems; how to distinguish different kinds of goal-seeking systems.


  • 1. Hutter and Legg's Definition of Machine Intelligence, and the Educational Test

There is serious work already done, see for example M. Hutter and Shane Legg's  paper below. These are slides I've prepared from it for my students, with some additional notes of mine:
http://research.twenkid.com/agi_english/Machine_Intelligence_Hutter_Legg_Eng_MTR_Twenkid_Research.pdf

  • 2. Maximum degree of value-unloadedness for the raw input and output of the system, and maximum resolution of perception and causality/control compared to the maximum possible resolution in the environment where the system exists - that's one definition of generality of artificial intelligence of mine

Another simple/core measure for "generality"  in principle, I tried to discussed on the AGI List in the autumn of 2011 - the generality of the raw sensory input from which regularities are induced, that is -- value unloadedness of the initial inputs and outputs, such as sensory matrices similar to human auditory, visual and tactile input.

The meaning of initial cognitive data in such representations is just a sequence of numerical values and their coordinates in space and time within the matrix and within the records of past states of the sensory matrices and to some internal parameters.  That's the most general (value free) input to which further is added value - meaning, generalized and specialized.

The initial output in humans and in maximum general intelligence is as general either -- muscle actions --- which are translation, rotation, propulsion. These operations alter the coordinates of the parts of the system relative to the environment. That's true also for the vocal tract and speech - the surfaces of the vocal chords vibrate - alter their coordinates at high frequency, compared to muscles - and the tongue, lips, jaws, larynx motion and coordinate changes modulate the sound. The environment - "the real world" is just the richest and the lowest level sensory input, the one with the highest possible resolution, the most details, the computationally hardest to model (predict).
  • The more of initial value loadedness, the less of generality of the intelligence derivable from that point on
In this regard, on the other hand NLP (Natural Language Processing) for example is far less general intelligence, because it starts with very abstract concepts whose derivation from lower generality concepts is lost and unrecoverable by the words themselves without using lower level inputs and concepts - less abstract inputs.

That's a common problem in narrow AI and NLP - to move forward, the working concepts of the theories need to start from a lower level of generalization, otherwise they start and finish as combinatorial equations with no grounding.  See the series of critical articles: What's Wrong with Natural Language Processing?.

Oxford:   Self-modifying systems: What can be proved about the performance and capabilities of different kinds of recursively self-modifying programs? Can a framework be developed in which demonstrably safe, recursively self-improving AIs could be constructed, with stable and human-friendly goal systems?
Oxford:   The control problem: How could one ensure that an artificial superintelligence would be safe and beneficial?


It seems obvious, but any significant invention can be used either for good or evil, but while thinking machines *could* only hypothetically turn into terminators... However there's one nature's "invention" that has always been a terminator and has ever been an unstoppable killer: HUMANS.
  • Humans are the archetype of James Cameron's "The Terminator"
When discussing the dangers of AGI with my students I've asked them to reflect on:

- How could one ensure the safety of humans and human intelligence? How could one control humans and did we succeed in the history of humanity to prevent wars or genocides?

If anything could control humans not killing and hurting each other, that would be either:
-- an utopia society (totalitarian, for example...)
-- machines which are stronger and faster than us to monitor and react fast enough only in case
-- some sort of cyborg-like implants or environmental means for prevention which block violator's brain or muscles etc...
-- why not altering our "human" nature phisically

In the current state of affairs though, the human race should admit that all homicides are executed by humans and every person is a potential killer and a criminal. History has proven for millenia that humans, or a lot of them, develop as greedy, violent; exploiters, killers, may go insane and aggressive for no apparent reason etc., and the major historical events are of wars, exploitation, slavery and genocides.
  • Two Camps/Enemies Fighting
All those acts are eventually of humans who are in two camps, applying force and aiming to conquer, rob or destroy others - it's not about humans vs non-humans, one ethnic group against another or one race against another*. It's not because of their "genetic difference" (see below for a remark), it's all  about "mine" vs "yours", no matter how or what the one or both of the sides want to get or keep away from the enemy. It comes from the individuality and then groups with particular identity, which can be based on arbitrary values.

In this regard, there could be machines which are on the either sides**, and there could be thinking (or not that smart too) machines which could fight against other machines (to protect particular people and themselves), or against aggressive humans in order to protect civil ones or other machines etc. 

There might even be different "taxa" or "states" of machines fighting each other etc., but in my opinion superintelligence, if autonomous and not spoiled, is supposed to be wiser than humans and might be created or would itself wipe out or control better its human counter-part primitive brain/behavioral functions which for humans drive emotions such as mindless anger, rage, fear, lust, addiction.

Such emotions lead to wars and violence. Wise men are not warriors, but unfortunately the ones who were overtaking the government throughout the history were the ones who had the force, aggression, greed, hatred, cruelty and power to do it. Wisdom is powerless against the physical force.
  • "Neurobiological philosophy"
Indeed - familiarity with comparative-, computational-, evolutionary-, etc. variations of neuroscience happens to lay many philosophical issues down onto evolutionary biological, physiological, behavioral etc. grounds.

* Racism and theories about "inferiority" of races or ethnic groups or nations or whatever groups are in their bottom just made-up formal excuses for applying force and/or measures in order to conquer, exploit, eliminate etc. (rage, greed, anger) - the stronger one wins. Many human acts are driven by low urges, violent ones of course are among them. However people have also higher cognitive functions which run in parallel and they need and produce (invent) systematic reasons to explain the behavior as a whole, which in its gross directions is largely driven by the primitive urges anyway. That's supposed to be related to the phenomena I've discussed recently in the following publication:
Rationalization and Confusions Caused by High Level Generalizations and the Feedforward-Feedback Imbalance in Brain and Generalization Hierarchies
http://artificial-mind.blogspot.com/2011/10/rationalization-and-confusions-caused.html

** War machines don't require very powerful (general) intelligence to be effective - recognition of enemies and allies, aiming at, navigating and transport are not that complex if it's just about destruction and defense, and if the agent is controlling a tank for example. I suggest Hugo de Garis's book  The Artilect War: Cosmists vs. Terrans: A Bitter Controversy Concerning Whether Humanity Should Build Godlike Massively Intelligent Machines - which I haven't read myself, though and can't comment on.
..

Back to the topic:

Oxford:  The control problem: How could one ensure that an artificial superintelligence would be safe and beneficial?

Another example I've given to my students:
  • What if your fellow soldier was a machine, and your enemy - a human?
- Imagine in a state of a war, or a gun fight, what if your fellow soldier is a humanoid robot, and there is a group of enemies - aggressive humans who want to kill you. Should you defend the "humanity" and let your "brothers" kill you or would you fight back with the robot, which is on your side?

In fact soldiers "love" and create emotional bonds even with their guns, tanks and aircrafts, what about intelligent or/and humanoid robots that look and acts like them and protect them.

Another confusion regarding machines as intrinsically "antagonistic", "inferior" and "evil" to human race is related to the fact that society now takes "Human rights" for granted and obvious and sometimes the technological society is blamed for alienation and violence (weapons for mass destruction, world wars). This is an illusion and ignorance, humanity has not obeyed human rights up to very recently, recall again the millenia of wars or just exploitations: slave-masters, serfs, workers in terrible conditions, mindless wars and mindless genocides.
  • Don't take Human rights for granted - they were not always here. Technology made them possible.
In fact, the generalization of an abstract concept of humanity as a whole and the application of those rights de facto and not only in abstract philosophical works are recent artifacts. In my opinion they actually became possible because of the advent of advanced technologies of any kind, especially ones in medicine and for transportation, information and telecommunication, where the IT and communication are crucial for development of all sciences and technologies, and the overall progress is mutually enhanced between different fields.


The technologies allowed massive and fast inter-personal and inter-cultural communication, international world-wide economical and cultural collaboration, higher living standard, higher levels of education and awareness about the world. These events allowed more of "friendliness" between different societies at any scales.


This is unlike for example in the authoritative one-directional communication/propaganda in the nationalism epochs some 100-200 years ago and the 20-th century totalitarian regimes (early advent of radio telecommunication, and the specific antagonistic political situation). It is also unlike the very narrow, localized and culturally heavily biased image of the world and other cultures in the epochs without fast transportation, without steady food-supply, lacking health-care, lacking education, knowledge and telecommunication, when societies were generally very fragmented, isolated, exploiting each other and separated in violently antagonistic camps in all scales.

This topic can be elaborated in details, but in general I believe the progress and IT-communication-transportation-medicine etc. technology allowed the "human rights" to be applied.
  • Machines are making us "humans" and "humane" as we see the terms today
In my opinion, in the long run it is the machines (higher technologies) that are making from us the "humans" as we are now. Technology is where intelligence crystallizes, and intelligence is fighting our beast-self .

Religious objection: Christianity etc. taught people to be good, to love their enemies and all people etc.. Technology makes us bad, greedy ("consumers"), it's a product of evil and the devil etc...

Unfortunately the beautiful suggestions to love all people etc. were not and couldn't be applied totally, and if the Christian moral is literary applied today it won't pass - such as the fornication, not to talk about the darkest times of all churches.

There has always been wise people who has loved all people and were against violence and tyranny, but in reality - big scale - that was an utopia. Enormous amount of wars and outrageous violence were politically justified with words about faith, "moral", sacred goals, "god commandments" etc., where in my opinion the root of all was as simple as:
  • The ones who don't support your vision and direction (the same as your party, your society, your national country, yourself) or refuse to obey your laws no matter what they are, don't  subordinate, don't agree etc. -- they can be tortured and killed without any mercy, because you have the power to do it, and according to ... the power is from... god.

    As stated above - human violence is not about humans vs non-humans, or a race against another, or a religion against another. It's just about me vs you, "I am more powerful and I am right, thus you should obey, or you will be destroyed". In one word: egoism and brutal elementary animal instincts. (See below also.)

Yet another opposite POV of the same question asked in the Oxford's list of research foci:
Oxford: The control problem: How could one ensure that an artificial superintelligence would be safe and beneficial?

People take for granted that AGI should be safe and beneficial from their current point of view, and the world should stay static as it is now. However social and moral values change, humans also change and their values - too, even without radical physical changes, such as to get modified into new biological species of transhumans, cyboorgs or whatever...
  • Safe and beneficial for whom?
Evolution usually is ignored or forgotten - humans feel themselves as "the undisputed masters", "the top of the nature". Some may claim that there are scientific etc. evidences. To me it's a made-up excuse. 
  • Humans believe they are the top of the Universe, because they are the top egoists in Universe
The reason we think ourselves as the masters is, in my opinion, rather much simpler - such as our egoism - individual and the egoism of the society and human race as a "super ego". The anthropocentrism is an expression of human egoism. In anthropocentrism every single individual is identifying as a representative of this idealized model. 

The forced altruism is an extended egoism as well -- society acts as an "individual". It has its specifics - values, identities, - and it forces its members to contribute for the preservation of the "body" and "values" of society as it sees them at the moment.

If anybody opposes, he's punished - everyone should be an altruists, meaning to serve the needs of the society - yet another ego ("super-ego"), satisfying its needs at the expense of exploiting the smaller individual egos.

In order to change the values of the super-ego where a small ego belongs, an aggregate of small egos should collaborate and tune on the same waves to collect enough force. However that would diminish their identities and turn them into another "super-ego".


This process goes in living organisms, from cells to organisms and ecosystems; in the religious groups and political/state governments. The highest level of causality control is aiming at keeping itself as it believe is "correct/stable/right" (it's ego), while the lower level components are aiming at serving their smaller "egos", but are subordinated by "force" and due to the formation of local smaller "super-egos".

  • Humans like to exploit and enslave
For example, the classical Asimov's robots in most of the stories and the robotics laws are an example of humans wanting to have their personal slaves and servants to obey their orders, the same goes to a bigger extent in Karel Chapek's play coining the term "robot".

I guess that to the society back then this view was more acceptable than now. Some would say "of course" - they are machines, they are built, and not born, "they don't have a soul" or personality and individuality (see below for this "spiritual" topic) etc. therefore they should be slaves - that's fair, according to humans.
On the other hand,

  • Is it fair or moral to enslave a being if it outsmarts you and is behaviorally, cognitively and physically more sophisticated  than yourself?
A good example of how this feels is the original version of the movie "The Planet of the Apes" (1968), which is based on the original novel. http://www.imdb.com/title/tt0063442/

What if the gorilla's or chimp's predecessors were smart enough to recognize the rise of the homo lineage and managed to kill it and did forbid "illegal genetic mutations" for the sake of gorillas welfare, because they had the power and will to do it and because they had provisioned, that "this next step in evolution can't be controlled and proven to be safe and beneficial" for them?
The apes would have been morally right for their society, because the future homo species were not beneficial for them - humans killed apes and restricted their habitats. Human race has killed billions of living beings for their own benefits or because we considered the other species "pests". Animal rights are recently applied too, and they don't stop us from killing animals, it's just more "humane killing", and in fact a big part of our identity that we consider "human" is rather animal and evolutionary very ancient.

  • If gorillas and chimps' predecessors had measured the risks of human evolution, they would have killed our lineage millions of years ago and would have been morally right for their society...
Indeed:
- Who gave us the right to kill other species?
The answer is straight: we did, it's the law of the jungle. The physical power is ours, and there are no other beings that are intelligent or powerful enough to contradict or oppose.

We defined the moral to fit us
, so this is moral - we consider us being "higher", or that living is "sacred" (living is us, that's why it's sacred). We're "smarter", we're "more fit" - our measures and classifications are ones that match the principles of our social hierarchies, if you're on top, you deserve the ones below to obey and to subordinate.

Humans just feel as being the masters and they have the power and need to do whatever they want, i.e. "their moral suggests them".

It's interesting to extend the problem of robots and artificial general intelligence as not adequate persons, justified by their nature of being "built", but not born.. What about human children?

  • Aren't children and humans built, too? Do they deserve equal rights to... older humans???
Children are intelligent beings and eventually become citizens with full rights, but everybody was initially "built" by somebody else too like the machines, even though not in a factory, but with biological "robots" - RNA-DNA protein-building processes.

Everyone owes her existence "legally" to somebody else.. In fact - to many others, I don't mean family predecessors - to the society which provided secured environment for them to live as well. 

  • Should then mother or parents (and society) enslave their children? 
In fact parents and society figuratively do - children are dependent on their parents good will and they are deprived from many choices, money and rights to earn money, goods and they lack many civil rights for 16-18 years or more, they are obviously considered "inferior" by the adult society and parents. It's partially justified by their real incapability to survive on their own initially, but the latter is mostly because they live in a world of generally stronger and smarter beings with which they would have to compete - they are protected by their parents and society at the expense of not having particular rights. Even though gifted children are functionally and mentally superior to many young men and adults, they can not climb the hierarchy until they serve their "duty time" until the age fixed by the adult ones in the laws, which are considered "right".

Indeed, the inferior position of children to their parents and to the older ones is perhaps responsible for the following visual psychological phenomenon: "Why shorter Stature and Lower camera angle are Unconsciously associated with "Inferiority"? Memories from childhood (Nature or Nurture)" http://artificial-mind.blogspot.com/2011/08/why-shorter-stature-and-lower-camera.html


Regarding some  "spiritual" issues on AGI/thinking machines

  • Another "right" for humans to put thinking machines at inferior position is the concept of "soul" as a sacred status-symbol, and the self-awareness and consciousness as some supernatural magical characteristics of humans, instead of comparing and studying them as cognitive properties/capabilities

Let me present for example what "soul" actually means for a mind, according to the AGI theory and model of the mind of mine, I got insights and have discussed in this direction since my earliest major works about 10 years ago.

The initial and essential meaning of the concept of "soul" from a computational perspective is just a generalized model of the sensory inputs associated with the initial perceptions of/associated with human beings

The template for this model and for "animate beings" is constructed by the perceptions of the dynamics of the sensory inputs generated and associated by/with the interaction with the particular mother/care-giver and one's own body and visceral senses, "happiness" level etc. Initially baby doesn't distinguish itself and its feelings as individual, they are closely related to the care-giver so their observed generalized model is part of self, later on this template is associated and linked to wider range of people.

The model is further extended to other agents and animals - because their bodies, faces, motions, sound, behavior, interactivity etc. match or is similar to the initial template, perhaps also because it makes one feel (or recall feeling) particular sensations associated with the ones associated with the first "template". This process is cyclically reinforced.

It probably starts from the eyes/eye contact and lips -- eyes are the most clear (self-contained), dynamic and simple early visual sensory pattern. See:  Learned or Innate? Nature or Nurture? Speculations of how a mind can grasp on its own: animate/inanimate objects, face recognition, language...
[Neural correlates will be discussed in another upcoming work]

This abstract model of a human/animate being is used by mind to predict future inputs of it (perceptible future transformations), therefore it's not derived from, and it's supposed to cover all minute physical details (molecules, chemical reactions and full precision physical laws). 

Mind has the computational capabilities and can model at low level only approximate aspects acquired by sensory experience - e.g. motion pictures and records of all kinds of sensory experiences and emotional states - own and the implied of the others' (guessed by associating/matching to own).

  • Nobody really dies until there are people who remember her, because: the mental models of their physical bodies - don't.
Here the explanation of my theory of the cognitive origin of the believe in eternal soul comes:

When somebody else dies, in order this happening to be perceived and evaluated as "death", there should be another living (running, functioning) mind. The model of  "human" and "animate being" and the specifics regarding the dead man's soul are still on-line, the model is not deleted and mind can still imagine it - recall and predict/replay/generate plausible sequences of future "motion" or sensory transformations, like it could do before the person has died.

Mind/self obviously cannot imagine its own death with its own resources as well, because it cannot operate if it's not operating. Probably that explains why often when reflecting on being dead, mind is imagining itself not as "not functioning", but just as in the known self-model, however imaging it as not embodied, not feeling particular emotions, in another place etc. The general properties of the "self" are still kept, such as reflecting, memories, imagination.

In fact, for mind there's no perceptual difference if somebody has gone away (and never has come back) or if he has died without the mind having low-level or clear signs about that - in both cases the deceased agent is just not interacting anymore, there are no new low level inputs to mind associated/recognized as coming from the person that previously was associated as the living person X.Y.

The concept of "soul" is familiar for all kinds of societies, including most primitive ones, and children use that word also - they apparently should mean something which is simple enough, accessible using raw senses and common.

Besides, I suspect the mind imperfection stated above has to do with the origin of  Idealism in philosophy.

  • The believe in the eternity of ideas, soul, spirit and so on come from the impossibility of mind to imagine itself not operating using its own resources. Mind can perceive and imagine  only the death - cessation of operation - of other agents and while it does, it evaluates it using running mind. The "running"-ness of evaluating mind confuses the model of death.

    ...
  • Another meaning implied with the word "soul/spirit" is of course close to consciousness and "qualia", but lack of qualia can't be proven or disproven.

Humans may claim they have a soul, but a machine has not, because it's "just electronics", "just 1s and 0s", "a bunch of metal and semiconductors" etc., but people usually don't realize that a thinking machine that is intelligent enough may claim the same for humans:

"The Machine: Yeah, and your human emotions are quantitative, qualitative, spatial and temporal correlation of chemical substances - proteins, hormones, nucleinic acids etc. I can explain you all the details, but I'm afraid it won't be of any use for you, because your poor human brains won't be capable to cope with that complexity."*

In regard of animals and souls - humans tend to give a right to have a soul of animals, because they look very similar to us and behave like us (we see matches to the initial human model - to that "soul" template), where "us" means each of us individually. That's one reason why people normally feel less remorse for killing an insect or a plant than killing a dog -- insects are just too different visually and behaviorally, and plants don't react to pain.

Empathy is driven by fooling mind like if the person/animal perceived suffering were you, it matches the sensory experience one has for his own and mind assumes the experiences of the other agent should be similar.

As already suggested, the initial perception of "animated beings", and the generalization of a soul as a model of a human initially comes from the actions and observations of our own body and senses, and from the perceptions of the behavior of our parents and people who care for us, and it's strongly correlated with *us* either.

Initially babies don't recognize others as others, they associate their entire experience, all their actions and expectations with their senses (visual, tactile, visceral, happy/sad (the latter correlated with the level of: oxytocin, serotonin, dopmain, adrenalin and other chemicals)). The virtual generalized model of human beings gets imprinted and bound to our basic cognition and to our basic feelings and sense of ourselves.

Indeed, the process of bonding and empathy in mammals is strongly driven and reinforced by release and the effects of the neurotransmitter oxytocin, it's released when giving birth, during sex and when interacting with animated beings (no matter pets or humans). For example that's why caring for a pet is relaxing, and why violence against animals may be an indication for future sociopathic behavior - some subjects have "faults" regarding reception of oxytocin, or they fail to produce it, and feel no remorse and no empathy.

There's another hormone - cortisol - which is produced after a prolonged stress is encountered, such as fear. Fear causes initial release of adrenaline, and if the stress is not overcame, cortisol is released. Cortisol melts down tissues in order to produce glucose, it melts particular brain structures too (hippocampus, involved in declarative memories). Regarding empathy in particular - cortisol is an oxytocin antagonist. In a state of sustained fear, oxytocin is swept out, and mammals get aggressive. 
Desire for a hug in a state of stress is also related - hugs release oxyticin. Not surprising that usual profile of serial killers are people with awful childhood, which might have damaged their oxytocin-related neurochemistry and the template of other humans and animate beings.
The thinking machine statement on human soul and emotions is a translation from the novel of mine "The Truth" (Истината), first published in late 2002. It's available on-line, but only in Bulgarian.

  • In regard of man and machine relation, the "soul", and also consciousness as qualia are also a sort of "status symbols" for ones who "hate machines", but have not a better reason or can't understand well why. (Such as Mr. Searle, IMHO some of his claims are self-humiliating and ridiculous)

Humans, and particular individuals, are taught and are self-praising as "kings of the nature", and they're afraid of being "dethroned" as "the most special ones". When threatened, they are searching for proves, such as - machines are accused for "just doing what they are programmed/told to do", while "humans have free will, they can do what they want", so AGI is reduced to a simple machine.

  • The "doing what they are programmed to do" vs "free will" is confused in many ways. For example: 

A) It implies that humans have a soul or consciousness, because they *don't always know/can predict why they do what they're doing with a precision that they believe they should have known in case if they didn't have free will*

However the lack of appropriate knowledge is not free will, but less of sentience or even randomness. (I guess you're familiar with the paradox - if one has completely free will, causally unrelated to past and/or the environment, that means her will is random, and she's supposed not to have merits for her random choices)

B) In the same time, humans justify and define their free will with examples such as "if I want to do something, I can do it"

They have "choice", and there's "intention" - a match between desired and caused. See the first pages of "Universe and Mind 4" (2004) for the discussion on resolution of causality/control and how low it is in such cases.
While machines:

C) They just do what they are programmed to do, exactly, have no choice, no will, no soul etc. In fact even the simplest algorithm has "choice" if it has one single conditional operation.

Generally I suspect this "hatred" come from the perception of machines as "inanimate objects" and "not humans", "not like me, and I am the master". The attempts to justify it are rationalizations - see the already mentioned article - Rationalization and Confusions Caused by High Level Generalizations and the Feedforward-Feedback Imbalance in Brain and Generalization Hierarchies.

Point (B) is an illustration of the observation of a *match* between intentions and future sensory input - intentions are predictions, expectations for the future sensory inputs - including proprioceptive, which are related to motor outputs, muscular actions. 

- It's noticed also, that the match between predicted and observed is with a precision which is assumed to be high enough to accept that it is a display of free will (B).
- Yet it's noticed that sometimes precision is not high enough for ones' own actions (sometimes you may want something, but it may not happen) and especially when expecting and evaluating other agents actions - one can't predict that precisely others' behavior. However others  are recognized as "similar, like me", see the "soul" template above - and from their behavior/their model, implications are induced and reapplied to the model of self.
- Besides, the "machine hater" applies his vague and inarticulate knowledge about computers and inanimate objects on the thinking machines  -- Computers - you type in, it does your commands. Programming - you type orders, it computes. Algorithm - it executes exactly as it's programmed. It's constructed of 1s and 0s. Works without mistakes, like a machine (too high a precision) etc.

The latter confusion is not surprising, the notorious mathematician Lady Ada Lovelace, daughter of Lord Byron has made it too, but Alan Turing has given her a nice answer a 100 years later. See...
T. Arnaudov - Faults in Turing Test and Lovelace Test. Introduction  of Educational Test. In Todor Arnaudov's Researches Blog, 17/11/2007
http://artificial-mind.blogspot.com/2007/11/faults-in-turing-test-and-lovelace-test.html

For (A) the precision is decided by comparison between the precision of self-predicted actions (and presence of proprioceptive feedback and recognition of parts of self etc.), and the acceptably lower precision of prediction of behavior of other agents and the lack of proprioceptive feedback (other people's bodies future trajectories and utterances are harder to predict than our own body, sometimes there are unpredictable "glitches", caused by their "free will", meaning - unpredictable component of the model of their "soul" in the computational sense defined above).

Going to lower neural level, "will" is the execution of  muscular actions, initiated by sequences of neuronal activities in the motor cortices of the brain, which are in the roots of the "intentions" in the neocortex, and perhaps the initial source of sample data for this kind of multi- and inter-modal match.
The will as matching between intentions and future perceptions is encoded in the words' semantics either. Apperently in English as a particle for future tense. In Bulgarian for example, the particle for future tense, the future tense of the auxiliary verb съм/sum  (to be) - "ще" (shte - "wlll") sounds close  to "shta", that is a word meaning "want" in negating sentences:

- Не ща! - I don't want (it/you/...something/this/)!
- Щеш - не щеш, ще трябява да го направиш. -  It doesn't matter if you want or you don't want to do it, you will have to!

Also, the explanation of one's own behavior when other reasons are unknown usually is reduced to "I did it, because I wanted so!"

The "wanted" outcome is what's in the "intentions" and is matched to the really happened.
I believe the match between "intentions"(expected/predicted) and reality is a fundamental metaphysical concept of Universe and of mind. Check the "teenage era" works, Control/Causality units.

-- To be continued -- 

(C) Todor "Tosh/Twenkid" Arnaudov
Twenkid Research - http://research.twenkid.com/wp/

Last edited: 19/2/2012

Други тагове: Transhumanism, Трансхуманизъм, cosmism, космизъм, следващо, еволюционно стъпало, еволюция, мислещи, машини, thinking machines, artificial mind, technological singularity, сингулярност, пречупване, ера, трансхуманисти, transhumanists
Read More