By Todor Arnaudov | 13 March 2004 @ 21:49 EET | 340 reads | First published at bgit.net and the e-zine “Sacred Computer”
Оригиналът на български: http://artificial-mind.blogspot.com/2008/02/2004.html или: Анализ на смисъла на изречение въз основа на базата знания на действаща мислеща машина. Мисли за смисъла и изкуствената мисъл. Една от основополагащите ми статии от тийнейджърските ми години. Тази е от късните, продължава с два дълги коментари, добавки, включващи и размисли върху принципи на ученето с подкрепление (reinforcement learning)
Included in “Unverse and Mind 4": a search of the meaning (http://eim.hit.bg/razum, in Bulgarian only yet) Finally I started to translate my old articles, including my teenage Theory of Mind and Universe - all were milestones of my AGI research. I wrote this particular one as a 19-year old freshman in Computer Science at Plovdiv University.
- Natural Language Processing, NLP.
- An example of a search through the linguistic knowledge base of the author.
- Many different meanings of "meaning" defined for different uses.
- Quasi-formal semantic analysis - from words to chunks (expressions) to clauses to complex sentences. A search for relations, links.
- Criticism of the usage of short ambiguous sentences as a way to explain the "impossibility" of creation of an AI and machine translation; the lacking context is filled by human imagination and could be filled with machine's imagination. A discussion about the artificial pruning of the set of interpretations that humans are doing when translating or interpreting, and the implicit denial of expecting unknown meanings.
- Thought experiment of how a 3-year old toddler boy interprets unknown sentence (tme flies) and how does he search for a meaning and maps meanings to his senses. It is told as a real story, what he knows, what he experiences and what he would experience if...
- The style in not really academic, there are some dialogues, discussions with a virtual opponent, when this is appropriate to display human biases.
- Others...
- Continues with long comments with additions about reinforcement learning and other topics (to be translated and linked)
Part 1 (и български):(This Post) Semantic analysis of a sentence. Reflections about the meaning of the meaning and the Artificial Intelligence
Part 2 (и български): Causes and reasons for human actions. Searching for causes. Whether higher or lower levels control. Control Units. Reinforcement learning.
Part 3 (и български): Motivation is dependent on local and specific stimuli, not general ones. Pleasure and displeasure as goal-state indicators. Reinforcement learning.
Some of the conclusions:
- More complex (smarter) the system – More Meaningless the Meaning
Because the interpretation depends more on the system.
- „What is possible to be done with” is yet another meaning of the concept of “meaning”.
Happy reading and enjoy the story of the little Johny and the flies that are flying around the watch... ;)
This article starts with the definition of “meaning” of a friend of mine, Ilian Georgiev, we shared thoughts an year ago.
Meaning/Sense (Ilian Georgiev's definition):
Meaning of a sentence (a thought) is a function that searches for controversy between the knowledge base of the evaluator (the one who thinks) and the thought being analyzed. All elements from the knowledge base and their connections are juxtaposed with the new thought, the one that is analyzed. If any of the elements of the thought has a connection with another element, and the connection cannot be found in the knowledge base, then the sentence is classified as a non-sense (has an error).
Using this definition, let's search for the meaning of a weird sentence, that I made up don't know how. I'll use also a made-up semi-formal syntax of NLP analysis, that you'll grasp on-the-fly.
(The original paper is in Bulgarian, and the author is not a native English speaker; it is possible that there are some mistakes in some of the senses used.) SENTENCE = “The cat drank the stone and flew out under the uphill.” ИЗРЕЧЕНИЕ == "Котката изпи камъка и литна под нанагорнището."
SENTENCE == SENTENCE_1 + SENTECNE_2 (clauses)
SENTENCE_1 == "The cat drank the stone"
- Agent_1 == The cat
- Action == drank
- Object == the stone
SENTENCE_2 == "Литна под нанагорнището".
- Agent + Preposition + Object --
- Agent_2 == Agent_1 - Preposition == under - Object == the uphill Basic syntactic parsing check passes. Moving to the Semantic Analysis.
Semantic Analysis
This is a quasi-formal sample semantic analysis, using author's personal linguistic knowledge base An improvised approach 1. Word-by-word semantic analysis "Котката изпи камъка и литна под нанагорнището."
The cat drank the stone and flew out under the uphill.
Do I know what does mean:
-- the cat --
:: YES, a defined “cat”, I know what “cat” means
-- drank --
:: YES, pass tense of the verb “to drink”
-- the stone --
:: YES, Noun, ...
-- и --
:: YES, Conjuction
-- flew out --
:: YES, Verb, past tense
-- under --
:: YES, Preposition
-- the uphill --
:: YES, Noun
This stage passes, connections are found in the knowledge base.
2. Chunks, Expressions, Multi-word Semantic Analysis
2.1. Two words, except cases where the first word is Noun and the second is a Conjunction.
– the cat drank ...- YES (AGENT + VERB)
-- (Someone) drank the stone - YES (AGENT + VERB + NOUN/OBJECT)
-- and flew out -- YES (CONJUNCTION + VERB)
-- flew out under -- YES (VERB + PREPOSITION)
-- under the uphill -- YES (PREPOSITION + NOUN/OBJECT)
Passed.
3. Semantic analysis of the clauses and the whole sentence.
"Котката изпи камъка и литна под нанагорнището."
Let's analyze all the clauses, this would give clues for the meaning of earlier clauses and the whole sentence.
„The cat drank the stone” Is "the cat" linked to the verb “to drink”? - YES. What are the links? - Usually “to drink” is linked with objects, which are linked to liquidity or semi-liquidity of a substance. In general, in the definition of an object, that is linked to the verb “to drink”, usually there's a morpheme or semantics of liquidity.
“Liquid” is linked to:
1. Flow
2. Run
3. Stream down
4. Roll down
5. Trickle down
6. Pour out
7. Pour into
8. Infuse
...
„To Drink” is linked specifically with:
1. To drink + water.
2. To drink + juice.
3. To drink + tea.
4. To drink + cold tea.
5. To drink + alcohol.
6. To drink + beer.
7. To drink + wine.
8. To drink + scotch.
9. To drink + vodka.
10. To drink soup.
...
The examples are checked easier if they are put down in a unified way: Verb + Object/Noun for quick comparisons. I don't know whether to check the general concepts/meanings first (like liquidity), or after all specific cases are checked first (as colocations, like “drink vodka”).
„Drink” is used in some other cases, where the object is not a liquid.
1. Drink some poison.
2. Take a pill. (Take == Drink)
3. Take a medicine. (Take == Drink)
In this cases the object is:
1. On the surface of a liquid.
2. Floats in a liquid.
3. Dropped in a liquid.
4. Sunk in a liquid.
5. Absorbed in a liquid.
The most common meaning of “to drink” is linked with an AGENT which is a living being. Living beings have a throat, where the drunk object passes. The liquid assists the object to pass through the throat, when the object is not a liquid itself.
I recall an idiom (Bulgarian, this is a literal translation)
"A duck has drunk his sense."
So, what about the linkage between “to drink” and the object “stone”?
Stone has direct links to verbs as "to throw” and “to crack” and similar, “to kick” and others.
Direct links are examples of usage which I have ever encountered in texts or speeches.
„Stone” can play different roles: AGENT (subject) or an object, that clarifies/specifies an ACTION, done by another AGENT.
Is it possible to drink a stone?
One can assume also, that if something is said, then this is a special kind of stone. If a meaning should be found in any price, it is possible for the searcher to invent meaning that matches the given sentence. It is possible also to add additional sense, using experience, so that the sentence that is “meaningless” up to now to get its explanation. This point will be discussed again later.
STONE, without a preposition
1. Kick a stone
2. Throw a stone
3. Push a stone.
4.... Lift, Roll, Hit, Crack, Break, Catch, Leave, Heat, ….
Etc... but “To drink a stone” is lacking.
However, this doesn't make the sentence meaningless, yet, because it was found above, that “drink” could be linked to objects which are not liquid, and in this sense, “to drink something” is a reference to “to swallow something” (to pass it through the throat) Drink == Swallow
Therefore: “The cat drank the stone” == “The cat swallowed the stone”
Therefore this clause makes sense according to my KB. Next clause: "...flew out under the uphill" Cat + fly?... No basic connections... Cat + jump, scratch, bite, drink, eat, … push, walk, run, fall, hide, pull, climb, jump over, hit, meaow, stalk, ... „Jump”, “Jump over” and “fall” captured my attention:
"Jump” is an action where the AGENT reaches to a state, where its body doesn't touch the ground.
It is the same for “Fly”. Therefore, “to jump” partially covers the meaning of “to fly”. Besides, I know examples of “flying” where to fly is used with the sense of “to jump” - directly or implicitly suggested by the context.
“Air Jordan” (in Bulgarian - "Въздушният Майкъл Джордан.") The basketball player Michael Jordan jumps and stays in the air for long enough to impress people more than the typical jumpers, this has caused his jumps to be linked with the morpheme “air”, which is used in words for flying (airplane, air force).
Therefore “(the cat) flew below the uphill” can be interpreted as:
(the cat) jumped below the uphill. Now, is it possible “jump” to be used with “below”. Can you jump below? To.. - Jump over s.t. - Jump into - Jump out
...
No “jump below”, but it doesn't mean that this expression is meaningless. (Actually there is jump below something, but say, not jump below the uphill) „Jump” has other meanings, like: doing something faster than usual, or moving fast. So... “[The cat] flew below the uphill” may mean:
[The cat] jumped below the uphill? Is it possible to jump below an uphill? Why not?
Uphill is an object, it can be located in а mountain, but we can imagine it to be any other object, over which somebody can move “up”, walking on. This “uphill” object can be made of wood or metal and can have a hollow inside, where a cat can hide.
The cat jumped under the thing, that had an uphill over itself... The whole sentence can turn to: "Котката изпи камъка и литна под нанагорнището." "The uphill had a hollow inside. The cat swallowed the stone and jumped below it. The additional clause fills up the uncertainty in the scenario.
Another interpretation, based on “to fall” could turn: "...flew below the uphill" to “..felt below the uphill". Also, “below the uphill” may mean below the part of the uphill, that is steep, i.e. just before the uphill starts to climb. Then: "The cat swallowed the stone and felt below the uphill." This gives a rise of another interpretation – stone is often linked with “heavy”. There is a proverb (Bulgarian) “Hang a stone on my neck”. Then: "The cat was climbing the uphill, but it swallowed the heavy stone - it threw it down below the uphill...” Or: While the cat was climbing the uphill, weird little balls felt down from the sky. They seemed like meet balls and smelled the same way. The poor cat was tired of hunger and the hard walk, and she bit one of the sky meet balls. A moment after she was frightened – the meet ball appeared to be as heavy as a stone. The cat was rolling down, until she stopped on the flat land under the uphill.
When searching for a meaning in very short pieces of information, such as single sentences, it is expected for the mind to invent, to imagine in order to fill up what is unknown with probable sets of circumstances. If the source doesn't deny, we can invent any plausible imagined circumstances. Short sentences as a way to deny creation of AI Actually, one very rarely meets single sentences in the reality, out of context to constraint and direct translation or interpretation of the meaning. However, short “nonsenses” – which one cannot interpret unambiguously or are often used to disprove the possibility of creation of an AI. Let's check out a classic from the NLP. Time flies. "It's so hard to translate to another language!”
How would we translate it in Bulgarian? "Времето лети" (Vremeto leti - The time is flying) or „Времеви мухи" (Vremevi muhi - Flies which are related to time)
Or another way - "time" doesn't mean only “time”, and “flies” doesn't mean only the little flying bug. These are just the first two items that came up to my mind! The search was obviously had been pruned up to two items, two possible interpretations.
This kind of unconscious pruning, limitation of the number of variants, will be discussed below.
Virtual Colleague: “The time is flying” is the correct translation. There's neither such an expression as “flies, related to time”, nor any other.
Author: Why do you think so?
V. Colleague: The other translations don't make any sense. Me myself, I would translate it that way, I think I'm good enough in English. Check out my personal web page.
Author: And why would you translate it that way?
Colleague: Because... I haven't heard of an expression meaning “flies, related to time”...
Author: Therefore you have excluded the possibility to hear a new sentence, where this expression is used in a meaning that was unknown for you before?
Colleague: Well, I think so...
Author: Who did tell you that sentence?
Colleague: I'm not quite sure... You? But... Well... It is possible that this is an idiom. Can you explain it to me? Maybe this is a special kind of flies? Or more likely... (What a SF fan I am not to guess this one!) Flies through time! Time travel! The ambiguity is caused by the lack of a criterion for pruning. Until the moment when an action is executed – an action caused by the input data, which are said to be ambiguous – the ambiguity is not an issue. The system can remember the whole sentence, word-by-word and until the moment of action, a decisive single action, it is known that all interpretations are possible.
And when the action should be done, e.g. a robot to capture the right cube or the middle cylinder – then the system should use an additional feature in order to disambiguate, to choose. However, since the input is not decisive, but ambiguous, then turning any of the interpretations to action is not a “mistake”, regarding the input.
Perhaps the Natural Language, or as the author calls it – The Language of Mind – allows ambiguity, because there are many “correct” possibilities. There are many cases, where each of the possible solutions/interpretations is “right” in the sense that the device (human) who took the decision continues to function after executing in effect, for real, an action, caused by the given interpretation.
So, if a given system continues to function – according to a given definition of “functions”, e.g. its heart continues to work at least for so-and-so long period after executing the given action – then this action was “right”, i.e. this action is assumed to have had followed laws that don't lead to malfunction. More complex control units have larger space of correct decisions, they have wider “freedom”, i.e. possibilities for future actions, after which they will continue to function right.
(see … @ quote “Conception for the Universal Predetermination”, a.k.a. “The Universe-Computer” or “The Mind and Universe”)
По-сложните управляващи устройства (виж "Схващане за всеобщата предопределеност": http://eim.hit.bg/razum).
I think that everything makes sense, colleagues. One can always find meaning/sense, i.e. a connection between items. The meaning is the connection between things. (The relations between things)
The easiest thing to do is to redraw an already known, drawn line, and this is what is done initially when one is doing a semantic check – whether precomputed links/connections/relations with the given expression do exist. If such links do exist, they are seen like “gray lines”, which mind can darken, one can do it when given a piece of paper with gray lines and is being told to draw lines without thinking a lot or planning. This is what the virtual colleague did above, he rejected the possibility that “time flies” has meanings, that are yet unknown to him, and need to be computed, “drawn” in his memory. Let's overview a case with the same sample expression in another case.
Time Flies... A three year-old little native English speaker – Johny. He knows, that “a fly” means the flying bug (something little, black, that is flying and when it land on your face it's !!! гъделичка and you're trying to let it go by waving your hands.
Johny knows how to create a multiple of fly – flies, but he doesn't know that “a fly” means also “a fly of an airplane”. For Johny, “time” means just “a watch”. Johny knows, that “a clock” and “a watch” have similar meanings – something circular, with a long things, that are rotating... and the longer things are rotating faster than the shorter and the thicker; the thicker ones sometimes appear not to move at all, but after you have played for a while with your toy cars and look to them – they seemed to be at another place...
All the times when Johny has heard talks about time, he has seen clocks or watches.
Johny has heard his father saying “I don't have time, we have to hurry up!” and when his father has told that, he has looked to his watch.
That way, the conception of “time” is linked to the image of “watch”, when hearing time, he sees a watch, no abstract concepts. Johny himself doesn't have a watch.
Now let's assume that we put on our hand a big and shiny colourful watch and go to play with Johny on the playground. What he is going to do, if we tell him “Time flies!” and he hears this for the first time in his life?
Colleague: Perhaps he will look to our shiny watch and will search for flies around it...
Author: Exactly! Can you imagine what he would do if we didn't have a watch on our wrist? Colleague: Maybe he would look to our hand, searching for a watch and flies... If he has remembered the pattern of watches being on the left wrist, he may first check there, or he may check both...
The images Johny has for “time” and “flies” are recalled, and Johny searches the expression of these images in the environment, accessible by his senses. The specific mean is not important.
The machine needs an external environment, where to search for meaning and senses – MATCHES of images, names, features, coincidences, patterns.
Author: What Johny is going to do, after realizing that there are no flies around the watch, or even there is not a watch? Colleague: It depends what behavioral models have been developed so far. He could remember the expression "time flies" as an image that represent what he has thought then - “flies which are flying around a watch”, but not to do anything further; Johny could wait, expect to face usage of this expression in an environment, which is richer of details and features, so that he would be able to extract or approve the meaning.
Details, specific cases are what limits the space of search, the domain. Details are forces for pruning...
Author: Johny may also not make any conclusion, but just taking the expression as a non-sense so he wouldn't remember it. Also, he can ask us immediately: - What does “time flies” mean? The machine should also be able to do like that, as we do, it will need teachers and supervisors, while it develops.
Our explanations and the degree of trust he has to what we explain to him will determine how the child is going to limit the space of search, but also how he will expand the space, by adding possibilities which he didn't thought of before.
If one explains to Johny, that “time flies” means “time is never enough”, the child may remember this explanation as a whole sentence, without interpretation. Just a reference, a link: “time flies” redirects to “time is never enough” and then he would search for a meaning for the new sentence. On the other hand, one can also explain to Johny, that “flies” means also “to fly”, to move like birds or like Superman or so (recall that he didn't know the verb; it's strange not to know it, but that's the assumption), but not explaining him about the abstract concept of time.
In this case, Johny could keep linking “time” to “watch” and may start to imagine “time flies” as “the watch flies”. He may look around, searching for a watch that is flying – generally, this is a search of features, input data/senses which could confirm the link that was made. Or... just anytime he hears “time flies” Johny would imagine a flying watch and would ask himself “Whether the flying watches have wings or they are magically flying?”
Imagination is a the point here. When searching for a meaning, we should be able to imagine, to fantasize. That means, one kind of inputs/senses to cause other kinds of inputs/senses. The primary may be “real”, taken from raw data from the reality, linked to what the machine or human takes for “Reality”; while the secondary input/sense could be imagined, fantasized, unreal.
Talking about reality, humans usually take for “real” input channels such as vision, hearing, touch, taste, smell; when he is receiving data at the maximum possible rate (max resolution, raw data).
Vision is a primary sensory input, when we're sensing images, where we can recognize individual pixels. Letters, numbers and any symbols come from a secondary sensory input, because from the primary sensory input, containing raw pixels, each one containing an independent value, are extracted data with a smaller size (in raw bits) – letters, digits, geometric shapes etc.
...
In order a system to find a meaning and make sense of things, it is very useful the system to have at least two different kinds of sensory inputs, and each of them to be able to invoke, to link to the other one. The images and relations between images (e.g. motion); sounds and relations between them can “generate” words: interpretations, which are described with smaller quantity of bits.
When Johny hears “a watch”, he can imagine, somehow to see the image of the watch and what is possible to be done with it.
„What is possible to be done with” is yet another meaning of the concept of “meaning”. There is no sense in meaning, if you can't do anything with it.
Actually, everything “makes sense”, or “has a meaning”, in the sense that it causes something to be done. It is so, because even the so called “non-procedural knowledge/data” are only “non-procedural”, non-active, in the sense that they are not causing a type of action that is formally defined as “procedural”. In a computer, an information processing system, any data cause actions. Data defines what happens in the machine's “mind”. For example, the descriptive data of a page of text determines what exactly the machine is supposed to do, while it's CPU is reading the data from the memory, processes it, displays it, prints the page.
The meaning is the action, that the “thing” which is evaluated invokes/causes/turns, and this meaning can be different, depending how deep we are searching and more – how and what.
How and What we do the search is more dependent on our past experience, than on the short input being evaluated. Finally... Let's assume that the meaning of a message/sentence is an action to be taken, assuming that it was caused by the meaning that was found (if another meaning was found, another action would be taken).
Higher the complexity of the system, higher the weight of memories/experience in the decision how and what to search, therefore – what could be found. It's a paradox, but:
More complex the system – More Meaningless the Meaning
The Smarter the system – the Meaningless the meaning
Because the system - the Artificial Intelligence or human - can more freely search and find meaning – links between the items, things, phenomenon, events, messages, memories, objects, images, sounds or whatever.
Continues with long comments after the article, about reinforcement learning and other topics... (to be published, when translated)
http://artificial-mind.blogspot.com
http://eim.hit.bg/razum (Bulgarian)
...
This is my favorite picture from the time... :-P
