Montreal.AI Academy Cheetsheet

A great summary and selection of links:

http://www.montreal.ai/ai4all.pdf?fbclid=IwAR25GS_VstvkQ9T5yc8LpgkjHHevxMfJwkvgLtk_DWLXXlKXuI7-8CVgSvg

Reinforcement Learning Study Materials

Selected resources (to be extended)

https://sites.google.com/view/deep-rl-bootcamp/lectures

The Promise of Hierarchical Reinforcement Learning

+ 15.3.2019

Учене с подкрепление/подсилване/утвърждение - езикови бележки | Terminology Remarks on Reinforcement Learning Tutorial in Bulgarian

Чудесни практически уроци за начинаещи от С.Пенков, Д.Ангелов и Й.Христов в Дев.бг:

Част I

Част II

Някои терминологични бележки от мен, ДЗБЕ/ДРУБЕ 

"Транзиционна функция" - функция на преходите, таблица на преходите.

В лекциите от курса "Универсален изкуствен разум" ползвах "подсилване" (може и подкрепление) и подчертавах произхода на термина от поведенческата психология ("бихевиоризъм").

Има връзка и с невропсихологията (действието на допамина) - награждаващите поведения се подкрепят/подсилват (и "затвърждават") и затова в последствие се изпълняват "с по-голяма вероятност". Тези, които не носят полза, не се подкрепят.

В друга литература обаче също ползват "утвърждение".

На езиково близкия руски също е "подкрепление".

http://research.twenkid.com/agi/2010/Reinforcement_Learning_Anatomy_of_human_behaviour_22_4_2010.pdf

https://en.wikipedia.org/wiki/Reinforcement  (виж други езици)

http://research.twenkid.com/agi/2010/Intelligence_by_Marcus_Hutter_Agent_14_5_2010.pdf

2. Част:

(...)

Имплицитен - неявен;
Произволни - случайни

(За числата, Random така се води на бълг. в статистиката; произволен достъп - за памет, но там има "съзнателна воля", избор)

Семплиране, семпъл - отчитане, отчет;

Терминално - крайно, заключително;

Еквивалентно - равнозначно, равносилно.

Мисля че за "пристрастно" зарче си има термин, но не се сещам.

Стилистична бележка - "Ако е," не звучи добре (предполагам превод от If it is such/so); може да се повтори условието, за да е най-ясно, или "Ако е така" или да се преработи предното изречение.

Корекции, които забелязах: "фунцкия" и "теоритично" (теорЕтично).

И накрая: би било полезно, ако има таблица или отделна страница с термините, с различни варианти на преводите, когато има разногласия.

Илюстрации на Монте Карло и постепенното получаване на по-точни резултати - все по-малко шум и по-ясно изображение:

https://www.shadertoy.com/results?query=+monte+carlo

https://www.shadertoy.com/results?query=global+illumination

Спомени за българската високотехнологична индустрия от фото архивите | Memories from the Bulgarian High-Tech Industry

Снимки от производството и настройката на компютри, електроника и други машини и техника от българския държавен архив от 50-те до 80-те години.

From the Bulgarian State Archives, from the 50-ies to the 80-ies.

http://www.archives.government.bg/bgphoto/004.08..pdf?fbclid=IwAR0WWNWCCaa41ZXVUSu8himrnqAC_fmIZ25REJ8OHP4DAlD_THpkNzTDWpw

Thanks to D. from the Compu  computer museum.

On the paper: Hierarchical Active Inference: A Theory of Motivated Control and Conceptual Matches to Todor's Theory of Universe and Mind

Comment on:

From "Trends in Cognitive Scinece", vol.22, April 2018.An opinion article 
Hierarchical Active Inference: A Theory of Motivated Control

Giovanni Pezzulo, Francesco Rigoli, Karl J.Friston
https://doi.org/10.1016/j.tics.2018.01.009

It's an excellent paper, would be insightful and accessible for beginners in AGI, psychologists and for seeing the big picture like "On intelligence" etc. and for readers who like divergent thinking and seeing mappings to real agent behavior and "macro" phenomenons. Good questions, huge set of references, mapping to brain areas and nueroscience research.

However as of architectural, philosophical ideas it sounds too similar to my own"Theory of Universe and Mind", published in articles mainly in the Bulgarian avant-garde e-zine "Sacred Computer" (Свещеният сметач) between late 2001 and early 2004. Its ideas were presented/suggested also in the world's first University course in AGI in 2010 and 2011.

Thanks to Eray Ozkural who was familiar with Friston's work and we had an interesting discussion in Montreal.AI FB's page, see a recent post regarding his work in "Ultimate AI" and "AI Unification".

The term "active inference" sounds pretentious, it means using a model of the world in order to act, I assume in opposite to being simply reactive as in simpler RL models. However IMO that's supposed to be obvious, see below.

Theory of Universe and Mind

The terminology and content of that 2018 "opinion" paper strongly reminded me of the teenage writings of myself from the early 2000s. The term "control" (the cybernetics influence), the need of prediction/reward computation at different time scales/time steps, cascade increment of the precision (resolution of control and resolution in perception); specific examples of "behaviorintrospective" analysis and specific selection of the actions etc.

"Theory of Universe and Mind", or "my theory", started with the hierarchy and "active inference" as obvious requirements (not only to me, I believe).

Mind is a hierarchical simulator of virtual universes, it makes predictions - controls ("cause" is a better term, though) at the lowest level. The hierarchical simulations are built from the patterns in the experience. Highest levels are built of sequences of selected patterns at lower level ("instructions", correlations) which are predictive.

At the lowest level all combinations of instructions are legal, the system shouldn't hang.

However at the higher levels, only selected ones work, not all combinations of low level instructions are correct which makes the search cheaper. That implies reduction of the possible legal states, which as far as I understand in F.'s terms is called "reduction of the free energy". 

So the mind, the hierarchy of virtual universes, makes predictions about the future in the Universe - as perceived at the lowest level virtual universe - and causes desired changes in the input, by aiming at maximizing the desired match. 

Through time it aims at increasing the resolution of perception and causality-control while increasing also the range of prediction and causation. That's what a human does in her own personal development, as well as what the humanity's "super mind", the edge of science and technology.

My old writings were also explicit about the predictions at different time-scales, precisions and domains - for a sophisticated mind there's no one single "best" objective behavioral trajectory, there are many, because there are contradictory reward-domains (like not eating chocolate, because it may lead to dental cavities, or eating it, because it's a pleasure now).  There's also a prediction horizon, uncertainty.

In the domain of Reinforcement learning, there are two types of reward, called "cognitive" and "physical". Cognitive is about making correct predictions, that is "curiosity", exploration etc., while physical is about having the desired input in the senses, implying a desired state - or "pleasure".

There must be accord between these domains and a sophisticated enough hierarchy and various time-space-precision ranges, otherwise the system would fall into a vicious cycle and have an "addiction".

In the paper, they have called my cognitive reward/drive "cold domain" (choice probability, plans action sequences, policies) and my "physical" one - "hot domain" (homeostasis, incentive values, rewards).

Etc.

...

The "Theory of Universe and Mind" works and the 2010's slides could be found in this blog, in the online archives of "Sacred Computer" (Свещеният сметач - the original texts in Bulgarian), and on twenkid.com, http://research.twenkid.com/agi/2010/ 

US Government Officially Declares AI as a Priority

In order to maintain the "economic and national security":

https://www.whitehouse.gov/articles/accelerating-americas-leadership-in-artificial-intelligence/?fbclid=IwAR33oWgvHXnzmfDrLLx3Qg01KmpLyyPQ8Oy2hZYLu4VK1PRPQSnaC6J2AMU

DARPA's Common Sense Reasoning Program

Bottom-up and top-down reasoning should meet in the middle.

DARPA and others seem to have realized that, as well as the importance of Developmental Psychology.

https://www.darpa.mil/news-events/2018-10-11


Redirected from: https://hbr.org/2019/01/the-future-of-ai-will-be-about-less-data-not-more?utm_campaign=hbr&utm_medium=social&utm_source=twitter


See a table with AGI/AI startups, key people and directions:

https://artificial-mind.blogspot.com/2016/06/agi-start-ups-and-research-institutes.html


...


Interesting AI news digest: http://artificial-intelligence.startupdigest.com/issues/future-of-ai-china-s-facial-emotional-recognition-weaponization-of-ai-more-154170

Origin of the Term AGI - Artificial General Intelligence

The fellow AGI researcher and developer Peter Voss told the story in his blog in Feb 2017:

What is AGI
https://medium.com/intuitionmachine/what-is-agi-99cdb671c88e?fbclid=IwAR1rm684qRx-oMZeyKWvioJDbhZT7YvVIp7HSZYpJAjMDsxN5lyow9T5XYM

"Just after year 2000" Voss and a few other researchers realized that there were hardware and scientific prerequisites to return to the original goal of AI. At the time they found about "a dozen" other researchers who were involved with research in that direction. Peter, Ben Goertzel and Shane Legg selected the term "A-General-I", which was made official in a 2007 book written together with the other authors: "Artificial General Intelligence".

https://www.springer.com/gb/book/9783540237334

(According to the info on Amazon, published Feb 2007)

I've encountered Ben Goertzel's thoughts about that early 2000s official embarking of the AGI as an idea again by himself and his friends and that the term was coining in the early 2000s, I started to use the term after influence by his circle. I haven't read the book, though.

I had a related terminology story also from the early 2000s, which I may tell in another post.

Other interesting AGI-related articles from Peter's blog:

https://medium.com/intuitionmachine/agi-checklist-30297a4f5c1f

https://medium.com/intuitionmachine/cognitive-architectures-ea18127a4d1d

CogAlg News - Boris Declares a Lead Developer

The "Cognitive Algorithm" (CogAlg) AGI project of Boris Kazachenko has found a new talent, which for a month of work is listed as "lead developer", according to the Contributing list in github

Good luck and looking forward for results!

I'm attached to this project in many ways, although I don't like some aspects of it.

AFAIK I was the first researcher-developer with credentials to acknowledge Boris' ideas back in time (since I found they matched "my theory").

Then I tried to promote him, his theory was suggested and presented in the world first University courses in AGI in 2010 and 2011*.

His prizes were first announced in the comments section of a publication of the AGI-2010 course program.

I've been the first and the only recurring prize winner so far with the biggest total prize for more than 8 years.

---

* In comparison, the artificial neural networks were given 3 slides in the lecture "Narrow AI and Why it failed". :) Now DNN achieved a lot, but it's still "narrow AI" without understanding and structure (not a "XAI", explainable/interpretable), poor transfer learning, shallow etc. Other authors have already defined extensively ANN's faults, such as Gary Marcus and in a more concise and his-theory-related form - Boris himself.

My slides:

The 2010 and 2011 course had only three slides specifically about ANN, only in one of the introductory lectures about “Narrow AI” and why it failed.

See slides 34–35:

http://research.twenkid.com/agi/2010/Narrow_AI_Review_Why_Failed_MTR.pdf

Translated from Bulgarian it says that ANNs:

* Pretend to be universal, but it is not working so far
* Represent TLU — threshold logic units
* They are graphs with no cycles, having vertices with different weights
* Input, output and hidden layers
* They are trained with samples (e.g. photos), which are classified by altering the weights
* Then when a new photo is fed, it’s attributed to a particular class
* Computationally heavy, holistic, chaotic
* Can’t work with events and time relations
* Static input

Slide 36, Recurrent NN, a more positive review:

Рекурентни невронни мрежи

● Мрежи на Хопфийлд. Асоциативна памет.
● Има цикли в графа – биологично по-верни.
● Започват да работят с понятие за време и събития.
● Long Short-Term Memory (LSTM) – Jurgen Schmidhuber – представител на Силното направление, опити за универсален ИР.
● Приложения: Разпознаване на ръчен почерк; робот, учещ с подкрепление в частично наблюдаема среда; композиция на музика и импровизация и др.

Developmental Approach to Machine Learning? - article by L.Smith and L.Slone - Agreed

Yes, agreed. A good read, suggesting developmental machine learning, spatio-temporally continuous input data etc.:

See the concept of “shape bias" from Developmental psychology. That's related to  discussions in the "AGI Digest" on recognition of "buildings, chairs, caricatures" ... and other articles from this research blog, regarding 3D-reconstruction at varying resolution/detail as one of the crucial operations in vision published in this blog and the general developmental direction which is driven from one of the very fist articles here about the "Educational test".

HYPOTHESIS AND THEORY ARTICLE

Front. Psychol., 05 December 2017 | https://doi.org/10.3389/fpsyg.2017.02124

A Developmental Approach to Machine Learning?

Linda B. Smith* and Lauren K. Slone

• Department of Psychological and Brain Sciences, Indiana University Bloomington, Bloomington, IN, United States

• 
  
• See also:

• Sunday, December 31, 2017

  CapsNet, capsules, vision as 3D-reconstruction and re-rendering and mainstream approval of ideas and insights of Boris Kazachenko and Todor Arnaudov

  An excerpt from an email from AGI list digest:
  
  

  Todor, 27.4.2012:
  
  "Invariants" in human fully functional vision are just those 3D-models (or their components, "voxels:) built in a normalized space, ...
  
  Every 3D-model has a normalized position per its basis, and also some characteristic division of major planes and position between the major planes, and there are "intuitive" ways to set the basis --> gravity/the ground plane foundations, which is generalized to "bottom", i.e.: ...
  
  

  As in other occasion - theoretically that seems pretty obvious to me , though.
  
  The way of constructing them robustly and "generally" are the problems to be solved.
  
  

  Wednesday, November 14, 2007

  Faults in Turing Test and Lovelace Test. Introduction of Educational Test

  
  

Human-centered AI by Stanford University - 8 years after Todor's Interdisciplinary Course in AGI in Plovdiv 2010

See: https://hai.stanford.edu/

Introducing the initiative - Oct 19, 2018:

"But guiding the future of AI requires expertise far beyond engineering. In fact, the development of Human-Centered AI will draw on nearly every intellectual domain"

The world first interdisciplinary course in AGI in Plovdiv University started in April 2010 and was proposed as an idea to my Alma Mater in December 2009.

Among the core messages of the course were the importance of the interdisciplinarity/multidisciplinarity and the suggested leadership in the research by such persons. I've been a proponent of that approach in my writings and discussions since my teenage years, being a "Renaissance person" myself.

See also the interview with me, published in December 2009 in the popular science magazine "Obekty"* after I have given a lecture on the Principles of AGI to general public in Technical University, Sofia for the European "Researchers's Night" festival.

I will Create a Thinking Machine that will Self-Improve (an Interview with Todor) 

          (...)

- Where do the researchers' efforts should be focused in order to achieve Artificial General Intelligence (AGI)? 

First of all, research should be lead by interdisciplinary scientists, who are seeing the big picture. You need to have a grasp of Cognitive Science, Neuroscience, Mathematics, Computer Science, Philosophy etc. Also, creation of an AGI is not just a scientific task, this is an enormous engineering enterprise – from the beginning you should think of the global architecture and for universal methods at low-level which would lead to accumulation of intelligence during the operation of the system. Neuroscience gives us some clues, neocortex is “the star” in this field. For example, it's known that the neurons are arranged in sort of unified modules – cortical columns. They are built by 6 layers of neurons, different layers have some specific types of neurons. All the neurons in one column are tightly connected vertically, between layers, and are processing a piece of sensory information together, as a whole. All types of sensory information – visual, auditory, touch etc. is processed by the interaction between unified modules, which are often called “the building blocks of intelligence”.  

- If you believe that it's possible for us to build an AGI, why we didn't manage to do it yet? What are the obstacles? 

I believe that the biggest obstacle today is time. There are different forecasts, 10-20-50 years to enhance and specify current theoretical models before they actually run, or before computers get fast and powerful enough. I am an optimist that we can go there in less than 10 years, at least to basic models, and I'm sure that once we understand how to make it, the available computing power would be enough. One of the big obstacles in the past maybe was the research direction – top-down instead of bottom-up, but this was inevitable due to the limited computing power. For example, Natural Language Processing is about language modeling; language is a reduced end result of so many different and complex cognitive processes. NLP is starting from the reduced end result, and is aiming to get back to the cognitive processes. However, the text, the output of language, does not contain all the information that the thought that created the text contains.

On the other hand, many Strong AI researchers now are sharing the position that a “Seed AI” should be designed, that is a system that processes the most basic sensory inputs – vision, audition etc. Seed AI is supposed to build and rebuild ever more complex internal representations, models of the world (actually, models of its perceptions, feelings and its own desires and needs). Eventually, these models should evolve to models of its own language, or models of human's natural language. Another shared principle is that intelligence is the ability to predict future perceptions, based on the experience (you have probably heard of Bayesian Inference and Hidden Markov Models), and that intelligence development is improvement of the scope and precision of its predictions.

Also, in order the effect of evolution and self-improvement to be created, and to avoid intractable combinatorial explosion, the predictions should be hierarchical. The predictions in an upper level are based on sequences of predictions (models) from the lower level. Similar structure is seen in living organisms – atoms, molecules, cellular organelles, cells, tissues, organs, systems, organism. The evolution and intelligence are testing which elements are working (predicting) correctly. Elements that appeared to work/to predict are fixed, they are kept in the genotype/memory, and are then used as building blocks of more complex models at a higher level of the hierarchy.

         (...)

* The original interview was in Bulgarian

As the colleagues at Stanford enumerate: their University was the place where the term AI was coined by McCarthy, where computer vision was pioneered (the Cart mobile robot; Hans Moravec), self-driving cars won DARPA Grand challenge in 2005, ImageNet, [Coursera], ... They are located in the heart of the Sillicon Valley, employ a zillion of the best students and researchers in CS, NLP, EE, AI, Neuroscience, WhatEver.

The Plovdiv course was created practically for free with no specific funding, just a regular symbolic honorarium for the presentation.

Note also that the course was written and presented in Bulgarian.

See also:
Saturday, February 24, 2018
MIT creates a course in AGI - eight years after Todor Arnaudov at Plovdiv University

...

The paradox is not so surprising, though, since most people and the culture are made for narrow specialists, both in Academia and everywhere. The "division of labor" etc. British and US wisdoms for higher profits in the rat race.

Thanks to prof. D.Mekerov, H.Krushkov, M.Manev who respected my versatility and especially to M.Manev who was in charge to accept the proposal of the course.

---

PS. There are other proponents of interdisciplinary and multidisciplinary research as well. I recall Gary Marcus from the popular AI journalists about; and of course as early as Norbert Wiener, if I'm not mistaken he explicitly suggested that. (The German philosophers such as Kant and Schopenhauer - as well...)

See my comment of a comment of Gary Marcus regarding Kurzweil's book:

Wednesday, January 23, 2013

(...) Domain Blindess Issues vs Truly General Versatile Intelligence. Comments on Kurzweil's Latest Book (...)