by Todor Arnaudov
http://research.twenkid.com
http://artificial-mind.blogspot.com
The course was taught to undergraduate students between 1/2011 – 3/2011 at Plovdiv University “Paisii Hilendarski”, Bulgaria, in the Faculty of Mathematics and Informatics. (Originally in Bulgarian, with a lot of additional sugguested materials in English). This was the second AGI/UAI course after "Artificial General Intelligence/Universal Artificial Intelligence" (originally: "Универсален изкуствен разум") from last year; now putting stronger emphasize on the most advanced lectures on theories of intelligence and the common principles, meta-evoltution and Boris Kazachenko's works, now reviewed more thoroughly in class (as far as my understanding and students interest went).
You may find a lot of materials in English and links in this blog. There's somewhat sorted list by topic done for the students, but it might be partially incomplete, because it's not updated immediately with the blog posts. Students are advised to check out the blog for the topics from the first course, which were omitted in the formal program of the second one, such as other AGI researchers work and directions - there was too little time available in class...
I guess the next more updated course is supposed to go even more deeper into formal models, maybe starting with some real basic AGI agents.
I'm preparing to publish slides in English (you can find Bulgarian ones in the course homepage on the top) - especially slides and translation of the old works from my "[Teenage] Theory of Mind/Intelligence and Universe", written between 2001-2004, which years later was how I recognized the "school of thought" I belonged to (see the annotation below).
This course could be taught in English as well, if there's an appropriate demand/place/invitation.
Annotation:
Mathematical Theory of Intelligence
This course is addressed to students who wish to work in the novel interdisciplinary field of Artificial General Intelligence (AGI & UAI) which is building the theoretical foundations and research methodology for the future implementation of self-improving human-level intelligent machines - “thinking machines“ (AI was one of the predecessors of this field, but went into solving too specific problems). The course introduces students to the appropriate foundations in futurology and transhumanism, mathematics, algorithms, developmental psychology and neuroscience in order to finally review some of the current theories and principles of general/universal intelligence from the “school of thought” of researchers such as Jeff Hawkins, Marcus Hutter, Juergen Schmidhuber, Todor Arnaudov and Boris Kazachenko.
Course Program: (as of 11/2010) (Syllabus)
1. What is Universal Artificial Intelligence (UAI, AGI, „Strong AI“, Seed AI). Technological Singularity and Singularity Institute. Transhumanism. Expected computing power of human brain. Attempts for literal simulation of mammalian brain. "Universality paradox" of the brain. Ethical issues, related to AGI.
2. Methodological faults in narrow AI and NLP (Natural Language Processing), reasons for their limited success and limited potential. Review of the history of approaches in (narrow) AI and its failures and achievements up to nowadays. Concepts from AI that are prospective and still alive in AGI, such as probabilistic algorithms, cognitive architectures, multi-agent systems.
3. Mathematics for UAI/AGI: Complexity and information theory. Probability Theory – statistical (empirical) probability. Turing Machine. Chaos Theory. Systems Theory. Emergent functions and behavior. Universe as a computer – digital physics. Algorithmic Probability. Kolmogorov's Complexity and Minimum Message Length. Occam's Razor.
4. Introduction to Machine Learning. Markov's Chains. Hidden Markov Models (HMM). Bayes Networks. Hierarchical Bayes' Networks and Hierarchical HMM. Principles of the algorithms of Viterbi and Baum-Welch (Expectation-Maximization). Prediction as one of the basis of intelligence.
5. Drives of human behavior - behaviorism. Classical conditioning. Operant Conditioning and reinforcement learning as universal learning method for humans and machines. Why imitation and supervised learning are also required for AGI.
6. Introduction to Developmental Psychology (Child Psychology). Stages in cognitive development according to Piaget, and opposing views. First language acquisition. Nature or Nurture issues and how specific cognitive properties, behavior and functions could emerge from a general system.
7. What is intelligence? Thorough review of Marcus Hutter's and Shane Legg's paper “Universal Intelligence: A Definition of Machine Intelligence”. Universal Intelligence of an agent as a capability to predict sequences of actions with maximum cumulative reward. Types of agents in environments of different complexity.
8. Beauty and Creativity as compression ratio progress in the work of Jurgen Schmidhuber.
9. Brain Architecture – functional anatomy of mammalian and human brain. Triune theory - evolution of vertebrate's brain. Neurotransmitters and hormones and their relations to emotions and behavior. Mini-column hypothesis and functional mapping of the neocortex. Attempts for biologically correct simulations of the neocortex such as the BlueBrain project.
10. Evolution in biological, cybernetical and abstract sense: genetic, epigenetic, memetic and its application in design of complex self-organizing systems. Review of Boris Kazachenko's work on meta-evolution as Abstraction of a conserved core from its environment, via mediation of impacts & responses by increasingly differentiated adaptive interface hierarchy. Entropy as equation and increase of order, not increase of chaos.
11. Introduction to the theory of Intelligence by Jeff Hawkins. Modeling the function of human neocortex – the Memory-Prediction Framework and the principles of operation of the Hierarchical Temporal Memory.
12. Introduction to the theory of Intelligence by Todor Arnaudov – mind as a hierarchical system of simulators of virtual universes, that predict expected sensory inputs at different levels of abstraction. Hierarchical prediction/causation of maximum expected reward, where correctness of prediction/causation is rewarding. The Universe as a computer and trend in the evolution of Universe (cybernetical evolution). Proposal for guided functional simulation of the evolution of vertebrates' brain, starting by a general cognitive module that is simpler than mini-column.
13. Theoretical Methodology of Boris Kazachenko. Generalists and specialists, generality vs novelty seeking. ADHD and ASD. Attention, concentration, distractions and avoiding them. Induction vs deduction.
14. Introduction to the theory of intelligence by Boris Kazachenko. Cognition: hierarchically selective pattern recognition & projection. Scalable learning as hierarchical pattern discovery by comparison-projection operations over ever greater generality, resolution and scope of inputs. Importance of the universal criterion for incremental self-improvement. Comparisons of greater power and resulting derivatives, and iterative meta-syntax expansion as means to increase resolution and generality. Boris Kazachenko's Prize for ideas.
15. Summary of the principles of general intelligence in the works of Jeff Hawkins, Marcus Hutter, Juergen Schmidhuber, Todor Arnaudov and Boris Kazachenko: incremental [hierarchical] accumulation of complexity, compression, prediction, abstraction/generalization from sensory inputs. Evidences and real-life examples for the reliability of this principles.
16. Practice in introspection and generalization. Expansion of the scope of cases, where cognitive algorithm is applicable.
17. Exam.
Update from 29/11/2011: Comments on the AGI email list of AGIRI:
John G. Rose ... via jeeves.archives.listbox.com to AGI show details Nov 24 (5 days ago) Great course programs covering AGI summary/introduction, I like the selection of topics discussed. You might consider opening these up online via streaming/collaboration in the future… John ...
Ben Goertzel ... via jeeves.archives.listbox.com to AGI, AGI show details Nov 28 (2 days ago)
Tosh
Looks like a great course you're offering!n FYI ... On your page you note that our 2009 AGI summer school didn't cover jeff Hawkins' work... I can't remember if any speaker mentioned hawkins, but, Allan combs gave some great lectures on neuroscience, which covered hierarchical processing in visual cortex among other topics ;) That AGI summer school presented a variety of perspectives, it wasn't just about open cog and my own views ... But it wasn't heavy on perception-centered AGI... Ben ...
Todor Arnaudov's answers: Thanks, John.
There are materials from the course online (on the blog and on the site); most of the lecture slides and details are only in Bulgarian yet, though. As of collaboration - maybe, as long as I manage to create a team, for the moment I prefer keeping the authorship for myself. ...
Thanks Ben! And thanks for the notes. :)
Ben>That AGI summer school presented a variety of perspectives, it wasn't just about open cog and Ben>my own views ... But it wasn't heavy on perception-centered AGI...
All I knew about the summer school was from the brief web page on your site: http://www.goertzel.org/AGI_Summer_School_2009.htm Hawkins wasn't mentioned in the program, and it sounded reasonable not to be, as he seemed from a distant "school of thought" compared to the lecturers' ones - as far as I knew or assumed theirs.
Ben>I can't remember if any speaker mentioned hawkins, but, Allan combs gave some great lectures on Ben>neuroscience, which covered hierarchical processing in visual cortex among other topics ;)
That's nice (I've noticed neuroscience in the program), but anyway I think HTM and the other sensorimotor topics are more general - memory-prediction framework and the other similar models are supposed/aiming to explain virtually all kinds of cognitive processes with an integral paradigm, and vision is just an example/case. In a POV of schools, there's a distinction whether it's suggested that vision is an example of a general framework, or it's one of the sub-architectures/sub-frameworks for an AGI.
