THE SACRED COMPUTER - СВЕЩЕНИЯТ СМЕТАЧ, a.k.a. ARTIFICIAL MIND: Thinking Machines, Creativity and Human Development. A pioneering virtual Research Institute for Artificial General Intelligence (AGI), Cosmism and Transhumanism, AI, Software, Interdisciplinary General Research, Creativity, Versatility, Universal Men ~ Twenkids etc. Created in 2000. (...)
Z***** 0.0001, watching Djokovic vs Federer match at Australlian Open 2020 on Eurosport. Art from my AGI Prototype, in this application it's in the form of computer vision.
Current name is still tentative since it's in "stealth", the old name of the system was "SuperCogAlg" - it was just a cheesy first pick due to "Super Contra" and "CogAlg" etc.
The latter name is also funny sounding abbreviation, again coined by me for BK's lengthy "Cognitive Algorithm" somewhere in 2015? during its pre-code era.
More recent visualisations from June, a completed primary bottom-up segmentation in the C++ version. Now working on deeper structures. I'm looking for partners and cofounders.
По-скорошни снимки от работата на прототипа на алгоритъм за универсален изкуствен разум, надграждащо се машинно обучение без учител (unsupervised learning). Засега изглежда като компютърно зрение и обособяване и разделяне на части ("клъстериране" и сегментация). "СуперКогАлг"* е на С++, за разлика от системата от която е разклонение - CogAlg, която е на Python.
*Супер... Има и друго име, но ще го обявя по-късно - засега така ми хрумна заради "Супер Контра", откъдето е кадърът по-долу.
Illustration of the scanning, segmenting and merging process - so called blob formations, the first level of the 2D-version of CogAlg.
A processed frame from the video game "Super Contra" and some segment-blob-like structures*, looking like Tetris, rendered by my work-in-progress implementation and environment for general incremental pattern discovery. Currently it follows some of the basic comparisons of the "Vanilla" version of CogAlg, but not exactly and probably will diverge and extend, since it's not supposed to be just a port.
It is also supposed to be interactive and "playable".
"The spirit of the video games" - now in color. :) The rocket-picture by the Vanilla CogAlg displays more advanced structures, though.
An Unreal Star Storm watched from the forests of the Rhodope Mountains, Bulgaria
The premiere of my new music video - a poetic and artistic production with beautiful 2D visual effects, produced using computer vision for automatic compositing, masks generation, objects removal etc. Edited and rendered using my inhouse software "Twenkid FX Studio".
Watch in darkness and on a big screen in 1920x1080!
"Twenkid FX Studio" is an endless "prototype" in which I've invested too little time and had to redesign a long time ago. Using Win32 sounds a bit insane, but my choice when I started was because there were issues with the usage of another "default" and having bad reputation simple windows library (MFC) - I used Visual Studio Express.
Sure, there were free GUI class-libraries, but I preferred a smaller code base that was not dependent on additional huge third-party libraries* such as wxWidgets (which was considered and maybe I was wrong not to develop with it).
Qt had some issues with the license - I didn't want my system to be GPL, and their other license fee was unreasonable. Maybe I could try GTK, but it's also bloated with a lot of dependencies and verbose method calls, similarly to wxWidgets, so apart from being multiplatform, I don't know would it be "simpler" to work with than Win32 or my own Win32 wrapper.
Furthermore, at the time when I started, FFmpeg or other Linux video libraries seemed undocumented/unaccessible, while I found a windows' one, although a bit outdated - VFW (Video for Windows). DirectShow was the more appropriate choice, but it seemed to me that it had more complex interface and harder access to the raw bitmap, so I decided to use VFW and not delve too much. Maybe I was wrong here again, I had to spend some more time on DirectShow.
(*Regarding huge code bases with too many fragmented modules - respectively I don't like Boost with its 999999 tiny little files, most of which not used.)
So I started with simple Win32, I developed also simple wrapper classes for some controls. I didn't care that it didn't look "beautiful" or "modern", the buttons look-and-feel was not important.
One reasonable design choice was to develop the GUI in C# with an interface to the core processing through pipes, sockets or memory mapping (file-mapping in Windows), it's still an option. It would go with a "standardized" interface to the core editor so that it could be controlled from all kinds of external GUIs. I did something like that with my speech synthesizer "Toshko 2.070", but only for simple input, not full API to its internals.
Another possibility is Lua and/or automatic generation of the GUI from the specifications.
...
Historically, there were years with zero or a few lines of code added to the project and unfortunately the editor's GUI is still underdeveloped and ugly for casual users which prevents it to be released for external usage out of my "in-house" needs.
It's pretty fast for some tasks, though. For example, Twenkid FX loads the long version of the "Star Symphony", including alternative disabled video segments and overlays, 200 full HD video files in total, for the first time in a fresh session in about6-7 seconds1.5-2 seconds from a laptop's mechanical HDD and external HDD. Maybe that's the total seek-time for so many files.
If the project is then closed and re-opened again, it loads and is ready in just 2 seconds.
(It seems that test run was with a highly loaded RAM and page-file slowing it down).
The GUI has to be improved, though, and possibly rewritten in a multiplatform way to escape that Windows dependency. I've been thinking about that from time to time, but it requires enough of focus to start.
Perhaps it would be based on FFmpeg, OpenCV and OpenGL, maybe using multiple programming languages (Python and C++, maybe others) with a custom GUI written on top of OpenGL and OpenCV or some light GUI or gaming library, unless I changed my mind and continued with Windows and a DirectX11-12
Also it's supposed to start utilizing some form of AI already, of course. Finally...
...
Custom VFX system and effects for the movie:
* Python,OpenCV with Python, Numpy; a little C++ and OpenCV in C++ for some retouch work of already rendered video segments during the final stage of the editing.
I started with Python because I had a prototype for simple reviewing and cutting, besides my main GUI NLE editor. Of course I was assuming that it would be easier to experiment with OpenCV, even though I knew it'd be slower, and initially I didn't know how far I'd go with the visual effects.
I could use C++ without a big hurdle, since I had experience and experiments with OpenCV C++ as well such as applying computer vision processing over pictures and frames of videos, traversing pixels and changing them during playback etc. The heavier editing system Twenkid FX C++/Direct3D9 was also an option, especially as HLSL shader, the only simple way to add new effects. However it needs a general and sophisticated plug-in subsystem, which is still lacking.
So I took the Python road this time.
It got too slow for some operations, then some tricks with Numpy fancy indexing sped it up, for one of the early effects: 60 times, from about 30 seconds per frame to about 0.5 seconds per frame. However it still remained slow for complex effects, sometimes taking several seconds per frame.
Ironically, the slow speed sometimes was "right", allowing real-time adjustments during rendering, virtual camera operating for Pan & Scan sequences etc., without slowing down the playback or stepping manually frame-by-frame.
Of course, I had better worked with C++ and GLSL or/and HLSL shaders from the start.
CogAlg Prize
Nevertheless that performance-wise wrong design decision and involvement with Python and Numpy directed me to check the CogAlg* project, then eventually to contribute to the debugging of the stuck frame_dblobs function and to win a prize.
Python is a bad choice for a "non-neuromorphic deep learning" for computer vision at the low level of the system, which is expected to require a zillion of operations before starting to produce meaningful output, though. Besides CogAlg's code is getting progressively unreadable.
This is another story, though.
* B.K. is the creator of "Cognitive Algorithm" project, but I recalled that I first called it with that shorthand "CogAlg" in an e-mail a few years ago, and he adopted it.
First impressions on Hinton et al. "Capsules"/CapsNet update to the convolutional NN/CNN that got popular recently with their latest paper on Dynamic routing.
1. Hinton approves Boris Kazachenko's old claim and criticism to ANN in his Cognitive Algorithm (CogAlg) writings that the coordinates of the input should be preserved and that this is one of the CNN/ANN design faults.
2. The "Dynamic routing" sounds to me as their way to generate "new syntax" in CogAlg terms, as different ways for evaluation of the input. Boris disagreed though, he corrected that it maps to his "skipping" (of levels).
3. The intended focus on particular smaller-region features per "capsule"/"group of neurons" ~ (mini-)columns reminds me of Numenta/Jeff Hawkins' approach, i.e.: a) cortical algorithm - a structure of functional modules, not just "neurons" b) higher modularity
All of the above seems as steps ahead to finer granularity of the patterns that the systems would model.
4. Besides, if I understand correctly, Hinton agrees with my claim/observation in early 2012 that vision/(object recognition) is ultimately 3D-reconstruction* and comparing normalized 3D-models of various level of detail - "inverse graphics".
My view* is that "understanding" is the ability of the system to re-render what it sees with adjusted or with changed parameters, which, in their terms seems to map to keeping the "equivariance" (or "match" in CogAlg terms), or as I see it: to simulate/traverse the pattern in the space of its possible states.
That’s according to:“Does the brain do Inverse graphics”, published in Youtube on 25.08.2015, a record from a lecture in a “Graduate summer school”, Toronto, 12.7.2012” from: https://www.youtube.com/watch?v=TFIMqt0yT2I
* To me it's supposed to be obvious, I think it's obvious to cognitive psychologists (Hinton mentions the mental rotation tests), to artists, to researchers, to ones who study human vision and optical illusions.
However it wasn't obvious for example in the AGI community below and if one is doing messy ANN where there's no reconstruction, but "weights", "convolutions". All were talking about "invariance".
** Boris' comment on capsules in his site:
"Actually, recently introduced “capsules” also output multivariate vectors, similar to my patterns. But their core input is a probability estimate from unrelated method: CNN, while all variables in my patterns are derived by incrementally complex comparison. In a truly general method, the same principles must apply on all stages of processing. And additional variables in capsules are only positional, while my patterns also add differences between input variables. That can’t be done in capsules because differences are not computed by CNN.
...
Archive from the AGI List from the year 2012
At that time the "invariance" was a buzz-word in the AGI email list. See more below in the digest I've prepared from 4 threads from that era back in 2012. I've not visited that place since a long time, the emails should be there if it's still active.
1. Generalization – Food and Buildings, 1/2012
2. General Algorithms or General Programs, 4/2012
3. Generalization - Chairs and Stools , 10/2012
4. Caricatures, 5/2012
The 4-th email from the "General algorithms..." thread:
Todor Arnaudov Fri, Apr 27, 2012 at 1:12 AM
To: agi@listbox.com
I don't know if anyone on this discussion realized, that "Invariance" in vision is actually just a
- 3D-reconstruction of the scene, including light source and the objects
- Also colours/shades and the textures (local/smaller higher resolution models) are available (for discrimination based on this, may be quicker/needed for objects which are otherwise geometrically matched)
[+ 16-7-2013 - conceptual “scene analysis”, “object recognition” involves some relatively arbitrary, or just flexible, selection criteria for the level of generalization for the usage of words to name the “items” in the scene. To Do: devise experiments with ambiguous objects/scenes, sequences. … see “top-down”, … emails 9, 14, 15]
If the normalized 3D-models (preferably to absolute dimensions), lights and recovered original textures/color (taking into account light and reflexion) are available, everything can be compared perfectly and doesn't require anything special, and no "probabilities" or something. The textures and light most of the time don't even alter the essential information - the 3D-geometric structure.
"2D" is just a crippled 3D
"Invariants" in human fully functional vision are just those 3D-models (or their components, "voxels:) built in a normalized space,the easiest approach for quick comparison is voxels, it might be something mixed with triangles, of course textures and colours also participate.
Every 3D-model has a normalized position per its basis, and also some characteristic division of major planes and position between the major planes, and there are "intuitive" ways to set the basis --> gravity/the ground plane foundations, which is generalized to "bottom", i.e.:
-- The "bottom" of an object, which faces the ground, is the part of the image of the object which projects on the "bottom" of the scanlines of the retina, because that's inferred for the first objects, which always have stable touch with the "ground".
When generalizing or specializing, the resolution of the 3D-models to be compared is changed (see the thread where I gave example of how the concept of a "building" is produced), at particular stage every two 3D-models match, eventually converge to a cube, or a plane.
IMO in fact brain is not very good in further mental rotation of those models, yeah we know those IQ tests, but humans do it very slowly and the tests consist of very few crossing planes, because it gets too complex.
Con: "How can you say that it's "just" 3D-reconstruction? That's so compex!"
- Well, one may think so only if she was not familiar with the triangulation (photogrammetry dates back to 19-th century) and/or the spectacular work of Mark Polleyfeys.
"How do you recognize that this is your chair, if it's upside down and you haven't seen it before"
Like the mistakes about generalization - a "chair" is a generalized concept, it's not a pixel-by-pixel image, rough 3D-models are compared for finding a match. And matching is a biggest number of high degree of match of the size relations of the boxes, planes color (after light correction) + texture, to the match to those of the chair from the previous day, than to those boxes, planes etc. of "chairs" found elsewhere, and of any other "objects".
A "chair" [a stool] generally is just:
-- A plane which is perpendicular to the "ground" direction vector, which is a vector which is parallel to "gravity" - that is the vector where objects go when let without a support;
-"support" is a vector consisting of "solid" connection (of forces, impacts) to the "ground" which when existing prevents objects from getting closer to the "ground" (falling);
- the "ground" is a plane where objects stop their motion (changes of coordinates between subsequent samples) if left without support or impacting by other moving "things", etc.
Most chairs can be reduced to a few solids and still be recognizable.
I've discussed about the must-be simplicity of 3D-scene-and-light reconstruction, back in the 1/1/2012 article about Optical Illusions, and later on the AGI-List*, and I see it very soon coming en mass.
"TILT" by Microsoft is yet another demonstration of this coming... :X
The RGB-D sensors (such as Kinect and ASUS Xtion) that are getting cheap and popular are also a jump to that direction, they make 3D reconstruction trivial.
In the last two months I've been also directing my mind into visual-vision-images R&D for pracical developments in computer vision, graphics (my Twenkid FX studio video editing/visual effects and general movie production) and on the 3D-texture-light-reconstruction, but I've been slipping and spreading into many other more general branches, for example mathematics, physics, philosophy and even some... music.
I hope I won't be too late, but the true beginning of the implementation of my first SIGI-AGI-prototype is also approaching, I'll show it when it's ready to do impressive job. :P
Some of the system's purposes will be immediately practical and aiming to be useful as a product.
It might be the smart mind behind an old project of mine, I planned to implement a little bit of it back in 2008 as a Master's thesis, but I was too busy and eventually ended up doing something else, i.e. described my 2004's Text-to-speech synthesizer "Glas" and proposed many improvements and an entirely new architecture. That's another project that I wish to improve and make that new architecture. For example that old version is still in use for reading out loud materials for me.
I've called that old advanced project "Research Accelerator" in some old posts, because it's supposed to help general research (all domains), it's also an "Intelligent Operating System".**
A few somewhat related systems/approaches/directions (quite not fully similar, though):
The research field of "Intelligent Environments" is also related to what my AGI prototype is supposed to deal with, e.g. activity recognition.
See for example: Fine-Grained Kitchen Activity Recognition using RGB-D
* I've talked about some obvious 3D-reconstruction clues even in the 2004's "Universe and Mind 4", and everybody knows them, but they are not consciously accessible to most people.
** Windows 7 has a function for smart prefetch of applications expected to be started by the user soon - "superfetch". My intelligent system is supposed to do, too, but in more general "superfetching sense". In general, that direction of predictive doing of things is obvious and everywhere, prediction is in the core of my theory of intelligence as well and prediction is in the essence of computing in general.
*** AGIRI AGI List
I've spent more efforts that I should in explanations of important concepts on the AGIRI AGI List from time to time, but I feel sorry afterward... :D The extent to which I do is a waste of time, especially since the ones who seem not to get it just doesn't get it, no matter how precise the explanation is. However I believe a lot of the materials are pretty detailed and useful if you're interested in AGI and want to learn.
They deserve a "digest", something I've promised long ago - well, I'll do when I can, some day. If it doesn't happen soon, it may come in the next 5-th part of my old series of big works.
While browsing some SIMD SSE optimized image processing code and digging from some time, I reached to a few excellent free resources for boosting your software, I recommend those:
Препоръчвам следните отлични матерали за оптимизация на код, многонишково/паралелно/матрично програмиране с CUDA, OpenCL, OpenMP, SIMD със SSE, суперкомпютри, оптимизация на код на С++ за различни процесорни архитектури.
I recommend checking those out, not bad quick intro to Computer Vision and links to leading research and technology, also some directions about sensory data procesing, Machine Learning and Self-Driving Cars (referred as Robotics there). NLP part is decent intro either, but for more technical intro, see another course, cited in my recent post.
Sebastian Thrun is very funny, he was the leader of Stanford self-driving car team from the DARPA Grand Challenge and DARPA Urban Challenge, then the self-driving car at Google, which is pretty impressive.
Thrun
has started other courses on a dedicated site, away from Stanford: http://www.udacity.com/, starting with Search Engines and Self-Driving cars.
- Building a Search Engine
Learn key concepts in computer science and build a search engine like google!
- Programming A Robotic Car
I am supposed to turn back to Computer Graphics, Image Processing and Computer Vision, because I need to add some visual magic to the epic movie "The Gift", which is in production now. Not that I will not enjoy going back to Image processing. :) I've just been too busy with other stuff recently.
Below, a short sequence, composed by "Magician" - an experiment I did in 2005. No blue/green screen, so composition is perfect without artifacts and with smooth noise. Експеримент от 2005-та. Не е използван зелен екран, а съчетаване на успоредни кадри.
What about the movie?
"The Gift" is supposed to be a stunning film. A film like no other. A short film, but not too short - maybe 15+ minutes, maybe 20 minutes. Or maybe more. Yet unknown. But long enough to tell an epic story. Stay tuned if you want to know more.
...
Вращам се към компютърната графика, обработката на изображения и компютърното зрение... :) Искам да придам малко вълшебство в един от филмите по които работя - "Подарък".
