GPT2-Medium Training from Scratch on Google Colab for Any Language - Tips & Tricks by Twenkid

My tips for training on Colab on Tesla T4 16GB, based on the obstacles that I had to overcome. I trained on a custom-built Bulgarian dataset. https://youtu.be/F-Xt-cK4L-g The code is based on: Arshabhi Kayal's tutorial: https://towardsdatascience.com/train-gpt-2-in-your-own-language-fc6ad4d60171 However his example was for local training on a modest Nvidia Geforce RTX 2060 (6 GB) for GPT2-Small (3 times smaller) and a fixed dataset. The code in the experiments in this video was extended and debugged for application in the Colaboratory which has its hurdles and for gradual extension of the dataset after each training epoch, without retraining the tokenizer (see Dataset Skip None in the video). ...

Some impotant points and discoveries: * Google Colab hurdles (the dataset should be sampled in parts, can't run too long epochs at once)

* The inputs/labels output of tokenization after changing the dataset should be filtered (Dataset Skip None)

* Etc.

...

Съвети за машинно обучение на GPT2-Medium модел на български или друг език от нулата през Google Colaboratory, от Тодор Арнаудов - Тош/Twenkid. Следва продължение. Errata: ~ 2:05 Tesla K80, not P100.

#gpt2 #transformers #tutorial

Read More

Смарти - Най-умният Английско-Български Речник: Безплатен и Мощен | Smarty - the most powerful comprehension assistant

Интересно и подробно видео представяне наа интелитентния речник, помощник в разбирането "Смарти", който беше най-мощното подобно приложение в света, когато беше създадено, въобще в категорията умни речници/езикови помощници - и все още ще да е един от най-мощните, макар че имаше огромно поле за развитие. За двойката английско-български съм срещал само един съвременен само с частично сравнима интерактивност и само в една от всички категории на "Смарти", защото работи само по с по една дума, няма изрази и пр. Другите онлайн речници са от най-простия вид като от преди 20 години.

Гледайте видеото:  https://youtu.be/QPWfpYwT_Ic

Виж също за изтегляне и др.: https://artificial-mind.blogspot.com/2020/06/smarty-comprehension-assistant.html

Най-мощният интелигентен английско-български речник "Smarty/„Смарти“, на който бях архитект и разработчик в изследователската група по компютърна лингвистика в Уулвърхамптън, Англия с научен ръководител проф. Руслан Митков. Речникът е безплатен и може да го използвате на всяко Windows PC, дори да е само с 256 МБ RAM и .NET 2.0 (C#) Какво е умен речник или помощник в разбирането? Това е приложение, което не извършва пълнотекстов превод, така че не може да допусне и фрапиращи грешки, и в същото време е много по-мощен и "умен" от обикновен електронен речник, "разбира" езика и подпомага и ускорява превода и изучаването на чужди езици, чрез по-сложна автоматична обработка на текста и чрез по-взаимодействащ и богат потребителски интерфейс. Разпознаване на изрази, на части на речта (съществителни, прилагателни, глаголи, наречия), търсене по окончания на думи, онтологията/синонимен речник - Wordnet и Balkanet и др. Обработка на естествен език, езикови бази данни, NLP, Natural Language Processing, Computational Linguistics. Wolverhampton Research Institute in Information and Language Processing. RIILP ... Разказва също за Xerox Locolex comprehension assistant, SA Dictionary, Езикотворец - речник на юнашкото наречие и сметачобългарския, ДЗБЕ. (...) Лематизация, нормализация, т.нар. Стеминг ((stemming) - намиране на корена на дума или основна форма – например при търсене или посочване на дума в мн.число или спрегнат глагол - програмата знае спрежения на неправилните глаголи, времена, образуване на мн.ч., части на речта и пр. Разпознаване на изрази чрез размито сравняване с шаблони и показване без да е нужно да ровите в статията. (...) Бързо разглеждане чрез посочване на дума от текста. Речник по окончания на думи/край на думи - търсене на рими, съчиняване на стихове. Виж също Reverso Context - много години/10-тина години след "Смарти". Българска и английска лексикография, езикознание. Дружество за защита на българския език - ДЗБЕ.

Read More

Who is the fittest AI researcher or developer in the world?* [FUN FACTS]

Watch the performance: https://youtu.be/e95ZdzVzSsc


The video is from late August 2021, these are an year old shoots for the project "Bulgarian Ninja", or a later name: "Balkan Ninja" -  a project for a fighting video game (in Bulgarian) ~ 10.2020.

https://github.com/Twenkid/Plovdiv-Kombat

 

Well: not the fittest developer in any domain, and not working out too hard either.

For the fittest developer and also man, see the Belarusian guy Max: Максим Трухоновец. He seems as the best/the beast & the G.O.A.T. Max is Guiness record holder in many street work-out disciplines.

Read More

Body Art & Muscle Art using 3D Depth Camera made of two Logitech C270 webcameras - Бодиарт с 3D-камера

https://youtu.be/pIGcdBaoZxU

Depth camera body art / muscle art - done with two Logitech C270 and opencv coding in python. If anyone cares, the code is in my github repository that is linked in the video description, in the recent commits. Note that the current version of the code is experimental and not polished. That output also could be interpolated etc. to fill-in black discontinuities etc., however as of "art" they serve as contours etc.

Author, coder and model: Todor.

To be continued...

...
"Бодиарт"/мускулно изкуство създадено чрез стерео камера (3D-камера, камера за дълбочина). Направих го с две камери Лоджитек C270 и програмиране на Питон с OpenCV.
Автор, програмист и модел: Тодор - Тош.

Следва продължение...

Read More

Видео: Общ изкуствен интелект – Конференция по Самоусъвършенстващ се ИИ в Пловдив: Self-Improving General Intelligence SIGI 2012-1

https://youtu.be/9a0IrhsLVgY

Видео разказ за първата по рода си в България конференция по Общ ИИ в Пловдив през 2012 г. и информация за развитието на някои от участниците в нея.

Имах намерение и желание да бъде редовна, поне по два пъти в годината, вероятно частично онлайн, затова и името беше "2012-1", но тогава не успя да се продължи.

Сега може да има нов опит. Очаквайте продължение: следете и ФБ групата по Универсален изкуствен разум.

 

 

 

Read More

Google announce they are working on an AGI algorithm called Pathways

Scarce info on reddit. Thanks to E. for the link.

Posted byu/QuantumThinkology - 23 hours ago
Google is developing a nimble, multi-purpose AI that can perform millions of tasks. Called Pathways, Google’s solution seeks to centralize disparate AI into one powerful, all-knowing algorithm, Jeff Dean, Google’s AI chief revealed

https://qz.com/2042493/pathways-google-is-developing-a-superintelligent-multipurpose-ai/?utm_source=reddit.com

Read More

ETH AI Center - Todor's 2003 interdisciplinary AGI research strategy materialised in Switzerland's Technological Institute

Compare with the strategical suggestions in this 18+ years old essay: https://translate.google.com/translate?sl=auto&tl=en&u=https://artificial-mind.blogspot.com/2020/07/interdisciplinary-research-institute.html

In the 10.2020 message they mention 29 professors, as of 7.2021: 93.
https://ai.ethz.ch/about-us/faculty.html

20.10.2020 

https://ethz.ch/en/news-and-events/eth-news/news/2020/10/pr-new-centre-for-ai-research.html

Press release ETH Zurich is opening a new research centre for artificial intelligence.

A core team of initially 29 professorships, a new executive director and a Fellowship programme aim to promote interdisciplinary research into this key technology. (...)

Core team with around 29 professorships The new research centre is made up of core members, associate members and AI fellows. The core members already comprise 29 professors from seven departments who specialise in fundamental AI topics such as machine learning, perception and natural language understanding. These members will also supervise the new fellows of the ETH AI Center, talented young AI scientists who have been recruited from around the world and awarded a scholarship. “This will enable us to train experts who will be able to implement interdisciplinary AI projects,” says Günther. The ETH AI Center will also maintain contact with associate members from all ETH departments, other institutions and the private sector in order to implement joint research projects. (...)
"

M.V's. post on linkedin (bold: mine):

"...The density of AI activities here cannot be found anywhere else in the world..." -- Dr. Alex Ilic Executive Director, ETH AI Center

Sounds about right :)

Alex Ilic doing amazing job in creating the ETH AI center, a world leading science and research inter-disciplinary think-tank in the general AI space, with nearly 100 world-class professors at ETH from a wide range of disciplines, including statistics, mathematics, computer science, physics and many more. Exciting times!

Comment of mine from a few days ago:

Finally... Well, Stanford and MIT embarked on something like that in 2018, but, in fact, such an interdisciplinary institute, even "more interdisciplinary" in its philosophy, was proposed *18 years ago* by me in Plovdiv, Bulgaria, where also the world's first interdisciplinary course in AGI was offered in 2010 and 2011 at Plovdiv University by the same author. Note that the MIT's course was from 2018 and was just a compilation of lectures by various guests, without a coherent structure, unlike the Bulgarian one. By the way, the 2003 proposal explicitly mentioned the term "Strategy" for AGI (Thinking Machine), as well as code synthesis (automatic programming) as an important milestone. The work was an essay for an investments proposals competition, called: "How would I invest one million Euro with the greatest benefit for the development of the country?": https://artificial-mind.blogspot.com/2020/07/interdisciplinary-research-institute.html

An archived version of the original: https://www.oocities.org/todprog/ese/proekt.htm

 
More info: in the book "History, principles and pioneers of the Artificial General Intelligence, or How an 18-year old researcher from Plovdiv was 15 years ahead from MIT and Stanford in AGI".

Kind Regards

As you know I've been preparing, mostly in mid-2020, a big book (initially in Bulgarian) which serves as yet another invitation for partners, investors and sponsors for the creation of that projected institute-research company. The title is being evolving, the current one changed to just mentioning "by the author of the world's first interdisciplinary course in AGI" or in occasions self-promotingly "by the prodigy author of ...".

However I am still unsure how exactly should I present it; probably starting with podcasts etc. Selling it commercially in its entirety seems unlikely and it may need heavy editing and trimming, also splitting it in many volumes. It is about 1030 A4 pages at the moment, in one file... :))

Read More

Общ изкуствен интелект и Трансхуманизъм: Какво е Универсален изкуствен разум - AGI?

Гледайте нов запис (с допълнителни бележки) на една от уводните лекции от първия в света интердисциплинарен курс по Artificial General Intelligence - AGI.

 

 

https://youtu.be/r582MVKj81A

Слайдове др.:

http://research.twenkid.com
http://artificial-mind.blogspot.com 

Следват продължения! 

Съдържание: 0:00 Въведение
0:30 Защо възникват новите термини: AGI, Универсален изкуствен разум
1:13 За обзора на тесния ИИ - препратка към друго предаване
1:40 Първенството на курса в света: 8 години преди MIT
2:42 Първата фейсбук група по AGI е българска: 5 години преди другите
3:16 Какво е Универсален изкуствен разум? (УИР)
3:28 За ИИ из интервюто за сп. "Обекти" бр. 5, 11/2009
4:20 Накратко за УИР - слайд 3
4:52 Как ще изглежда мислещата машина? - из интервюто за "Обекти"
5:39 Някои термини, накратко за УИР - Слайд №3
6:11 Машината на Гьодел на Юрген Шмидхубер
6:21 Личности в общия ИИ
6:21 Какви опити се правят за създаване на УИР? - из интервюто
7:33 Разграничения между общия и слабия/тесния ИИ и недостатъци на втория - Слайд №4
9:25 Какво все още пречи да се създаде мислеща машина? - из интервюто
10:24 Технологична сингулярност, Кърцуейл, миниатюризация
10:42 Singularity Institute - из интервюто
10:50 Технологична сингулярност - закон на Мур - Слайд 7
11:16 Прогнози за планирания технологичен размера на транзисторите
11:34 Какво реално се случи в полупроводниковата индустрия
11:41 Краят на миниатюризацията и какво следва?
12:05 Квантови компютри
12:13 Нанотехнологии и молекулярни технологии
12:17 Киборзи
12:22 Ред е на мислещите машини - надтелесност и многото възможни форми на машинния ум
12:54 Мислещите машини са...
13:23 Разумът е предсказател на бъдещите възприятия въз основа на миналия опит - из интервюто
14:04 А ако машините решат да ни унищоат като въстанат срещу нас? Истинското послание на филма "Терминатор 2" - кои всъщност са "лошите"?
15:20 Етика за роботи - трите закони на роботиката на Азимов
15:45 Приятелски настроени машини - Friendly AI
16:28 Singularity institute - институт по сингулярността ("сингуларността") на Рей Кързуейл
16:59 Какво е "душа"? - препратки към литература
17:49 Какво е съзнание?
19:42 Трансхуманизъм или H+ - Transhumanism; свръхчовечност, надтелесност, космизъм
20:25 Прехвърляне на съзнанието и възможно ли е? - Mind Uploading
21:13 Парадокси на човешкия мозък: универсален или специализиран? Изчислителна мощ.
22:25 Хибриди: универсален разум + универсален компютър
23:12 Brain-Computer Interface BCI - пряка връзка мозък-машина
23:20 Буквална симулация на мозъка: Almaden Institute на IBM и Blue Brain
23:49 Заключителни думи и покана за следващите лекции и предавания
#изкуственинтелект #agi #artificialgeneralintelligence #ИИ #УИР #футурология #трансхуманизъм #космизъм #сингуларност #сингулярност #h+

Read More

Intel is approaching AGI - Cognitive AI

A recent talk by Intel is revealing that they have a clue about the direction:

SL 5 Gadi Singer - Cognitive AI - Architecting the Next Generation of Machine Intelligence

Some commented that this is a new hype term, however in mid-2000s there was "Cognitive Computing" (it is used here in the blog, too). Hawkins/Numenta were part of that branch, so it is not so new, related to Cognitive Architectures etc.

Read More

ASTRID - Cognitive Architecture by Mind Construct based on bootstrappng Common Sense knowledge and knowledge graphs

ASTRID, or "Analysis of Systemic Tagging Results in Intelligent Dynamics" is a Cognitive Architecture project of the dutch AGI company "Mind Construct". They express bold statements about their prototype.  Learn more:  https://mindconstruct.com/website/astrid
and the paper: https://mindconstruct.com/files/selflearningsymbolicaicritcalappraisal.pdf

I agree with the claim ot the ASTRID authors that "symbolic" systems can learn. The problem of the systems which don't is that their "symbols" are too narrow and lack the appropriate "connectedness"/scanning/exploring perspective, generality, potential to map to the others, in short: too "flat" and simple.

P.S. However see the article: Neural Networks are Also Symbolic - Conceptual Confusions in ANN-Symbolic Terminology: https://artificial-mind.blogspot.com/2019/04/neural-networks-are-also-symbolic.html

I don't like the misnomer "symbolic". More appropriate terms for "symbolic" AI are logic-based, logical, conceptual (opposed to [purely] "numerical"); discrete? (or more discrete and more precisely addressable than the "subsymbolic"), "structured", "knowledge-based", "taxonomy-ontology-...", more explicit etc. (many other possible attributes).
 
On the other hand some methods which are called "subsymbolic" are more "numerical", but they eventually or at certain levels also become "symbolic" or logical and usually are mapped to some "symbolic" categorization (last layer). 

Both do and should join and the "logical" and "discrete" at the lowest level and in its representation is also "numerical" and "analog" to some extent. It is some digital operation in a computer, when operating on visual, auditory etc. raw sensory data it is numeric etc.

Read More

On Free Will as an Ill-Posed Problem | Improperly-posed problem

Comment of mine on a post in Real AGI group.
referring to an article "The clockwork universe: is free will an illusion?"

Todor: Whether or not somebody or something has a "free will" depends on how exactly "free will" is defined, both "free" and "will", also "you". I think all that discussion and the "catastrophic" consequences are largely anglosaxon-centered views or ones belonging to "control freaks" and sound like sophisms. Of course one can never be of full control of "his" choices, you are formed by an endless amount of "external" forces, there are large portions of life where "you" is unconscious, the processes of your orgranism in complete detail depend on everything, it's of your "full" control only if you are God. It's hard to define what "you" is, where exactly the boundary lays, and obviously what "you" can realize and enumerate or deliberately control is almost nothing of the bitrate that describes your body and all processes in maximum resolution. The intentionally muscle-controllable trajectories are of mere bits, while a zillion bits describe just one CELL of the body in any encoding. The body is also a tiny bit of the whole Universe, where a principle of superposition is in effect etc., everything is computed by interacting with everything else.

IMO that is not supposed to cause existential catastrophe unless one is prone for that due to "control-freak-ism" or something - nothing follows from the lack of the "complete control", it's not the end of the world, unless one believes he was god and now he finds that he wasn't.

"They argue that our choices are determined by forces beyond our ultimate control – perhaps even predetermined all the way back to the big bang – and that therefore nobody is ever wholly responsible for their actions."

This is not a new argument.

Ultimate control - the mania of some cultures and tyrants.

However responsibility as localisation of causal forces, given a resolution and method of factorisation, is another question.

An answer from the poster of the link:

A.T.: I don't want to get into the discussion of free will. If you don't think humans have free will, then you won't care that robots with AGI almost certainly don't have free will. Humans may or may not have free will, but robots cannot, as we know the "clockwork" of their operation. Even using a rand() function that uses pseudo-random numbers won't change that fact, even if the seed is altered. That can always be determined so that the deterministic outcome is theoretically known. As I said, I thought it might be a post to pose a question that I have not seen posed this way before.Even using a rand() function that uses pseudo-random numbers won't change that fact, even if the seed is altered. (...)

...


(The following is posted only here)

Todor: You only believe that you know the "clockwork" of their operation (of a machine, AI etc.). In fact you may say the same for anything with a certain resolution. If randomness is the "free will" freature or component then electrons and all physical particles in the quantum model have "free will", which is fine, however therefore everything has that "free will" and defined like that this concept is meaningless, because it applies for everything and clarifies nothing.

The "theoretically known" part is true for everything as well: if you were God, if you could read the content of everything fast enough without changing it etc., then you would know the "clockwork" for everything. "In theory" you could be, and as of computers and robots: they are part of the whole Universe and in interaction with them, if they interact and their behavior, operation, knowledge etc. are impacted by entities or parts with "free will" within Universe, then they also would have that property and their actual "body" extends to everything that impacts them.

Therefore one must first define exactly what "free will" is and what it is not. Whether or not anything has or doesn't have anything depends on the exact definition. Also humans or whatever can have "free will" even if it's considered "deterministic" or predictible, will and free will as I see it is not about being not deterministic, "free" is not about being random (except in these confused believes).

For example see the definition of Hegel, Engels/Marx, thus Dialectical Materialists: they are deterministic, their definition of free will is to act in accordance with the necessity, i.e. to understand what to do, to be conscious of the possibilities and the desired outcome and "the best" way to achieve it etc. and a lack of free will is if the agent "doesn't understand" (but yet that must be precisely defined, otherwise it's generalities and sophisms), thus if your choice is random and you can't explain it you are also not free, but dependent on the "will" of the randomness or "the Fortune" (instead of "your own" also).

Having or not having anything, doesn't imply anything on its own and has no intrinsic ethical consequences by itself; the ethical consequences are of political and ideological origin. "Lack of God" doesn't mean that "everything is permitted" (Dostoyevski sophism), neither if you consider that an ant or a PC or a tree "does not have free will", that consideration on its own does not impliy (or it does not follow from it) that you have or don't have to do anything with it.

Similarly the fact that other humans are supposed to have "a soul" or "free will" and if the agent "believes that" couldn't stop a murderer, a psychopath or a criminal, a warrior/general or a plain soldier, or any "evil one" etc. Respectively, if you like/love animals or even "inanimate objects" - plants; weapons, cars, computers, toys, books, precious memories - you may handle them with care and love, because that's what *you* feel, it's subjective.

The randomness (disconnected from everything, supposedly) for "free will" is actually dependent on the Universe as a whole which only "predicts" the exact values - so that "freedom" is most dependent (of the whole).

"Freedom" in general as some kind of "independence" (or dependence) is within the decided/given framework and resolution.

Read More