Monday, May 29, 2023
Wednesday, May 10, 2023
2023 Update: Training GPT2-MEDIUM etc. from scratch and unlimited-length generation with hidden direction of the prompt for any generative models
Training GPT2-MEDIUM from scratch on Colab. UPDATE: 6–5–2023: float32 fix (no mixed precision — error) and Chained-generation with unlimited length with directed prompt injection (see the other video from Todor’s youtube channel)
An update for my Youtube tutorial and notebook from 2021, after I discovered the code needed a little tweek to run again. I also added the code for the unlimited-length generation with hidden prompt injection.
The same idea could be applied for any generative model, if you have control of it and can inject prompt in the context when later to be hidden from the output.
GPT2, Collaboratory
#gpt2 #colab #python #promptengineering
Monday, April 17, 2023
The hardware and resources inequality in AI/AGI: an old story now rediscovered by the worried mainstream — a 2013 & 2009 articles vs 2023 paper
I start to publish in Medium as well - I had to to long ago, as it has a community and "social life", but: better later than never. I may republish some of the articles here there in order to hopefully extend the appropriate audience reach.
Editing The hardware and resources inequality in AI/AGI: an old story now rediscovered by the worried… – Medium
The hardware and resources inequality in AI/AGI: an old story now rediscovered by the worried mainstream — a 2013 & 2009 articles vs 2023 paper
“Montreal.AI: 23 ч. · Choose Your Weapon: Survival Strategies for Depressed AI Academics Julian Togelius, Georgios N. Yannakakis : https://arxiv.org/abs/2304.06035
#ArtificialIntelligence #DeepLearning #MachineLearning”
While it is true that even 8 years or 10 years ago even regular programmers could have the GPU power ( the well-paid and owning their time on the right target; but usually the ones who make money lack the vision and they buy GPUs/hardware for games, and the ones who had vision and intelligence had no money), the “inequality of opportunities” is of course not a new phenomenon, including in AI. I’ve written about it in 2013 and it was valid for the pioneer AGI researchers one of which was I, publshing substantial works since 2001, aged 17, and being author of the world’s first University courses in AGI in 2010, 2011 with theories and a course program that still stand and are only confirmed and elaborated by more and more researchers and publications. The inequality phenomenon was valid for the AGI researchers versus both the well-"fed" well-funded high-profile and famous academics who “rolled their eyes” when they heard about AGI (ask Hassabis, Legg; and Altman even about 2010-early 2010s in MIT, Altman refers to 2015 when they found OpenAI). It was vlaid versus any researchers from the Academia (with students working for them, “free” laboratories etc.), and of course: the industry.
https://artificial-mind.blogspot.com/2013/08/issues-on-agiri-agi-email-list-and-agi.html
A part of the conclusion of this work:
“… — WORKABLE THEORIES and IMPLEMENTATIONS
Some people try to work on workable theories and implementations, but this list is a home of the poorest and the most lonely ones in the AGI community, even though some of them were some of the pioneers of the new wave of that community, long before the “institutionalized” researchers took it as “prestigious”.
The list’s researchers poorness impedes their opportunities/motivation for concentrated work/producing academic-style materials — many believe the mainstream academic system (including many aspects of the peer-reviewed journals etc.) has intrinsic corruptions and have left it for “political” reasons.
Moreover, even if they do know how or have potential to develop working machines, this is a big effort that may take a lot of time before they could have a complete system — coded and running. If they haven’t produced visible results already, that doesn’t imply they wouldn’t do after years of collection of critical mass, as long as they could work.
Besides they are supposed to be 10, 100 or 1000 times more capable than the normally funded and organized ones from the academic/industrial competition. Current ones can’t afford visiting appropriate conferences or travel around research centers and are alienated.
They should have much broader knowledge and skills, acquire new knowledge and skills in a shorter time and work much faster, because:
— they can’t afford truly focussed work — too much other troubles, too much sub-problems they should solve alone, a lot of wasted time in attempts to find partners or develop some “booster-funding” technologies, plenty of frustration due to the isolation and helplessness against all the problems [including the dumb financial etc. ones] they have to solve [implement] alone (or give up)
— they do not have students, partners or “slaves” to give the dirty job to [or barely have, but it’s hard to motivate anyone without funding]
Overall, they should shoot 100 or 1000 targets with one bullet, or they “die out” [in the race]
Welcome to the list of the losers… :))
However some of these “losers”, due to the extreme requirements they face, may really be 50 or 100 times more productive or knowledgeable and non-conventional than the “ordinary” funded and supported competition, and may have guts and balls that the others lack.
Otherwise they should have given up, be part of the existing institutes — “institutionalized” — or from the “AI”. But they are not from those institutes, because when they proclaimed that “AI was wrong” they were outsiders already, heading towards new directions.
Furthermore, those brave ones are supposed to believe and find a way to make thinking machine possible on cheap, old and slow hardware, otherwise they should have another reason to give up to the supercomputer owners and the rich institutionalized researchers…”
The same about NLP:
“What’s wrong with NLP? Part 2”, 3/2009
https://artificial-mind.blogspot.com/2009/03/whats-wrong-with-natural-language.html
One other option for the academics, who are pretty wealthy but complain about that OpenAI, DeepMind etc. are wealthier:
Invent somethign that’s really innovative, different and more efficient. Everybody prefers to just spill in more hardware, make a little change and engrave her name for “new contributions” (what about the credit for the hardware designers and producers?), it was similar in 2000s with NLP: change one bit of some algorithm, produce an increase of 0.1% of some measure/bechnmark and there you are: “a new NLP model”, “moving the SOTA”. Why not building a new paradigm from the ground up. But yes, you can’t, because the dafault is that if you try, you won’t be accepted until you beat the competition and as explained above, in order to do and be accepted, you have to be 1000 times more efficient than them while working on your own with no resources. :)
Wednesday, April 5, 2023
Memory of the Visionary Research Directions from 2007's second blog post and a comment on the visual transformers and their representation
Looking back to the second post in this blog (after the first which was a placeholder)...
https://artificial-mind.blogspot.com/2007/11/research-directions.html
In // // Leave a Comment
Research Directions
Target research directions so far:Research Directions
- Artificial General Intelligence
- Artificial Mind
- Artificial Life
- Cognitive Computing
- Cognitive Science
- Computational Linguistics
- Data Mining
- Computer Vision
- Image Processing
- Sound Processing
Main direction:
Understanding the processes of learning, thinking, imagination, problem solving, decision making and development of evolving, thinking and creative machines.
Sub directions:
- Perceptions, mind states, thoughts, memories, imagination, desires, intentions etc. representation, simulation and generation.
- Natural language understanding.
- Natural language generation.
- World-knowledge representation, world-physics and human behaviour simulation for NLU, NLG and for perceptions, thoughts etc. simulation.
- Machine imagination and creative machines. Creative writing by machines. Dreaming machines.
- Machine learning, based on world-knowledge representations and simulations evolved from the input.
- Building world-knowledge and language competences by semi-supervised machine learning, using the web as world-knowledge feeder and language teacher.
- Differential intelligence researches.
- Didactics methods for measuring general intelligence of machines.
- First lanuage acquisition by humans. Modeling language skills development. [lanuage = language]
- First language acquisition by machines, which learn their knowledge, "corpora" and grammars like children do - by reading, analyzing and building new knowledge step by step with optional support of supervizing knowledge, given by human "teachers" or taken by the machine from ordinary textbooks and interaction with people on the Internet.
- Conversation agents. "Chat bots", "Virtual bloggers" and "Virtual forumers" which do NLU, "imagine" what the conversation is about, have intentions and express thoughts about the topics, aiming to keep real conversation.
- Intelligent Desktop and Network Search Engines, Intelligent Personal Organizers, Document and Notes Classifiers and Virtual Assistants.
Sound Processing:
- Speech Modeling
- Speech Synthesis
- Synthesis of Singing
- Speech Mimicry (extracting voice features from an input speech, then application in speech model and synthesis of speech with the same voice as the voice of the example).
- Advanced preprocessed image formats, assisting computer vision.
- Memory and heuristics based generation of photo realistic images, without complete 3D-modeling and rendering.
- Memory and heuristics based generation of 3D-models from single or multiple images.
- Computer Vision - Image/object recognition, categorization, generation, combination. Bots and robots, moving in virtual 3D worlds, a real world or in hybrid 2D-3D world simulations like in Quest games, which percept the world by vision systems.
...
Regarding the Image processing, lately I've been playing with Bing image creator, DALL-E. I'll show pictures from my plays with it later, a comment of mine in an AGI chat two days ago:
Todor: (...) Also, "meaningfully selective" is questionable, in some POV transformers are amazingly meaningfully selective, much better than humans in text-to-image, or "concept to image".
The generative models are better than humans in analysis and synthesis, especially with images, human synthesis capabilities with images for most humans are almost lacking at all, while DALL-E and MidJourney produce amazing and aesthetically pleasing photorealistic images which apparently are rendered by a process which is isomorphic to a classic rendering system that has an implicit 3D models, with a designer who places them in reasonable composition, with proper materials, lights and ray tracing or global illumination. Most humans struggle to draw even stick figures or in handwriting, how good and robust are their features, they are incapable to reconstruct the output. Average humans are capable only in superficial recognition and in evaluation of photorealism, if they have whole images in front of their eyes for inspection, and also the artistically gifted and trained could recreate some of the general principles of lighting, but painstakingly slowly or/and usually with references, photos etc., aan they would hardly struggle when light is interacting with refletive and transparent elements in the scene and if they lack referenes.
I.e. these transformers are far superior in that aspect of analysis and then synthesis of the "causes and effects" in the world of their inputs than even super talented humans, they have better articulated and mapped internal models of the visual physics of the light and the objects.
As of directly adjusting parameters of objects like texture-pixel-by-pixel or the tiniest 3D-model detail: humans also need explicit 3D-models and 3D-editors for that, such as Blender or 3D-Studio Max, which besides the structures of the brain have explicit meshes, triangles, materials etc. defined, and they adjust these details iteratively in a slow process. Given so general representations and directions, the generative models are excellent.
Also, I remember an old note of mine: there could be different approaches to AGI, but at their latest stages and levels they are supposed to get more and more isomorphic and to converge, because they are supposed to work with and represent similar cognitive structures, and they start with similar ones.
Monday, March 20, 2023
In
AGI,
УИР
//
//
Leave a Comment
ПРОРОЦИТЕ НА МИСЛЕЩИТЕ МАШИНИ - ИЗКУСТВЕН РАЗУМ И РАЗВИТИЕ НА ЧОВЕКА : ИСТОРИЯ ТЕОРИЯ И ПИОНЕРИ - The Prophecies of the Thinking Machines: AGI and Transhumanism: History, Theory and Pioneers
Заглавието и заглавната страница на мега-книгата на един от най-старите изследователски "институти" по Общ изкуствен интелект и Развитие на човека ("трансхуманизъм"), "библия" на пророците, създадена и редактирана от "най-младия пророк", се развива. Най-новата версия от днес и как може да помогнете на проектите на "Свещеният сметач":
Някои примерни начини в началния етап:
• Като станете съдружници.
• Като се включите пряко в изследванията и проучванията, разработката, експериментите с различни технологии и измислянето на нови; поддръжка на уеб сайт, споделяне на изчислително време от ваши машини – например за обучение на модели за машинно обучение, еволюционни и самоорганизиращи се алгоритми, други търсения и обработки, обхождане, извличане на информация, рисуване на графики и пр.; изпробване на програми и проекти на дружеството, експериментиране с нови технологии и пр.
• Като преки дарители и инвеститори с финансови средства – обаче все още не мога да съобщя подробности и условия, и към днешна дата (27.2.2023 г.) няма формално учредено юридическо лице и не се знае точно какво ще бъде. Затова на първо време финансовото дарение или инвестиция би могла да бъде лична и/или оформена като проект или работа, която да свърша за вас като възложители, или ако ме наемете като консултант и пр; би могла да бъде например конкретно разширение на някои от съществуващите и бъдещи разработки и проекти (някои са проекти за компютърни игри ) като определени функции, опити, насоки за работа, доклади и пр. и да бъде основа за бъдещо сътрудничество по други начини, а също така бихте могли като начало да помогнете чрез някой от другите варианти.
• Ако сте от университет или друг изследователски институт – с участие в съвместни семинари, работа по съвместни проекти, връзка с докторанти, студенти и учени; с условия за създаване на нов интердисциплинарен курс и/или учебна програма от много семестри или бакалавърска, магистърска, докторска по Общ изкуствен интелект
• Дарение или отдаване за ползване на изчислителна техника за по-ефективна работа: компютри, дискове, видеокарти, монитори, други компоненти и пр.
• Дарение на хостинг, реклама, облачни услуги.
• Предоставяне на стая, офис, помещения за ползване.
• Продаване на техника и др. на изгодна цена, с отстъпка и пр.
• Като ни наемете като консултанти по даден проект, за да се финансираме – стига да е подходящо и да сме съгласни, да не ни отнема цялата ни енергия – и като станете клиенти на комерсиалните проекти на дружеството или направите дарения в подкрепа на онези, които и сега се разпространяват безплатно и/или са с отворен код.
• Като разгласявате информация и положителното си отношения към начинанието и ни представите пред други хора, които биха били заинтересувани да участват или да помогнат: като споделяте клипове, публикации, впечатления.
• Като ни поканите на уеб семинари, беседи, подкастове и пр., за да говорим за дружеството и проектите.
• Като ни поканите за лектори и пр. или ни помогнете да организираме събития, от които да може да се получат и приходи, с които да се финансираме.
• С правна, счетоводна и друга помощ за формалното учредяване на дружеството или дружествата – ако онова за Развитие на човека се оформи като отделно, и/или ако има едновременно дръжества с нестопанска цел и фирма, които да си сътрудничат и пр.
• Като направите дарение лично на мен: за да подпомогнете изследванията ми, за тази книга или други мои произведения и дела; за пионерството ми в редица области, което се доказва в този труд; за развитието на човека; в подкрепа на многообразното ми и своеобразно творчество и самобитни работи във всевъзможни области: художествена литература и поезия, публицистика, езикознание, обществени и хуманитарни науки, философско-техническа литература, разработка на безплатен и свободен приложен софтуер, видео продукции, музика, „спорт“ и пр. и също творчеството и изследванията върху самия мен (бъдещи публикации): да бъдеш универсален човек. Или пък просто ей така.
• Като станете мой импресарио, агент, представител, „PR”, за да ми помогнете да комерсиализирам вече съществуващи и бъдещи творчески произведения и начинания, софтуерни продукти, както и способностите и личността и битието ми на универсален човек .
• Като ми помогнете с издаването, комерсиализирането, разпространението на мои произведения във всички видове жанрове и сфери, без да бъдете импресарио в пълния смисъл.
• Като помогнете в създаването и поддържането на кампания за финансиране: “crowd-funding” (от типа на Kickstarter и др.) или др.
• Като си купите тази книга и други, когато бъдат издадени комерсиално, или преди това – като направите дарения.
• С добри съвети, препоръки и идеи как да се финансираме по други начин; с положително отношение; със звездичка в Гитхъб за нашите проекти; с добра дума и морална подкрепа.
• По други начини.
Във всички случаи – вижте адреси за връзка по-долу.
Благодаря Ви!
Friday, March 10, 2023
In
Arnold,
Comedy,
Deep Learning,
Deepfake,
Fun,
Programming,
Video,
Видео,
Комедия,
програмиране
//
//
Leave a Comment
ARNOLDIFIER: 3X Higher Performance Version of DeepfaceLab 2 Deepfake Library by Twenkid
ARNOLDIFIER: 3X Higher Performance Version of DeepfaceLab 2 Deepfake Library, achieved by refactoring it to work with Grayscale images. It achieves similar quality with either 3X less VRAM, or 3 times faster, 3X more detail/higher resolution faces for the same size of the models in VRAM and disk. All high quality models in the video are trained on a Geforce 750 Ti 2 GB (a 2014 entry-level GPU) - Arnoldifier allows training of quality and complex datasets with 192x192 or even 256x256 faces. Make your amazing deepfakes with any low end GPU and imagine the quality even with 1060 6 GB ( performance of a 3080, if it had 18 GB RAM ), or turning a Geforce 1070 GPU 8 GB into a 3090-24-GB-equivalent monster. The only trade-off is the color, but there is a colorization prototype in Arnoldifier, see in the video, and a post-production colorization can be done with additional tools as well.
Watch the series "Arnold Schwarzenegger: The Governor of Bulgaria", produced with the library and find it on Github. https://github.com/Twenkid/DeepFaceLab-SAEHDBW
Subscribe, like, share, comment - if you want to see more videos about the library, tutorials, code review etc. Thanks and see you in the next video!
Wednesday, March 8, 2023
Tuesday, February 7, 2023
In
About Tosh,
Games,
Video Games,
За Тош,
Игри
//
//
Leave a Comment
Интервю с отбори "Неврони" и "Плевели" на Plovdiv Game Jam 2023 - настолни и видео игри - Neurons and Weeds
@ Пловдив Гейм Джам 2023 г. @PlovdivGameJam 2023
Следва продължение. Бордови игри, настолни игри, стратегически, логически, пъзели. Бъдеща релано временна стратегия за компютър, симулация. https://github.com/Twenkid/PlovdivGameJam2023-Neurons
https://youtu.be/jlJMeo3d8uE
Sunday, January 29, 2023
In
Natural Language Processing,
NLP,
Programming,
Tutorial,
Video,
Videocast
//
//
Leave a Comment
GPT2 Unlimited-Length Generation with Hidden Prompt Injections - Code Review
* A Longer Title: Unlimited-Length Imagination Directed GPT2 Chained Generation by Overlapping Prompts-Injection and removing the injected beginning of the following generated sequence
Tuesday, January 3, 2023
In
About Tosh,
Fun,
Sport,
Video,
Видео,
За Тош,
Спорт
//
//
Leave a Comment
Be Shredded All-Year-Long! Bulk & Cut is Bad for Your Health! What is my Latest Superfood? - Twenkid's Natural "Bodybuilder"-Self in a Funny Video - Културисткото "Аз" на Универсалния човек в: "Бъди Нацепен Целогодишно!"
Съвети на целогодишно нацепеният универсален човек Тош. Каква супер храна си хапва в последно време, за да бъде толкова изчистен? Полезно ли е да се прави Bulk & Cat/Качване и изчистване?
Абонирайте се, харесвайте, коментирайте, ако искате да чуете още съвети и опит за здравословно хранене, живот и вечна младост от всестранната личност, "дете чудо" и баща на българските Общ изкуствен интелект и Развитие на човека (космизъм, трансхуманизъм), създател на Дужеството за защита на българския език и др. Проектът за универсална търсеща и пораждаща (мислеща) машина: https://github.com/Twenkid/Bulgarian-Internet-Archive-And-Search-Engine Свещеният сметач: http://eim.twenkid.com Книгата "Изкуствен разум и развитие на човека: история, теория и пионери" - покана за съдружници, изследователи, дарители, инвеститори, последователи за развитие на всестранното дружество за Общ изкуствен интелект "Свещеният сметач": https://github.com/Twenkid/izkustven-razum-i-razvitie-na-choveka-kniga Потърсете клиповете от плейлиста "Samurai Tosh" за други позирания. Фитнес, калистеника, мускули, бодибилдинг, културизъм, натурален, естествен, набиране, лостове, уличен фитнес, ластици, спортуване, здравословно хранене, вечна младост, нестареене, подмладяване, ниски мазнини, fitness, bodybuilding, low body fat, leanк Самурай Тош #fitness #fitnessmotivation #healthylifestyle #фитнес #healthy #влог #влоги #lean #турник #фитнеспарк #хранене
Wednesday, December 7, 2022
In
Analysis,
Fun,
Software
//
//
Leave a Comment
Tabs disappear - Chrome feature/bug request :)
I've been very busy with a lot of research and R&D and one funny things which I discovered a while ago while working with Chrome (I use many browsers, including one "mine", controlled from a custom research accelerator that I'm using, unpublished).
I post this as a bug report, let's see will it be fixed/they would care about this rare condition.
...
I post this as a bug report, let's see will it be fixed/they would care about this rare condition.
...
1) a scrolling feature, scrolling controls to appear in order to see the hidden tabs;
2) smaller size of the icon-buttons or
3) at least a notification that the number of tabs per window is exceeded and a new window has to be created, or automatically creating one with a warning.
Yes, it is a bad practice to open so many tabs in one window, but if not using a more sophisticated session management and smarter bookmarking, it's a way to quickly store and mark a set of currently researched items which are for study/exploration/additional reading in a following session.
Thanks
