THE SACRED COMPUTER - СВЕЩЕНИЯТ СМЕТАЧ, a.k.a. ARTIFICIAL MIND: Thinking Machines, Creativity and Human Development. A pioneering virtual Research Institute for Artificial General Intelligence (AGI), Cosmism and Transhumanism, AI, Software, Interdisciplinary General Research, Creativity, Versatility, Universal Men ~ Twenkids etc. Created in 2000. (...)
Реалистично фентъзи, вдъхновено от Властелинът на пръстените - по пътя към Хилядолетния рай на светлото бъдеще. Сатира, комедия, фентъзи, анимация.
Произведен с "Arnoldifier", системата за дийпфейк кино, Twenkid FX Studio (кадрите от "Звездна симфония в Чепеларе") - създадени в Пловдив от Тош/ "Свещеният сметач".
Използва кадри от "Властелинът на пръстените III", реж. Питър Джаксън. OpenDalle11 за заглавната картина. DaVinci Resolve (краен монтаж).
С участието на модели на Мария Габриел и Григор Сарийски
Гледайте филма: https://youtu.be/KXC68MbczMg#българия#еврозона#политика
Гледайте и първата част: "Амбългъл: Властелинът на пръстените на България I" (дийпфейк).
Сценарий, звук, монтаж, актьор в ролите, програмист: Тош
Дийпфейк система: Тош.
Ambulgul is a multi-personality creature which is searching his precious: the Power in the forests of Bulgaria. The new work from the series "Arnold Scwarzenegger: The Governor of Bulgaria" which was created using Arnoldifier, the deepfake library developed by The Sacred Computer to work with grayscale images for higher performance, by extending Deepfacelab 2.0. Find it on Github. https://github.com/Twenkid/DeepFaceLab-SAEHDBW
This is the most complex Deepfake movie I've produced so far, utilizing almost two-figure number of different faces. It could be more advanced, but some compromises were made for faster creation etc., some directions for improvement of "Arnoldifier" are shared in the github repo. Also of course - a more powerful GPU and better source video*, I didn't use one with a high quality, it wasn't critical, actually a partial imperfection and "mystery" in the images is fine for animated caricatures and fantasy.
Амбългъл: Властелинът на пръстените на България: I част. [Сатиричен дийпфейк филм: анимация, комедия, фантастика, фентъзи, драма, 3 мин. Цветност: част цветен, част черно-бял. Език на звука: български и английски (със субтитри на български в картината). Автор: Twenkid Studio и авторите на The Lord Of the Rings. Премиера 30.1.2024
Using my Arnoldifier (DeepfaceLab-SAEHDBW); Wav2LipHD-Super resolution, RealTimeVoiceCloning and DALL-E 3. Enjoy! Like, Share and Subscribe, of course! :))
ARNOLDIFIER: 3X Higher Performance Version of DeepfaceLab 2 Deepfake Library, achieved by refactoring it to work with Grayscale images. It achieves similar quality with either 3X less VRAM, or 3 times faster, 3X more detail/higher resolution faces for the same size of the models in VRAM and disk. All high quality models in the video are trained on a Geforce 750 Ti 2 GB (a 2014 entry-level GPU) - Arnoldifier allows training of quality and complex datasets with 192x192 or even 256x256 faces. Make your amazing deepfakes with any low end GPU and imagine the quality even with 1060 6 GB ( performance of a 3080, if it had 18 GB RAM ), or turning a Geforce 1070 GPU 8 GB into a 3090-24-GB-equivalent monster. The only trade-off is the color, but there is a colorization prototype in Arnoldifier, see in the video, and a post-production colorization can be done with additional tools as well.
Watch the series "Arnold Schwarzenegger: The Governor of Bulgaria", produced with the library and find it on Github. https://github.com/Twenkid/DeepFaceLab-SAEHDBW
Subscribe, like, share, comment - if you want to see more videos about the library, tutorials, code review etc. Thanks and see you in the next video!
I had the colorization in mind before I started the refactoring of Deepfacelab to include grayscale models, and lately I added this functionality, for now as a POC, to be published later. The colorization is done with a Pix2Pix model (based on the example on Colab with the facades, maps etc.), trained on the faceset of the color video - grayscale faces converted to color. There is a step of color stabilization, which was required for more pleasant output, because without that there was slight, but noticeable flickering. The experiments for now were only on a single video/segment (about 1000 frames training of the pix2pix model) and without pretraining on other/various faces - that is something to be done in the future.
I am still pushing the limit of a GF 750 Ti 2 GB - so now it can produce 192x192 color lip-synced deepfakes with reasonable quality. The SAEHDBW DF-UDT model is about 345 MB initially (a bit more when trained), the Pix2Pix model is about 131 MB.
Note that the pix2pix model did fit in GPU only 128x128, but a sharpening at the end of the pipeline improves the image even more than the original grayscale one. Training at 256x256 on the CPU is a possible option, too, because the pix2pix model seems to be fast and also possibly the applied color stabilization can repair some fluctuations, i.e. possibly it can be not perfectly trained and still capable to produce decent results - that is to be verified with other videos. I haven't tried to colorize the Arnold's model yet.
The glasses of Stoltenberg are in most cases reasonably depicted, except a few little glitches from the grayscale model.
After investigation of the properties of the colorized faces, debugging of the merging, there was a successful application of an idea for stabilization of the colorized output and merging with precomputed faces (for other usages as well, e.g. prerendered 3D-models or synchronously performing faces etc.). In the video example below the output is also sharpened after merging (whole frame) - it needs to be per face only etc. or to have some antialiasing eventually.
The color-gamma stabilization is done by first probe-rendering all faces, computing their total pixel weight per frame and the average of all frames, then adjusting the gamma for each frame according to the average in order to flatten the fluctuations: if the face is too dark - it gets lighter and vice versa. Indeed, this phenomenon itself is to show some intrinsic properties of the pix2pix model.
Finally there is sharpening and merging is performed using these corrected faces.
...
* The neural model didn't capture the blue tint of the model's eyes, but it had a little excuse - the color of the eyes in the video varies and even there are frames where the ground truth eyes are different colors: one is very gray-blueish-purple and the other one - brown.
Advanced deepfake fun: "Arnold answers NATO's Stoltenberg's proposals for Bulgaria", Part IV from the Deepfake series "Arnold Schwarenegger: The Governor of Bulgaria", demonstrating the custom DF library DFL-SAEHDBW for grayscale etc.
DFL-SAEHDBW - K.P. etc. to Arnold models
Select video, edit "stunt-double" sequences
Original Script, Directing and humor,
RealTimeVoiceCloning, actors TTS
Wav2Lip lipsync; a modification for higher quality output (images per frame, not DIVX video)
Finetune DFL-SAEHDBW for repair and upscaling of the Wav2Lip output (it's original output is low quality with glitches and 96x96, the DFL models are 192x192 and high quality)
Merge improved lip-synced sequences
Recording an Easter egg cover song according to the script (LOL)
Another demo of the capabilities of my new modification of DFL2.0 - this is trained just on a Geforce 750 Ti. See the previous post and github: https://github.com/Twenkid/DeepFaceLab-SAEHDBW
Regarding the modification of the library to work with grayscale images instead of color ones, that I did in order to create the film with theoretically 3 times higher performance and/or details with the same hardware - see info in future posts.
Български
Встъпително обръщение на Арнолд Шварценегер - Губернаторът на България в интервюта в Народното събрание, пресконференции в министерския съвет и БТА, Би Ти Ви, Дневник и др. Какви са целите на Терминатора на България? Кого иска да представлява? Какви са идеалите на Партията и пр. Арнолд е избран от народа, за да донесе политическа стабилност в най-стария щат на Балканите и да приведе Зелената сделка в действие - така както го направи в родния си щат Калифорния по време на двата му предишни мандата.
Автор: Тодор Арнаудов - Тош/Twenkid
Следва продължение
English
The inspiring inauguration Speech of the new Governor of Bulgaria: Arnold Schwarzenegger from his interviews in the Parliament, the council of ministers, for bTV, Press conferences in BTA, Dnevnik etc. What are his goals, who he wants to represent, what are the ideals of his party etc. Created by Todor Arnaudov - "Twenkid". Arnold will be back!
Idea, production design, story and script, programming, dataset collection, training, retouch, editing, music directing, timing, sync etc.: Todor
Deepfake software developers:
1) DeepFaceLab 2.0 by iperov et al.
2) DeepFaceLab-SAEHDBW by twenkid - the grayscale model and mode for the movie, allowing me to train a 3 times bigger model on the same GPU.
