Yann Lecun cites a post, which is acknowledging that his ideas were correct etc.
* It is "dime a dozen", but people decades older than me who *literaly* repeated and ripped-off my suggestions and observations decades later, got prized with billions to *waste* and I am not even mentioned. They did it even in my own country, where one Bulgarian-Canadian became an "architect" of an institute in Sofia, with statements which were a *20 years late rip-off* of the above-cited essay, which were sold as "innovative" and ground-breaking :))), "for the first time in Eastern Europe" etc.
* As of "Dime a dozen"--> yes, or even "Five a dozen" -->
The current "supercomputer" of my lab is called "PETAK I", where "Pet" means "5": from: 1. "Pentium" (historically the CPU and brand on which TUM was created), 2. The CPUs of all nodes: Core i5 (all old ones, 11-14 years old models, LOL); 3. Five nodes of the cluster (the initial full configuration) 4. A parody CPU-name from a science fiction work from 2004 from that theory ("Pentium 5") and 5. In Bulgarian it also means "5 cents"... LMAO
"""Tunisia.AI
Yann LeCun may have been right about something important: next-token and next-pixel prediction are probably not the most efficient path to real world understanding.For years, the industry has been scaling generative models under the assumption that bigger models, more data, and more compute would eventually produce deeper intelligence. LeCun has been arguing the opposite: predicting every word or every pixel forces models to spend huge amounts of compute on surface details instead of learning the underlying structure of reality.That’s the core idea behind JEPA (Joint-Embedding Predictive Architecture): instead of reconstructing the world pixel by pixel, learn a compact latent representation and predict what happens next inside that space.The problem is that these models have historically been unstable. They suffer from “representation collapse,” where the latent space becomes too simple to carry useful information unless you add complex training tricks, auxiliary losses, or frozen components.A new paper, LeWorldModel (LeWM), shows a much cleaner approach. It trains end-to-end from raw pixels using only two losses: a next-embedding prediction loss and a Gaussian regularizer on the latent space. This drastically simplifies the training setup compared to prior approaches.The efficiency gains are striking. The model has around 15 million parameters, trains on a single GPU in a few hours, and can plan up to 48× faster than larger foundation-model-based world models, while staying competitive on several 2D and 3D control tasks. Its latent space also appears to capture meaningful physical structure and can detect physically implausible events in controlled environments.This doesn’t mean generative AI is a dead end. LLMs remain extremely powerful. But it does reinforce a key technical point: for world modeling and physical reasoning, predictive latent-space approaches may be far more compute-efficient than brute-force generation.The real shift might be this: not models that generate everything, but models that understand enough of the world to predict what actually matters."""
Todor Arnaudov
This idea, together with the prediction and next-token prediction (but in multi-scale, multi-precision hierarchy of resolutions of causality-control and perception), was published and explained nearly 25 years ago in Theory of Universe and Mind and presented during the world's first university courses in AGI in 2010 and 2011. Y.Bengio also rediscovered it 2017-2018 (Consciousness prior) and his example is almost literary repetition of an introductory definition from a treatise published about 14 years earlier. The author was a teenager, LOL.
Yann LeCun:
@Todor Arnaudov as I pointed out on another platform, ideas are a dime a dozen. The hard part, for something like this, is to implement it and to make it work.
The whole idea of hierarchical representations and learning by prediction is very old.
But learning hierarchies of representations didn't really work until convolutional nets were shown to do it in the late 1980s and more forcefully in the early 2010s (this took a while).
===
Todor Arnaudov:
Hi, first thanks for your answer as I didn't expect this honor. I don't disagree that there were earlier "prophets", I recently published a hyperbook with a related name (nearly 5000 pages in total), where one of the intros in one of the sections with collectons of related, prior and later work is a citation from the Holy Bible:
"There is nothing new under the Sun"
Some of the prior work doesn't get enough credit and is unknown, even the "fellow AI historian" Schmidhuber doesn't mention them, e.g. the Soviet lab of Bongard and his colleagues etc. (E.g. once I caught Chollet literary restating insights from the 1967 book "Проблема узнавания" - perhaps he didn't know; he also rediscovers definitions for general intelligence of mine, published in 2001 (he couldn't know about it) - see the link at the end and the reviews of the LLMs).
The Bible is called "The Prophets of the Thinking Machines: Artificial General Intelligence & Transhumanism: History, Theory and Pioneers; Past, Present and Future", SIGI-2025 - and yes, almost nobody will bother to even open it. :))
BTW, e.g. IMO your PhD student Marc’Aurelio Ranzato deserves more credit for his pioneering work in DL and his insights (which perhaps [are] ~ also yours) -- his work is credited in my historical collections here: https://twenkid.com/agi/Lazar_The_Prophets_of_the_Thinking_Machines_20-8-2025.pdf ~p.21.
I do agree that I had to push to implementations immediately (not your type of NNs though) and perhaps my claims would be accepted after I implement them all by myself (Or if I or somebody else had - 20 years ago with no collaborators or any funding, no mechanical Turks to labe a gazillion of data and computing iterations, compared to 20 years later and all the collected resources in all senses of the word: i.e. IMO the difficulty of the implementation is supposed to decrease and be "discounted" with time like in RL; an idea 25 or 50 years ago may end up more "valuable" than an implementation in the present - see generative AI and the final citation below)
* I know about your dismissive opinion about "ideas", e.g. your comments to Schmidhuber's recent challenge, that you also could find ideas in your unpublished notes or something etc. and I've listened to your answers to him since 2022, "The path towards autonomous AI..." - I remember you defended yourself with referring to Optimal Control etc.
However many works are proposals, theoretical etc. but still get recognized, while other prior ones - don't and are even "humiliated". Also the core novelty there in my reading of the paper was also matching the mentioned TUM (and too general, it was not an implementation too); in general it looked like another cognitive architecture, which were popular in the cognitive science and the AGI community decades earlier, perhaps I have to reread it.
* I understand that if you dismiss even the German, who is at a comparable status as yours or, say he has more ground to be believed that he is, then you (and almost anyone) wouldn't recognize the claimed "priority" or even just the "contribution" of some obscure "self-proclaimed" "crank" or the mentioned theory, no matter the evidence (maybe you wouldn't even bother to check any evidence or count it as "theory" or anything).
BTW, your recent work about the brain/humans as "not general ..." also matches and is closely related to my prior work/accounts, beginning in early 2000s, however with different interpretation of the observations. The limitations don't deny the concept of general intelligence and the possibility of general principles and modules (prediction-compression etc.) I may address the correspondences in a paper.
* Stack Theory is yet another Fork of Theory of Universe and Mind, SIGI-2025
* The first modern AI strategy was published by an 18-year old in 2003 and repeated and implemented by the whole world 15-20 years later: Bulgarian Prophecies: How would I invest one million for the greatest benefit for the development of my country? https://twenkid.com/agi/Purvata_Strategiya_UIR_AGI_2003_Arnaudov_SIGI-2025_31-3-2025.pdf (Bongard, 1967 vs Chollet,2024 p.169-170)
* BTW, cheers from Kyuchuk Paris - that's the district in the city of Plovdiv, where TUM was created. 🙂
* This is the world's first modern "AI strategy", 2003, repeated and implemented by "the whole world" 15-20 years later: https://twenkid.com/agi/proekt.htm
* It is "dime a dozen", but people decades older than me who *literaly* repeated and ripped-off my suggestions and observations decades later, got prized with billions to *waste* and I am not even mentioned. They did it even in my own country, where one Bulgarian-Canadian became an "architect" of an institute in Sofia, with statements which were a *20 years late rip-off* of the above-cited essay, which were sold as "innovative" and ground-breaking :))), "for the first time in Eastern Europe" etc.
* As of "Dime a dozen"--> yes, or even "Five a dozen" -->
The current "supercomputer" of my lab is called "PETAK I", where "Pet" means "5": from: 1. "Pentium" (historically the CPU and brand on which TUM was created), 2. The CPUs of all nodes: Core i5 (all old ones, 11-14 years old models, LOL); 3. Five nodes of the cluster (the initial full configuration) 4. A parody CPU-name from a science fiction work from 2004 from that theory ("Pentium 5") and 5. In Bulgarian it also means "5 cents"... LMAO
Also as I predicted in 2013 (counterintuitive to all "experts" up to just a few years ago, I namely wrote this article *because* of clueless "experts" predicted the opposite; they were later cited thousands of times for their *WRONG* world-model and wrong predictions):
"Creative Intelligence will be First Surpassed and Blown Away by the Thinking Machines, not the "low-skill" workers whose jobs require agile and quick physical motion and interactions with human-sized and human-shaped environment"
https://artificial-mind.blogspot.com/2013/10/creative-intelligence-will-be-first.html
" (...) For the intellectual jobs - it's much easier to pick a computer, run the appropriate software or connect it to the service,
and get it thinking - you already have decent cameras, microphones and many sensors even in smartphones. (...) The bottom line is that the "white collars" are more endangered in current-time economy. Perhaps that kind of economy could hardly survive the AGI revolution. I guess it may turn upside down for a while - the low-skill workers could get higher pay, because intellectual activities will be done in 1 ms for free... 😉 We, the smart guys (the smart asses, see "Super Smartasses" the graphical series ) wouldn't be needed by anyone... Not that we are needed now. :))"
* The prediction of the generative AI (however it could have been created by the late 2000s-early 2010s - it came *too late*, not too quick as Hinton and Bengio "complain"; not with gradient-descent of course):
-- Creativity is Imitation at the Level of Algorithms - An outline sketch of a possible path of development of the Artificial Intelligence "Emil"
* Petak I: https://github.com/Twenkid/SIGI-2025/blob/main/petaki.md
0 коментара:
Post a Comment