Humans also experience "model collapse" like LLMs - Todor's comment on the talk "Model Collapse Ends AI Hype" ... LLM new information generation

Todor Arnaudov's comment on the conclusions of: "Model Collapse Ends AI Hype

Theos Theory и още 2

4,8 хил. показвания преди 2 дни #AI #aihype #llm

https://www.youtube.com/watch?v=ShusuVq32hc

Todor: Humans suffer the same phenomenon*, but even worse.

What if the next generation of a human couple (living isolated) learns only from the mental output of the previous generation? What if you had to learn only from what your parents have written or what they can write? (Even the stone age populations weren't trained like that.)
In addition, humans don't learn and their behavior is not defined only from the textbooks they read.

See the early AGI in 2001-2004 and 2000s - Theory of Universe and Mind (TUM). Language is not the text, the linear sequence of letters, "tokens", "symbols", as it is still confused in many occasions; the complete and functional language is multimodal, multiscale, multiprecision, ... "omnimodal" in modern terms and is connected with the will, or "action", "agentic" etc.

It's  "grounded", mapped to other modalities, to other kinds of data and transformations. There is inter-level, inter abstraction level, range-level, precision, domain, context ... etc. mapping, general "grounding".

So, what if a human learns only from books or notes, written by his immediate parents: mother and father? (How did he learn to read is another problem.)

They have to create the entire "curriculum" (and in the current LLMs it's not the human way, see again Theory of Universe and Mind and the "зародиш на разум" - Seed AI).

Your mother and father have to write you the primary book to teach you the alphabet, to write and to read, to calculate, the textbooks and corpora of texts for all grades, your university textbooks etc., the common sense corpora, the novels, the news, dictionaries, references, manuals, programming code, programming code of the computers you use, their electronic schematics, design of all the equipment, architecture of the buildings, mechanical engineering, material science, chemistry, ... etc. and the child should learn it itself *by reading* this body of work, "input-output" pairs, possibly RLHF, but created by the parents and fixed.
What have your parents written? 

<li> How much of the dataset could an "average" of even "smartest" parents complete, if they have to work in a "closed books"-like exam as well, only what they have already remembered?<li> What did they remember from what they studied? It would be ridiculous.

Most humans struggle with writing and understanding complex texts in general even for the very moment of reading, e.g. measured by answering questions on a long sentence with entangled clause structure, what if they have to explain it? 

What humans believe they know or understand, if measured as they capability to express or encode it, write it, recreate it, is ridiculously smaller than what they can actually express givn such strict requirements, it is a passive "believe of knowledge and understanding".

Then, their children could be allowed to start interacting after this kind of instruction; then they shouldn't learn from everything they know or that they believe they know, say in text only, and their children can only learn from that body of work, without additional interaction with the real world or other resources? the interaction).

//////Another take:

We see that humans have studied a lot of things and they believe they understand them, but in fact they barely remember a tiny portion in their "personal" memory, the speed of output is so low that even if they could reproduce a lot of it, it would take too long with current means and technologies*; they need to constantly help themselves with references and external tools where a lot of their "hallucinated" scale of memory and skills have always resided and they only "orchestrate" with their brain and body - just like current LLMs; finally, like the LLMs, humans often make silly cognitive mistakes, although they believe how clever they are. 
Probability estimations are a famous case, although I think it's too unsurprising as when asked verbally humans do not think this way, they obviously don't follow the procedure or the formulas, but may say something ad-hoc, be confused and misguided by the way the questions are formulated etc. (Which item is heavier? A kilogram of wool or a kilogram of iron? Ask a child or an adult who is under stress.) - if you know the LLMs weaknesses, that would sound familiar to you, too. (...

See externalism, extended mind - also part of the foundational TUM.

* TOUM, also TUM https://github.com/Twenkid/SIGI-2025/

* See "Humans are far worse than LLMs in many ways - a brief analysis", 2025
https://artificial-mind.blogspot.com/2025/08/humans-are-far-worse-than-llms-in-many-ways.html 
~38 min: ~ Sampling from good distribution is easy, but the saved information is about the quantity needed to find the good distribution.

Can LLM produce new information etc.

The ultimate source of "new" information is the Image/Intuition (Schopenhauer), нагледът (Bulgarian) or "empirical data" (and the internal intuition, Will/introspective discoveries.

Final conclusions, 44 min:

Which one creates new information?1.Randomized sampling?
2.Interpolation?
3.Extracting implications?

How exactly all of these are defined, including "new" and the evaluator-observer, the environment (the Universe) etc.?
The definitions will give the answer.

Either there is "Nothing new under the sun" or "new" given this and that requirements, memory, definitions.

See also the classic TOUM, where this is already addressed as well, since 2001, "Man and Thinking Machine: Analysis of the possibility of creating a Thinking Machine and some disadvantages of humans and organic matter in comparison". "Creativity is imitation at the level of algorithms", 2003. For example a discussion on the Kibertron's forum in 2004 where I reasoned  about whether any system inside the Universe can really be flexible, if the lowest level laws/rules/machine code of the Universe is fixed or at least the subuniverses cannot change them. They may appear as "modifying" their code or "evolving", but in fact they would only "execute what they were told" - by the machine code of the Universe, the mother virtual Universe, their actual description at the highest possible resolution of causality and control, and not the format, parameters, features, capabilities, low resolution virtual universes which they can perceive, comprehend and control; what they believed they modified or could modify is in fact a low resolution representation and their own image, representation, believe, "hallucination" about what their behavior, intentions, structure etc. actually reflect and "are" in their foundations.

* Imagine also a relaxed requirements in "normal circumstances of education", after say 7 or 11 or 12 years or whatever age, if you can use only your own notes for reference. Only what you have handwritten or typed yourself, and if you are "unimodal", text-to-text, you are not allowed to read anything else, to have other sensory inputs etc.  

** impossible for a human, because everything is multimodal