In November 2022, OpenAI released ChatGPT and the world was never the same — or so the dominant narrative tells us. In reality, large language models (LLMs) had existed for years, and the underlying techniques (transformers, introduced by Google in 2017) were not new. What was new was accessibility: for the first time, anyone could converse with a language model through a simple interface.
But how does an LLM actually work? In simplified terms: it is a model that, given a sequence of words, predicts the most probable next word. It does this because it has been trained on enormous amounts of text — books, articles, websites, forums, source code — and has "internalised" the statistical patterns of human language. When it generates a response, it does not "reason": it calculates, word by word, the most probable continuation of the conversation.
This mechanism produces results that seem magical — coherent texts, fluent translations, effective summaries — but it has intrinsic limitations. The most famous are "hallucinations": the model produces false but plausible information with the same confidence as true facts. This is not a bug: it is a direct consequence of how it works. The model does not distinguish between true and false — it produces what is statistically probable in context.
Emily Bender and Timnit Gebru coined the expression "stochastic parrots" to describe these models. Like a parrot that repeats phrases without understanding them, an LLM produces text that seems meaningful without having any understanding of meaning. This does not mean it is not useful — it means we must use it consciously, knowing what it can and cannot do.
Generative AI is not limited to text: it produces images (DALL-E, Midjourney, Stable Diffusion), music, video, code. In all these domains, the principle is the same: a model trained on enormous amounts of data generates new output that is statistically coherent with that data. And in all these domains, the same questions emerge: who owns the training data? Who benefits from the output? Who is harmed?
An often-overlooked aspect is the relationship between generative AI and creative work. These models are trained on the work of millions of artists, writers and programmers — often without their consent and without compensation. When an AI generates an image "in the style of" an artist, it is extracting value from that creative work. The question is not only technical, it is political: who has the right to use and monetise other people's work?