POPA ReconHow LLMs Actually Work
How LLMs Actually Work An LLM is basically a machine that reads text and guesses the next word, over and over, really fast • Models can't read letters so it chops texts into chunks ("tokenization") -> each chunk gets a number • Each number gets looked up in a giant table that turns it into a long list of numbers (a "vector") • Words with similar meanings end up with similar vectors. "King" sits near "queen," "Paris" near "France" • Tracking word order. A vector for "dog" is the same whether it's the first or fifth word, which is a problem since order changes meaning. So the model adds positio…
The excerpt is quoted from a public Telegram channel, which owns its text.