How AI Thinks: Math, Not Magic

When you ask an AI a question, it doesn't "think" in ideas, memories, or feelings the way you do. There's no little digital brain in there pondering the universe. Instead, it works a lot like the autocomplete feature on your phone — just on a scale so massive it starts to feel eerie.

But here's the thing: once you understand six simple ideas — tokens, embeddings, probabilities, temperature, context, and attention — the magic disappears and something better takes its place: understanding. And that understanding will make you a much better user of these tools.

The Ultimate Autocomplete (Tokens)

Before an AI can even begin to respond to you, it has to chop your text up into small pieces. These pieces are called tokens. A token can be a whole common word like "sky", or just a fragment of a longer one — "unbeliev" and "able" might each be their own separate token.

Think of it like how a child reads using letter blocks — you don't hand them a whole book, you break it into individual pieces they can pick up and work with one at a time. Your prompt goes in, gets broken into tokens, and that sequence is what the model actually sees and processes.

graph LR
    A["Your Prompt"] --> B["Tokenizer"]
    B --> C["'The'"]
    B --> D["'sky'"]
    B --> E["'is'"]
    B --> F["'blue'"]
    B --> G["'today'"]
    C & D & E & F & G --> H["Token Stream — AI Model"]

Here's the key insight: the model doesn't read your whole message and then think about a reply the way you'd read an email. It reads the token stream and predicts the very next token. Just one. Then it adds that token to the stream and predicts the next one. And the next. It's doing this one-token-at-a-time prediction, thousands of times, until your response is complete.

The Word Map (Embeddings)

Here's a question worth sitting with: how does a math-based calculator understand that a "dog" and a "puppy" are closely related, but a "dog" and a "toaster" are worlds apart? It isn't looking things up in a dictionary. It's using something called Embeddings.

During training, the AI builds a massive, invisible, multi-dimensional map of words. When it encounters a word, it converts it into a list of numbers — coordinates on that map. Words that appear in similar situations in the real world end up geographically close to each other on the map. Words that rarely share context end up far apart.

graph LR
    subgraph Animals ["Animals cluster"]
        DOG["dog"] --- PUPPY["puppy"]
        CAT["cat"] --- KITTEN["kitten"]
        DOG --- CAT
    end

    subgraph Royalty ["Royalty cluster"]
        KING["king"] --- QUEEN["queen"]
        MAN["man"] --- WOMAN["woman"]
        KING --- MAN
        QUEEN --- WOMAN
    end

    subgraph Fruit ["Fruit cluster"]
        APPLE_F["apple — fruit"] --- BANANA["banana"]
        BANANA --- ORANGE["orange"]
    end

    subgraph Tech ["Tech cluster"]
        APPLE_C["apple — company"] --- MICROSOFT["microsoft"]
        MICROSOFT --- GOOGLE["google"]
    end

    DOG -.-|"far apart on the map"| TOASTER["toaster"]

Notice something subtle in that diagram: the word "apple" lives in two different places on the map depending on context. Near "banana" when you're talking about fruit; near "Microsoft" when you're talking about technology. The AI doesn't hardcode this — it infers it from billions of sentences where those words appeared together.

This is how the AI grasps the "vibe" of what you're saying, even though it has no idea what a dog actually looks like or what a toaster physically does. The meaning lives in the geometry of the map, not in any real-world understanding.

The Probabilistic Mind (Playing the Odds)

Because the model doesn't actually understand what words mean (it has no concept of a sky, or the colour blue), it works entirely on probability — statistical patterns it picked up from reading billions of documents during training.

At each step, the model looks at every token it has ever seen and assigns a percentage chance to each one. The question it's always answering is: "Given everything written so far, what is the most likely next piece of text?"

If the prompt is "The sky is...", the model ranks its options something like this — with the most statistically common continuation sitting at the top:

graph TD
    Q["The sky is ...?"] --> A["blue — 90%"]
    Q --> B["cloudy — 5%"]
    Q --> C["clear — 3%"]
    Q --> D["dark — 1.5%"]
    Q --> E["falling — 0.4%"]
    Q --> F["pancakes — 0.001%"]
    A --> WIN["Model picks: 'blue'"]

The model doesn't know what a pancake is. It just knows that statistically, in all the text it was trained on, the word "pancakes" almost never followed the phrase "The sky is". So it gets a vanishingly small probability. "Blue," on the other hand, appeared after that phrase constantly, so it sits at the top of the list.

The model picks a word, appends it, and then runs the exact same calculation for the next position. Every single token is its own fresh probability problem. That's all it's doing — over and over, at lightning speed.

Temperature (The Creativity Dial)

If the model always picked the absolute highest-probability token, the output would be robotic, repetitive, and painfully predictable. (Try finishing every sentence with the most statistically obvious word. It gets boring fast.) To fix this, there's a setting called Temperature.

Temperature doesn't change what the model knows. It changes how confidently the model leans on its top guess. Think of it as a dial that goes from "strict logician" to "unhinged poet."

graph LR
    T["Temperature Setting"]

    T -->|"Low 0.0 to 0.3"| L["Cold / Strict\nAlways picks the safest,\nhighest-probability token"]
    T -->|"Medium 0.7 to 1.0"| M["Balanced\nOccasionally picks the 2nd\nor 3rd option"]
    T -->|"High 1.5 and above"| H["Hot / Wild\nRare tokens get a real shot.\nCreative but risky"]

    L --> LU["Best for: Code, SQL, factual Q&A, math"]
    M --> MU["Best for: Blog posts, emails, chat"]
    H --> HU["Best for: Poetry, brainstorming, storytelling"]

Here's the practical upshot: when you're using an AI for anything where accuracy matters — writing code, answering factual questions, summarising a document — you want a low temperature. When you're using it for creative work, brainstorming, or drafting something that should feel alive and varied, nudge that temperature up.

But push it too high and you get the digital equivalent of someone who's been awake for three days: technically producing sentences, but saying things like "The sky is pancakes."

Context (The Clues You Give the Calculator)

Context is simply the clues the AI uses to cheat at the guessing game. When you type something, it doesn't just look at your last word. It looks at the entire conversation so far and feeds all of it into the probability calculator before deciding what comes next.

To see why that matters, let's use one phrase — "I am from" — and watch what happens as we add more context around it.

The Blank Slate (No Context)

You open a fresh chat and type only: "I am from". The AI has almost no clues. It falls back on how people generally talk across the entire internet, so its odds are spread wide and generic:

graph TD
    Q["I am from ..."] --> A["London — 8%"]
    Q --> B["New York — 7%"]
    Q --> C["India — 5%"]
    Q --> D["out of town — 2%"]
    Q --> E["... dozens of other places"]
    A --> R["Generic answer — could be anywhere"]

Because there are no clues, the AI will likely pick a common placeholder or respond with a question like "Where are you from?" — it genuinely doesn't know, and the math reflects that.

Shifting the Odds (Adding Context)

Now imagine your conversation looked like this before that same phrase:

You: "I love deep-dish pizza, the Cubs, and the freezing wind off Lake Michigan."

You: "I am from"

The AI now has a context window — its short-term memory of everything you've said. It feeds all of those previous words into the calculator before it guesses the next token. The math changes completely:

graph TD
    CTX["Context: deep-dish pizza + Cubs + Lake Michigan"] --> Q["I am from ...?"]
    Q --> A["Chicago — 99.9%"]
    Q --> B["Illinois — 0.09%"]
    Q --> C["London — 0.0001%"]
    A --> WIN["Model picks: 'Chicago'"]

By giving it context, you eliminated millions of possible wrong words and forced the probability engine to converge on exactly the right answer. You didn't make the AI smarter — you just gave it better clues.

The Invisible Context (System Prompts)

There is one more layer to this that most people don't realise. Even when you start a brand-new chat, the AI already has a tiny bit of hidden context — before you've said a single word.

Developers feed the model invisible instructions called a System Prompt. It looks something like this:

sequenceDiagram
    participant Dev as Developer
    participant AI as AI Model
    participant You as You

    Dev->>AI: System Prompt (hidden)
    Note over Dev,AI: "You are a helpful assistant."
    Note over Dev,AI: "Do not use bad words."
    Note over Dev,AI: "Today's date is August 2026."
    You->>AI: "I am from"
    AI->>AI: Combine system prompt + your message
    AI->>You: "Chicago" (context-informed answer)

So at every single step, the AI is combining three things: your words, the history of the current conversation, and its hidden system instructions — all to calculate the most likely next token. The more relevant signal you give it, the tighter and more useful that calculation becomes.

The Highlighter (Attention Mechanism)

Here's something that sets AI apart from a simple word-guessing program: it doesn't process tokens one at a time in order, left to right. It reads all the words at once and draws invisible connecting lines between the ones that relate to each other. This is called the Attention Mechanism — and it's the reason AI can handle nuance and ambiguity far better than you'd expect from a probability calculator.

Think of it as giving the model a set of highlighters before it makes any prediction. Consider this sentence:

"The bank of the river was muddy, so the fishermen slipped."

When the AI reaches the word "bank", it faces an ambiguity problem — bank can mean a financial institution or the edge of a river. The attention mechanism immediately scans the entire sentence and draws weighted connections:

graph LR
    BANK["bank"]

    BANK -->|"high attention"| RIVER["river"]
    BANK -->|"high attention"| MUDDY["muddy"]
    BANK -->|"high attention"| FISHERMEN["fishermen"]
    BANK -->|"near zero"| DEPOSIT["deposit money"]
    BANK -->|"near zero"| ATM["ATM / finance"]

    RIVER & MUDDY & FISHERMEN --> RESULT["Meaning resolved: riverbank"]
    DEPOSIT & ATM --> IGNORED["Financial meaning ignored"]

The moment "river", "muddy", and "fishermen" light up as strongly connected to "bank", the financial meaning collapses to near-zero probability. The AI resolved the ambiguity not by guessing from position, but by reading the room — scanning the whole sentence at once and weighing every word against every other.

A Dynamic Example: The Apple Game

To see how powerful this is, consider how Attention prevents the AI from reading word-by-word like a rigid dictionary and instead lets it understand shifting context dynamically:

In short, Attention is the math that lets an AI say, "Based on the words 'sweet' and 'healthy' right now, I need to pay 99% of my attention to the fruit definition of 'apple' and 0% to the iPhone definition."

Attention Across the Conversation

The same mechanism works across your entire chat history. When you ask a follow-up question ten messages later, attention doesn't just look at your last sentence. It scans back through the whole conversation and draws connections between your new question and the most relevant things said earlier.

sequenceDiagram
    participant You
    participant AI

    You->>AI: "I love deep-dish pizza and the Cubs."
    You->>AI: "The wind off Lake Michigan is brutal."
    You->>AI: "Anyway, I am from"
    AI->>AI: Attention scans all 3 messages
    Note over AI: "pizza" + "Cubs" + "Lake Michigan"
    Note over AI: All roads point to Chicago
    AI->>You: "Chicago"

Every response is the product of the AI simultaneously attending to — weighing, connecting, highlighting — all the relevant words across everything it can see. That's why a longer, richer prompt almost always produces a sharper answer. The attention mechanism has more signal to work with.

Conclusion: Math, Not Magic

There's no mind inside a large language model. No ghost in the machine. What there is, is a breathtakingly large mathematical function that is extraordinarily good at one specific task: given what has been written so far, predict the most sensible next piece of text.

Tokens are the raw material. Embeddings are the map that gives words meaning. Probability is the engine. Temperature is the creativity dial. Context is the clue set. And attention is the mechanism that connects every clue to every other, making sense of the whole picture at once.

Once you internalise all six of these ideas, a few things shift. You stop being surprised when AI makes things up — it's just a probability miss. You understand why "apple" means something different in two conversations. You know why a richer prompt produces a sharper answer. You know which temperature to reach for. And you understand why the model resolved "bank" as a riverbank without you ever having to explain it. It wasn't a guess. It was embeddings and attention doing the math together.

The tool doesn't become less impressive once you understand it. If anything, knowing it's doing all of this with math alone makes it more remarkable.