aiwithgrant
about me
LESSON 2 OF 126 min read

How LLMs Actually Work

How LLMs Actually Work

You don't need a PhD to understand how language models work. Here's the mental model that will make you a better prompt engineer.

The Core Mechanism

Large language models (LLMs) like GPT-4, Claude, and Gemini are trained on massive amounts of text. They learn patterns — how words relate to each other, how sentences flow, what typically follows what.

When you give them a prompt, they're essentially predicting: "What text would most likely come next?"

Why This Matters for Prompting

Understanding this changes how you write prompts:

1. Context is everything

The model uses your entire prompt as context for its prediction. More relevant context = better predictions = better output.

2. The model follows patterns

If you start with a formal tone, it continues formally. If you provide examples in a specific format, it follows that format. This is called "few-shot prompting" and we'll cover it later.

3. It doesn't "know" things — it predicts

The model isn't looking up facts in a database. It's generating the most likely response based on patterns. This is why it can sometimes be confidently wrong (hallucination).

Temperature and Randomness

Most AI tools have a "temperature" setting:

Low temperature (0-0.3): More predictable, focused responses. Great for factual tasks.
High temperature (0.7-1.0): More creative, varied responses. Great for brainstorming.

Key Takeaway

The better you set up the context and pattern for the model to follow, the better your results will be. That's prompt engineering in a nutshell.