What is a large language model (LLM)?

A large language model (LLM) is an AI model that has read enormous amounts of text, learned how words tend to fit together, and uses that to write text that follows a question or instruction. Chat assistants that answer in plain language, tools that draft emails, and features that summarize long documents are usually built on an LLM.

LLMs are genuinely useful, but they don’t look facts up the way a person would before answering. Once you understand how they work, it gets much easier to tell which jobs you can hand over and which answers someone should double-check.

What does an LLM actually do?

In one sentence, an LLM predicts what text is most likely to come next. Your phone’s keyboard does a tiny version of this when you type “Thanks so” and it suggests “much.” An LLM does the same kind of thing at a far larger scale, while taking much more of the surrounding context into account.

Say a yoga instructor asks, “Write a thank-you reply to someone who just booked a trial class.” The LLM picks one piece of text that would naturally come next. It then adds that piece to everything so far and predicts the next one. After repeating this hundreds of times, you get something like “Thank you for booking a trial class with us. We’re looking forward to seeing you.”

What is a token?

The unit an LLM predicts at each step is called a token. A token can be a whole word, part of a word, or a single character. Each model has its own way of splitting text, so the same sentence can come out as a different number of tokens in different models.

Tokens are also how usage is measured. Many AI services count the tokens in the text you send and the text the model writes, and base usage limits or pricing on that count.

How does an LLM learn? (Training and inference)

An LLM’s work falls into two stages: training and inference.

Stage	What happens	Example
Training	The model’s internal numbers (parameters) are adjusted, using huge amounts of text, so it gets better at predicting words	Practicing over and over on enormous amounts of text to guess the next word or a hidden word
Inference	You give the finished model some text and it produces an answer	Asking it to “draft a reply to this email”

Training mostly consists of guessing the next part of a text, or a part that has been hidden. Repeated at enormous scale, this practice stores grammar, word meanings, and common patterns of explanation in the model’s parameters, which are a vast set of numbers. Most LLMs then get extra training to follow instructions (such as instruction tuning and adjustments based on human feedback), which is what lets them respond to questions conversationally.

When you use an AI chat, you’re running inference. Inference doesn’t change the model’s parameters. That means an LLM only knows what was in the text it was trained on. If your prices changed or a new regulation came out after training, the model on its own can’t answer correctly about it.

If you’d like to go deeper, Google’s Machine Learning Crash Course module on large language models explains the mechanics in more detail.

How much can it read at once? (The context window)

There’s a limit to how many tokens an LLM can handle in a single round of inference. This limit is called the context window. Think of it as how many papers you can spread out on your desk at one time.

Everything below has to fit in the context window:

• Instructions the AI service gives the model behind the scenes
• The conversation so far
• Files you attach or text you paste in
• The answer the LLM is about to write

An LLM doesn’t carry memories from one conversation turn to the next. A chat seems to remember what you said earlier because the service sends the earlier conversation to the model again with each new message. When a conversation gets very long, older parts may be dropped or summarized to stay under the limit, so something you said at the start may stop shaping the answers. If a condition really matters, restate it when you need it.

Why do confident mistakes happen?

When an LLM states something false in a confident tone, it’s called a hallucination.

The cause lies in how LLMs work. An LLM doesn’t look up the correct answer and write it down. It writes whatever reads naturally as the next part of the text. So even when it doesn’t know something, it can still produce a convincing-sounding answer.

Imagine someone who runs an online course asks, “How much does my course cost?” The LLM has never seen that course’s price list, so it may give a plausible figure based on what similar courses tend to charge. In the same way, it may produce web addresses that don’t exist or describe rules that are out of date.

Better wording in your question alone won’t make hallucinations go away. What helps is giving the LLM material to base its answer on, and having a person check anything important.

What changes when you give it sources?

Basing an LLM’s answer on specific material is called grounding. The basic approach is simple: put the material that contains the answer into the context window along with the question, and instruct the model to answer only from that material.

In the online course example, if you provide the price table, the LLM can answer from the actual table instead of guessing from typical prices. If you also have it say which material it used, it becomes much easier for someone to check the answer later.

When there’s a lot of material, though, it won’t all fit in the context window. A common solution is to first search for the parts that relate to the question and pass only those to the LLM. This is called retrieval-augmented generation, or RAG, which is covered in What is retrieval-augmented generation (RAG)? The search step often relies on embeddings, which represent the meaning of text as lists of numbers.

Even with sources, the material itself can be out of date, and the LLM can still misread it. Keeping your material current and making it easy to trace an answer back to its source are what let you rely on the results with confidence.

Summary: tips for using LLMs at work

LLMs can be a big help with language work such as drafting, summarizing, rewording, and sorting incoming inquiries. If you run a small business, keep these points in mind:

1. For facts like prices, schedules, and policies, give the model your own material before asking it to answer.
2. Have a person read anything that goes to customers or involves money or commitments before it’s sent.
3. Never enter passwords or card numbers. If you handle personal information, check how the service treats your data.
4. Start small with what LLMs do well, like drafts and summaries, and widen what you hand over once you know where they tend to slip.