What are embeddings?

An embedding is a way of representing the meaning of something, such as a sentence or an image, as a list of numbers called a vector. Embeddings are built so that pieces of text with similar meanings end up with similar lists of numbers.

Embeddings power search that works on meaning rather than exact words, and they’re the foundation for AI tools that look through your own documents before answering. You’ll never see them on screen, but knowing what they do helps when you’re choosing a search or chat tool that uses AI, or trying to work out why it isn’t finding the right thing.

What does an embedding represent?

Computers can’t compare the meaning of two sentences directly. Take “How much is a trial class?” and “Fees and plans.” They share almost no words, so comparing the text alone wouldn’t reveal that they’re about the same thing. The idea behind embeddings is to express meaning as numbers so that the numbers can be compared instead.

To get a feel for this, let’s score a question sent to a cooking school, along with a few of the school’s FAQs, on three homemade scales.

Text	About money	About dates	About location
How much is a trial class? (incoming question)	0.9	0.2	0.0
Fees and plans (FAQ)	1.0	0.0	0.0
When does the next course start? (FAQ)	0.1	1.0	0.1
Getting to the school (FAQ)	0.0	0.1	1.0

“How much is a trial class?” has become the list [0.9, 0.2, 0.0]. That’s a tiny embedding. Comparing the lists, you can see the question is much closer to “Fees and plans” than to “Getting to the school.”

Real embeddings use far more than three numbers, anywhere from several hundred to well over a thousand (768 or 1,536, for example). The scales aren’t chosen by people either. The model works them out on its own from huge amounts of text. That’s why you can’t read what any single number means. Taken as a whole, though, the list captures the meaning of the text remarkably well.

How is “close in meaning” calculated?

You can think of a list of numbers as an arrow (a vector) with a direction and a length. Arrows for texts with similar meanings point in nearly the same direction.

A common way to measure closeness is cosine similarity. It expresses how closely two arrows point the same way as a number from -1 to 1. The closer it is to 1, the more the arrows line up, which means the meanings are closer. Another approach measures the distance between the tips of the arrows.

Using the scores from the table above, the cosine similarities come out like this:

FAQ compared with the incoming question	Cosine similarity
Fees and plans	about 0.98
When does the next course start?	about 0.31
Getting to the school	about 0.02

“Fees and plans” is the closest, so that FAQ is the best candidate for the answer.

So why does closeness in meaning turn into closeness in numbers? Embedding models are trained on large amounts of text so that sentences used in similar situations, or that paraphrase each other, end up with similar lists of numbers. Because the model has learned that words like “how much,” “price,” “fee,” and “cost” show up in similar contexts, it places them near each other even though the words themselves differ.

How are embeddings made?

You create an embedding by sending text to a dedicated AI model called an embedding model. It returns a list with a fixed number of values.

When embeddings are used for search, the process usually looks like this:

1. Split the documents to be searched (FAQs, articles, and so on) into reasonably sized pieces.
2. Create an embedding for each piece and store it.
3. When a question comes in, turn the question into an embedding with the same model.
4. Look through the stored embeddings for the ones closest to the question’s embedding.

The key point is to use the same model for both the documents and the questions. Different models build their lists of numbers in different ways, so comparing across models is meaningless. If you switch models, you have to re-create the stored embeddings.

Documents are split up because there’s a limit to how much text one embedding can take in, and because cramming many topics into a single embedding blurs what it’s about. On the other hand, pieces that are too short lose their surrounding context and can become harder to match.

A storage system designed to hold large numbers of embeddings and quickly find the closest ones is called a vector database.

How is this different from keyword search?

Most traditional site search is keyword search, which looks for documents that contain the words you typed. Meaning-based search with embeddings is good at different things.

	Keyword search	Meaning-based search with embeddings
What it compares	Whether the words match	Whether the meanings are close
Good at	Exact terms like product codes, product names, and error numbers	Paraphrases, everyday phrasing, and vague questions
Weak at	Different words for the same idea, like “how much” and “fees”	Exact matches on codes and proper names

For example, if someone searches a class website for “can’t make it this week” and the FAQ is titled “Absences and make-up lessons,” keyword search may come up empty. Meaning-based search has a good chance of picking it up as a related topic. But if someone searches for a specific name like “Course B-12 schedule,” keyword search is the more reliable option.

That’s why many systems run both searches at the same time and merge the results. This is called hybrid search.

What are embeddings used for?

• Site search: showing relevant pages even when visitors describe things in their own words.
• Sorting inquiries: grouping incoming messages into topics such as pricing, scheduling, or problems.
• Helping with replies: finding the FAQ closest to what someone wrote in a contact form, as a starting point for a reply.
• Spotting similar content: finding articles that overlap or questions that keep coming up.
• AI that answers from documents: finding material related to a question and passing it to a large language model (LLM).

Embeddings aren’t limited to text. Images and audio can be turned into embeddings too, which makes it possible to, say, find products that look like a photo.

That last use in the list is called retrieval-augmented generation (RAG). The embeddings handle the searching, and the LLM reads what was found and writes the answer. To learn more, see What is a large language model (LLM)? for how LLMs work, and What is retrieval-augmented generation (RAG)? for the full flow.

What should you watch out for?

• Close doesn’t mean correct. Meaning-based search returns the closest match even when no document actually answers the question. Set a minimum score, and decide what to show when nothing fits.
• Models differ in which languages and subjects they handle well. If your documents aren’t in English, choose a model that supports that language or is multilingual, and test it with real questions and documents.
• When you edit a document, re-create its embeddings. If you rewrite an FAQ but keep the old embeddings, searches will still be matched against the old content.
• Treat embeddings with the same care as the original text. They look like plain lists of numbers, but research has shown that much of a short original text can be reconstructed from its embedding. Protect embeddings of text containing personal information the same way you protect the text itself.

For a closer look at how embeddings work, see the embeddings module in Google’s Machine Learning Crash Course.