Embeddings: Meaning as Numbers

A computer does not know what a cat is. It cannot understand meaning, but it is very good at one thing: working with numbers. So here is the trick. We turn every word or sentence into a list of numbers, in a way that gives similar meanings similar numbers. That list is called an embedding. Once meaning is numbers, "how similar are these two things?" becomes a simple question of distance.

Let's start small. Below, every word is just two numbers, so we can draw it as a point on a map. Words about animals sit in one area, food in another and vehicles in a third. Hover over or tap a word to see its numbers and its closest neighbors.

Look at "hot dog". It has the word "dog" in it, but it sits with the food, because what matters is meaning, not spelling. And "jaguar" lands between animals and cars, because it can be both an animal and a car brand. Real embeddings work the same way, with one big difference: they have hundreds or even thousands of numbers, and those numbers are learned by a model from huge amounts of text, not chosen by hand.

How do we measure how close two embeddings are? In math, a list of numbers like this is called a vector, and we can draw it as an arrow starting from the center. A common way to compare two of them is to look at the directions of their arrows. If they point the same way, the meanings are similar. This is called cosine similarity. It gives a number between -1 and 1. Pick two words and watch the angle between them.

Arrows that point the same way get a score near 1. Arrows at a right angle get 0. Arrows that point in opposite directions get -1. Be careful with one thing, though: opposite direction does not mean opposite meaning. Words like "hot" and "cold" usually end up close together, because they show up in very similar sentences. An embedding captures how words are used, not whether they mean the same thing.

Two numbers are not enough to describe much. Every extra number adds a new direction that things can differ in. Below, each word has three numbers that we can actually read: how much it is about animals, about food and about vehicles. Move the sliders to build your own vector and see which word is closest.

A fish is both an animal and food, so its numbers mix the two. With more numbers, an embedding can hold many of these shades of meaning at once. In a real model, nobody decides what each number means. The model learns them, and most single numbers have no simple name like "animal".

This is where embeddings become really useful. Sentences can have embeddings too. To search by meaning, we turn the question into an embedding and pick the sentences whose embeddings are the most similar. Compare it with a simple keyword search, which only looks for shared words.

Keyword search misses sentences that use different words for the same idea, and it can pick up sentences that share a word but not the meaning. Search by meaning can find the right sentences even when no words match. This is why many search tools, recommendation systems and AI assistants use embeddings to find related content.

That is the whole idea. An embedding turns meaning into numbers, and similar meanings point in similar directions. Just keep in mind what "similar" means here. Embeddings measure how alike things are in the way they are used, not whether something is true. A sentence and its exact opposite can still be very close.

@aykutkardas