Every embedding in my production system is a point in 1,024-dimensional space. Human beings top out at three. So when I say “similar documents cluster together,” you have to take my word for it — the space where the clustering happens is fundamentally invisible.
Unless we flatten it.
The problem: too many dimensions
Dimensionality reduction is the family of techniques that project high-dimensional data down to two or three dimensions while preserving as much structure as possible. The catch is in that word possible — you cannot flatten 1,024 dimensions into 2 without losing information, any more than you can flatten the globe into a paper map without distorting Greenland. Every technique chooses what to sacrifice.
The three most common you’ll encounter:
- PCA (principal component analysis) finds the directions of greatest variance and projects onto them. Linear, fast, deterministic — and usually disappointing for embeddings. Semantic structure isn’t linear.
- t-SNE preserves local neighborhoods beautifully but is slow, fiddly about hyperparameters, and notorious for inventing visual structure that isn’t real.
- UMAP (Uniform Manifold Approximation and Projection) is the modern default: it preserves local neighborhood structure like t-SNE, runs dramatically faster, and tends to keep more of the global arrangement intact.
What UMAP actually preserves
UMAP promises that points that were neighbors in high-dimensional space stay neighbors in the projection. If two meditations sat close together in embedding space — because they mean similar things — they’ll sit close together on your screen.
Just as important is what UMAP does not promise. The distance between clusters in a UMAP plot is largely an artifact of the projection. Two clusters that appear far apart aren’t necessarily “more different” than two that appear close. Cluster size is similarly unreliable. Reading a UMAP plot means focusing on which points are grouped together and remembering that cluster size and distance can be misleading.
Honest reading habits matter because UMAP plots are seductive. They look like scatter plots, and we’re trained to read scatter-plot axes as meaningful. In a UMAP projection the axes mean nothing at all.
The pipeline
The flow from database to picture is short:
# 1. Export embeddings (id, vector) from Postgres
# 2. Reduce with Python's umap-learn
import umap
reducer = umap.UMAP(n_neighbors=15, min_dist=0.1, metric='cosine')
coords = reducer.fit_transform(vectors) # (n, 1024) -> (n, 2)
# 3. Plot — color by category, label on hover
Two parameters do most of the work. n_neighbors controls how much of the neighborhood UMAP considers — lower values emphasize fine local detail, higher values favor the big picture. min_dist controls how tightly points pack. And since embeddings are compared by cosine similarity, tell UMAP to use the cosine metric — the default Euclidean metric subtly misrepresents the space.
I serve the result as an interactive scatter plot — pan, zoom, hover to read titles — to aid in my own exploration of topics and content. A static image shows you that clusters exist; the interactive one lets you discover what they are.
A first glimpse
This is a corner of the projection of DailyMedToday’s meditation archive — several hundred meditations, embedded and flattened to a plane. Every dot is a meditation. I did not manually assign those groupings. It’s not based on taxonomy, tags, or an editor sorting content into buckets. The structure you’re seeing came entirely from the embeddings — which is to say, from the meaning of the texts themselves.
What exactly those clusters turn out to be — and what it means that a model with no theological training found them — is the subject of a later post in this series. Before we get there, we’ll further develop our stack.
Next up: the case for running all of this on hardware you control.