A text isn't about one thing. It's a soft blend of a few hidden themes, each a way of leaning on the vocabulary — and the blend is inferred, never given. Place a document in the space of mixtures, watch words pour out of it, then watch the model read the words backwards to recover where it came from.
LDA is a recipe for writing documents, run in reverse. The recipe has exactly three moves. Everything interactive below is just these three moves made visible.
A topic is a probability distribution over the whole vocabulary — where it puts its mass is what the topic is.
φₖ ~ Dirichlet(β)Not one bin. A mixture θ — 60% this theme, 30% that, 10% the other — a single point on a triangle of blends.
θₒ ~ Dirichlet(α)For each slot, draw a theme from the mixture, then draw a word from that theme. Repeat. Order is never used — a bag of words.
z ~ θₒ · w ~ φ٠Here are the latent themes for this sandbox — borrowed from the essay's own world. A topic is nothing but the heights of these bars: the probability it assigns each word. The model never sees the labels “meditation” or “cooking”; those are our names for distributions it induced. Notice the leakage — no topic owns a word outright.
The triangle is the space of all three-way mixtures: each corner a pure topic, the inside every possible blend. One picture holds the whole model — the prior that says which blends are plausible, your document as a point, and the estimate the model crawls toward as it reads the words your document emits.
The glow on the triangle is the prior — before seeing any words, which
mixtures does the model expect? That's the only thing α controls, and it's the whole reason it's
a Dirichlet allocation. Slide α down toward 0.1 and the mass flees to the
corners: documents are expected to be about one thing, sparse, specialised — which is
why real LDA usually runs α<1. Slide it up past 1 and the mass pools in the
centre: every document an even wash of all three themes. At α=1 exactly, every mixture is equally
likely — a flat triangle.
A second prior, β, does the same job one level down — peaky topics (each owning a few words) versus flat ones. Both priors are knobs on the carving: change them and the same corpus decomposes differently. Hold that thought.
Eigenhector's line was: see entire belief structures as induced grammars over something more continuous, perhaps like latent Dirichlet allocation over word space. Now the analogy has teeth. Read the model with one substitution.
| In LDA | In the essay's move |
|---|---|
| words | your beliefs, judgments, the discrete this-and-thats you actually notice yourself thinking |
| topics φ | a few latent dispositions — the organising leanings that generate those beliefs |
| mixture θ | any given mental state as a soft blend of dispositions, a point in the triangle, never a single bin |
| the prior α, β | how reified your carving is — corner-sharp dualities, or loose continuous washes |
| inference | the carving is fitted, not found — and therefore re-fittable |
The broccoli trick de-reified one axis into a probability. LDA does it to the whole scaffold at once — and because the decomposition was inferred, you can re-induce it: change the number of themes, the priors, the carving, and re-emerge with a reorganised view rather than the one your environment happened to install.
That's the freedom in “go up the ladder of abstraction and induce new grammars over thought space.” Each level is a soft model of the level below; at every level the carving is one fit among many, so dissolving it doesn't lose you — it just frees the next re-fit.