The Future Lives in the Residuals

2026-09-05developingessay

subtitle
Signal, noise, smoothing, and why the first evidence of a new regime usually looks like an error.

In January 2025 I was looking at a volatility surface and wrote something which, in retrospect, is much closer to the Cultural Volatility Index than most of the grander descriptions I gave it:

a volatility surface isn't flat. It's interpolated to be flat, but we can add in some of the noise. And by noise, I really mean signal, because it's not noise. When we do the bivariate spline … we're kind of assuming this signal to be noise and kind of grinding it down. We're sanding or polishing it down. But that's to get a very clear … generalization and understanding of what the volatility surface looks like.

This is the ordinary bargain of modeling. Reality gives you more detail than you can use, so you fit a surface through it, average things, interpolate, compress, remove enough local variation that the larger shape becomes visible. Otherwise the model is just reality wearing an expensive name tag. A map which reproduces every pebble is not a very useful map.

The problem is that the model does not receive a helpful annotation saying which deviations are disposable. It gets a cloud of observations and an instruction to explain them cleanly. The leftover part becomes the residual: the distance between what the model expected and what actually happened. Usually this is exactly where you want noise to go. Measurement error goes there, weird one-off behavior goes there, stale data goes there, somebody fat-fingering a field goes there, etc. A lot of the residual is garbage because reality produces an enormous amount of garbage.

But a regime change also starts in the residual.

I. Before the model changes, reality has to disagree with it

Suppose a model has learned the world reasonably well. As long as the underlying process stays similar, new observations fall near the fitted surface and the residuals remain boring. Then something changes. At first the model does not know that a new regime exists; “new regime” is a conclusion available only after enough contrary evidence accumulates. The first few observations therefore appear in the same category as bad measurements and irrelevant deviations. They are simply places where the old model is wrong.

That makes prediction at boundaries different from prediction inside stable regimes. Inside the regime, smoothing is your friend. Near the boundary, excessive smoothing can erase the only evidence that the regime is ending.

This is not a reason to worship outliers. Most outliers deserve their lonely little lives. It is a reason to preserve them long enough for later evidence to change their classification. Signal and noise are not always intrinsic properties of an observation; sometimes they are verdicts delivered by subsequent observations.

A misspelling nobody repeats is a misspelling. The same misspelling copied by a scene, mutated into three jokes, printed on a shirt and adopted by people who do not know the original post is no longer usefully described as an error. Nothing about the first observation changed. What changed was the evidence around it.

II. A trend dashboard is usually a model of the past with excellent typography

Most cultural measurement systems are optimized for legibility after the fact. Count streams, searches, views, sales, followers or mentions, sort descending, put the largest number at the top. This is useful if the question is what is popular. It is much less useful if the question is what is becoming possible.

The future is badly behaved at small sample sizes. New scenes are tiny by definition. New slang is initially used by almost nobody. A visual motif looks like coincidence until it has descendants. A producer who will influence a genre can spend years being numerically less important than a competent person making the thing everybody already wants. If you filter aggressively for scale, consistency and statistical cleanliness, you are selecting for things which have already survived long enough to become easy to measure.

This is why I keep coming back to waves. The crest is obvious. The interesting information is in the disturbance before anybody agrees that it is a wave.

In August 2023 I described the intuition with a tangent line: treat each person, trend or component of the zeitgeist as a point in a cultural wave system, then estimate its local direction. The metaphor is imperfect in exactly the useful way. A tangent tells you where the function is heading here; extend it indefinitely and it becomes fiction. Cultural systems are worse than smooth functions anyway: they have hidden state, thresholds, discontinuities, imitation, backlash and people who will change what they do because you published a prediction about what they are going to do.

So the point is not prophecy. The point is to get better at noticing when the current model is beginning to accumulate interesting errors.

III. Not every residual deserves promotion

The obvious failure mode is astrology with a data warehouse. If every weird thing is secretly the future, the system can explain anything and therefore predicts nothing. The filtering problem cannot be escaped; it can only be delayed and made inspectable.

I would start promoting a residual from “probably noise” to “keep watching” when several different kinds of corroboration appear. Does it repeat without obvious coordination? Is its rate accelerating relative to its own tiny baseline? Does it survive being copied by people who alter it? Does it jump from music into clothes, language, interface design or some other domain? Does a similar form appear in disconnected communities? Are the people touching it the same people who have repeatedly appeared early around other changes? Does participation grow, or is everybody merely looking?

Those are not proofs of importance. They are reasons not to throw the observation away yet. A good cultural instrument would let a weak signal remain weak while its evidence accumulates instead of forcing it immediately into either “trend” or “noise.”

This is where one of the recovered ideas from the CVI work becomes useful: the “live player” is not necessarily the person with the largest audience. A live player is somebody whose choices change the choices available to nearby people. They can be a mutator, translator, bridge, legitimizer, cluster-builder or first credible adopter. Reach is one variable. Causal position is another.

IV. The model should keep its mistakes

If I were building this seriously, I would distrust any system which only preserves the signals it eventually got right. Retrospective culture is already too good at manufacturing inevitability. Once a movement has a name, everyone can draw a beautiful genealogy from the obvious precursors to the obvious outcome and quietly delete the hundreds of neighboring experiments which looked equally promising at the time.

The dead branches matter because they are the denominator. If the instrument says “we saw this coming” but forgot every similar thing it also thought it saw coming, it has built mythology, not forecasting.

So the raw layer should remain raw: timestamped observations, provenance, transformations, missing data, uncertainty, failed predictions, things whose relationship is only similarity rather than demonstrated lineage. The clean surface can exist above it, but it should be possible to drill back down and see what was sanded away to produce the curve.

This also creates a useful test. Pick some prospective outcome before it happens—sustained growth, cross-domain adoption, geographic diffusion, derivative work, whatever—then ask whether the fancy residual-aware system beats a stupid baseline like present popularity. If it does not, wonderful: now we know we built a beautiful explanation machine instead of a forecasting instrument.

V. Kairos looks like an error term from inside Chronos

This is, annoyingly, another route back to Kairos and Chronos. Once a cultural movement has happened, Chronos gives us a timeline. The important people become precursors, the style gets a name, the influences become legible, and the path through history looks much cleaner than the field of possibilities looked to anybody living inside it.

Kairos is the interval before that cleanup, when the future is still negotiable and the thing which will later become “obviously important” is sharing a statistical bucket with typos, failed experiments, bad copies and people being weird on the internet.

That may be the most concise statement of what I want from cultural forecasting. Not a machine which tells me the future. A machine which is unusually reluctant to destroy the evidence that the present has stopped behaving like its own explanation.