Market indicators

...are also anti-inductive?

Why are all of our market indicators so difficult to read?

Earnest (if rhetorical) question, not complaint. A while ago I asked Claude about Treasury yields, since it was in the news at time time (...still is...), and his response boiled down to:

There's a bunch of different reasons Treasury yields might go up, or things Treasury yields going up might mean. Some of them include (1) supply and demand, (2) inflation expectations, (3) economic growth expectations, (4) Federal Reserve policy expectations, and (5) fiscal sustainability expectations.

This is a lot of different mechanisms bundled up under one indicator! Or rather, this is many different predictions bundled together into one number.

Qua prediction markets

I have a slightly weird perspective on financial markets: I understood prediction markets mechanistically before I got around to properly understanding, e.g., the stock market. So even though this kind of market indicator is objectively more normal than the clearer-cut(? foreshadowing...) indicators we get from prediction markets, it still seems kind of odd to me.

And it's not just the Treasury yield; seemingly every economic indicator is like this. What I learned from reading Money Stuff is that every market, from oil futures to inflation percentages to stock buybacks, has twenty different ways to read it to get twenty different understandings of what is going to happen in the future economically. And so it is said, never reason from a price change, I guess.

When you're young and chipper and just finished trading on a market where shares pay out $1 if a YouTuber uploads a video in the next two months, this seems like much more trouble than it needs to be. If we want to know if inflation is going to go up, make a prediction market tracking inflation! If we want to know if the US is going to default on its debt, make a prediction market tracking US defaults! And of course these exist now.

And equally obviously--though maybe not to the youthful me (...of two or three years ago...)--these markets do not precisely track the things they purport to track. Between frequently low liquidity, the cost of money, and the expense of keeping extreme probabilities extreme (but I repeat myself), simple market dynamics alone often suffice to explain why a given prediction market might be off what you'd expect the "true" probability to be, even before considering whether the traders involved have any real alpha. But it gets worse: even when the resolution criteria are spelled out as straightforwardly as possible, are they still straightforward?

You don't get what you trade for

It's well-recognized in the prediction market community at this point that good resolution criteria, especially about things people care about, are very hard to design. Scroll through Kalshi and Polymarket and you see questions targeted at very specific observables. You can find "How high will unemployment be in year 20XX?" but nowhere will you see "Will the economy be good in 20XX?" This is only natural; no one agrees on what a "good economy" looks like even when it's supposedly happening or not happening, so how could we write down a set of criteria for what a good economy might be in the future that anybody would want to bet on, other than by shelling out to extremely legible (but maybe not good-economy-tracking) statistics?

The more complicated the domain, the worse this gets. For example, it would obviously be useful to know whether the US-Iran war will end any time soon. However this question--"when will the war end?"--doesn't come close to uniquely specifying some set of worldstates the inquirer is considering. And so of course you don't see any question like this on Polymarket. Instead you have options like:

This last is especially interesting because this is yet another level of indirection. The question is not, "will the US's blockade of Iran end by [date]?" but rather will the end of the blockade *be announced.* You don't have to have any particular political opinions to agree that the US government announcing something will happen currently has only a loose relationship with that thing actually happening; and even this very guarded attempt at tracking the thing we actually want to know still has its flaws. Looking at the resolution criteria, there are of course yet more caveats:

This market will resolve to “Yes” if the United States government, or an authorized representative of the United States government, publicly and officially announces the end, termination, lifting, or suspension of the United States’ naval blockade on Iranian ships and ships of Iranian customers...

An announcement qualifies if it communicates that the United States will generally cease blocking vessel traffic for Iranian ships and customers, including an announcement that the blockade will not take effect at all, even if some restrictions remain...

A qualifying announcement must clearly and unambiguously identify the end or suspension of the blockade. Statements that merely allude to, reference, or describe an end to the blockade, without clearly communicating it, do not qualify. The announcement need not use specific terminology or reference the end of a blockade by name...

A qualifying announcement must be made through official channels, by an individual acting in an official capacity.

People say all kinds of things! How authorized is authorized? How blocked is blocked? How unambiguously is unambiguously? How official is official? This market sounds like a nightmare to resolve. It currently has $40m in trading volume. What we want to know is whether the blockade on Iran will be announced to end. If I were putting my money up on this market, I would not be doing so under the impression that the resolution of my shares would reliably coincide with whether or not I found myself in a world where I considered the end of the blockade to have or have not been announced.

Laws of epistemic physics

Of course this is approximately the least isolated problem of all time.

Everybody knows about Goodhart's Law. At parties everyone stands around and says, when confronted about their clever plan to build a measure and then use it as a target, "oh, yeah, Goodhart's Law, we'll have to think about that." Sometimes I think we all collectively agree to quietly understand Goodhart's Law as less of a Law and more of a kind of thing that happens sometimes due to a regrettable lapse in polite behavior. Whence else prediction markets?

If you build a system that identifies a measure and then immediately begins to use it as a target, optimization pressures will come to bear. If your market has three traders on Manifold they will be weak and you may be safe. If your market has $40m in volume and the people trading it know how to use cryptocurrency they presumably also know how to go on Twitter, and you may not be safe. If you build a system that identifies a measure and then immediately begins to use it as a target but do so while chanting "Goodhart's Law, Goodhart's Law, Goodhart's Law" to yourself, the exact same things will happen.

My pitch is this: Goodhart's Law is a law of epistemic physics. You cannot engineer around it in the same way you cannot engineer around the Second Law of Thermodynamics. That doesn't mean refrigerators or markets that don't happen to get gamed are impossible, it means that "ceasing to be a good measure" is a physical process that takes time and energy to come about. The solution, goes my pitch, is to stop building systems that identify measures and then *immediately* start using them as targets.

This sounds difficult. Everyone does this because it's great; what would we even do instead? Unfortunately my best answer is "remember the traditional market indicators?" Treasury yields aren't a capital-T Target, per se, though this wasn't always true; a quick search shows that at minimum the Fed pegged yields from 1942 to 1951, and apparently other countries' central banks have done the same. Nevertheless the situation in the US seems to be closer to everybody sort of low key keeping tabs on yields but not trying too hard to do anything about it specifically.

Given they're a measure that serves as input to a process that can affect the measure, though, it's hard to say that they're structurally anything other than a target, capital T or no. However, it seems important that the system doesn't *immediately* start using the measure as a target, and doubly that it doesn't do so in a legible way. Therefore the earlier confusion: Treasury yields being interperable as meaning 8 million different things is the same thing as no one being able to argue that they're used legibly to mean anything in particular.

In short, your options are a bad measure (someone used it legibly as a target and it imploded) or a bad measure (it's causally complex so no one can agree on what it means, and even if it gets used as a target the way in which it's a target won't be legible). Systems built using bad measure #1 implode, while systems built using bad measure #2 can sometimes semi-accidentally optimize the thing they're "not" targeting without anyone noticing, so they stick around; therefore most measures we know about look like bad measure #2.

Coda: variously legible targets

Some metrics that are famously used as targets:

The degree to which these control systems produce worthwhile results seems directly inversely related to how much control the system has over the metric it is targeting. My little baby understanding of the Fed is that Jerome Powell (until like six months ago) would turn a big dial that said INTEREST RATES on it while looking back at the economy to see if it only good happen. This is apparently very useful, but I hope I can assert without proof that it has only very indirect influence over whether more people are employed. Likewise, while corporate strategic decisions are important and CEOs make many of them, many more things come to pass in the world and sometimes your stock price explodes (goodly or badly) because a Redditor decided it should.

On the other(?) hand, unfortunately for all of us it seems that social media companies have quite a bit of direct control over how engaged we are with their products, and can pretty directly optimize to get more of it. In retrospect the consequences of letting this process play out on the world for twenty-odd years seem predictable, and yet. Similarly telling schools that the amount of money they get is determined by how well their students perform on a standardized test did not, in my personal experience, lead to a robustly improved system of education.

All of these interactions can be summarized as basically "reality comes apart at the tails" plus "value is fragile." In lieu of solving alignment and designing the perfect metric that perfectly reflects your actual values (which would call into question the law-of-epistemic-physics status of Goodhart's Law), my recommendation is to pick some other values that look kinda right and aren't so fragile and not push so hard that your tails come apart.

The kanji of the day is 勿

I had forgotten DO NOT, actually, so I guess this counts as cheating on my flashcards(?). Conning notes that this kanji is relatively rare and basically just shows up in 勿論, "of course"/"certainly", from DO NOT-ARGUE. (Cheating again; I won't learn 論 for several months.) I am in it, as usual, for the mnemonic: this kanji appears frequently as a radical, and Conning terms it the "wooly mammoth" radical. Wiktionary claims that the original pictogram is of blood on a knife, which is less cute but plausibly makes more visual sense. I just get excited when the wooly mammoth shows up.

Discuss this post on Bluesky