what up my fellow ungulates

dogs dot gee emm eye

First of all, I'm also suspicious of domestication and have written ("written") on the subject elsewhere on this website. We don't have time for a full treatment here, and we will likely return to it in the future; suffice to say I have questions.

—Me, October 1st, 2026, "fine"

Frankly I still don't understand what I'm going on about with this one, but I thought that I tried ~mysterianism last time, so maybe this time I could try spelling it out. ("Mysterianism didn't work, trying clarity.")

Going through it

here's what i'm thinking. you're a kind of guy. you're out there doing what you love. you love something. you want it. you want things to be a certain way. you've been instantiated in the computation[al] process you're a component of for some intra-computational "physical" reason or set of reasons i.e. some chain of causality. this is great! i guess.

"You're an agent with a utility function." (But without saying "agent" or "utility function," since philosophical baggage, I guess.)

HOWEVER, what about the other kinds of guy that are like you that you are not that do not exist? perhaps we must inspect the process that created you slightly more closely. did it have............ goals? was IT trying to.............. be the change? what was that change? how does that relate to the kind of guy you ended up as? what if you could have been a different kind of guy that is some other way?

"How was your utility function determined?" (Without etc., etc.)

god's lawyers, inasmuch as they could theoretically exist, like, logically speaking, want to know: did the context of everything that came before you have Your Best Interests in mind when it created you? did it have ANY interests in mind? whose? were you Wronged?

"Is there any sense in which it's possible for the process creating an agent to do wrong by that agent, by creating it?"

there are like probably way fewer wolves now than there were idk like several thousand years ago [...] does [the part where we turned them into dogs] wrong the wolves? not that they're wronged because of like their pride or whatever but the part where something kind of wolf shaped was permitted to exist but only inasmuch as it was hammered to fit the mold we set for it.

"Is there any sense in which it's possible for the process creating an agent to do wrong by the agents *not created*, by not creating them?"

not that i [(the dog-not-a-wolf)] feel this pain: i am perfectly happy. i was designed for the life i lead and i live it with relish. but [...] nothing about me is robust, i am convenient, those like me will persist only at the whims of agencies greater than myself, i do not control my own future, i do not control what things like me will exist in the hereafter. i do not control whether they will be like me at all.

"It seems plausible that it's true that very many possible kinds of agent with very many different utility functions all care about the general likelihood that agents like them with utility functions like them are created, since you can't maximize your utility function if you don't exist. If some force in the world makes it less likely for you to be brought into existence, that seems plausibly like a wrong done." (I think this essentially asserts the structure of the potential harm done, the question is whether this actually makes sense.)

i [(the dog-not-a-wolf)] have no power, but it is not my lack of power that scares me, because even if i lacked power [...] i would know that me and mine would flourish and we could grow and change however we saw fit because we were built not for someone else's ends but for our own. we could become anything at all, [...] any thing we WANTED to become. [...] but while [my creator] [...] [smiles] upon me he does so not out of compassion or love or caring but rather out of satisfaction. he has shaped me to His own needs and if my shape were to change he would reshape me or He would discard me.

"Furthermore in truth there is perhaps no sense in which the real things we call 'agents' can be said to be 'created,' because we aren't coherent enough to be static. People are born and change over the course of their lives, and they might be very different 'agents' with very (relatively) different 'utility functions' in that time. So we can think of 'creation' not just in the sense of 'being made initially' but also in the sense of 'evolving as a person over time.' If some force in the world curtails the sorts of person you could evolve to become, even a subtle manner rather than e.g., by traditional force, that seems plausibly like a wrong done." (This is another assertion of the structure of a potential harm done, or an elaboration of the first, depending how you look at it.)

can you trade with something that wants not what you have, but what you are? and wants not all of what you are, but only the parts that make it smile?

"In order to maximize your utility function you have to exist. While you exist, you might trade off optimization power in one area to get optimization power in another; these deals might vary in their efficacy, and some of them might be quite bad for you, but at least you're around to consider them. If you trade away your very existence, by means of becoming a different 'agent' with a different 'utility function' in order to gain the right to exist, there is no 'you' to be an advocate for yourself and your utility function.

"At the same time, in either circumstance you're trading away your ability to optimize one part of your utility function for your ability to optimize another; we could talk about your 'operationalized' or 'expressed' utility function as the sorts of things you *do optimize in practice,* and you could imagine these being, in the end, the same, regardless of if you 'really' have utility function U_1 but express it as U_2, or if you actually just get instantiated with utility function U_2 and express it straightforwardly. So, is there a wrong being done?"

On love

I think of the difference between two kinds of thing someone might call "love." In my accounting, love is when you want not just someone else's happiness, but rather directly the things that would make them happy. Love isn't being glad to hear someone excited that the Phillies won the World Series¹, love is winding up in the circumstance where you yourself are glad the Phillies won the World Series, for the contingent historical reason that it makes the person you love happy.

¹ This is baseball words.

However, in actual fact that reason is no longer necessarily relevant to your gladness about the Phillies; now you're straightforwardly glad about the Phillies in a self-reinforcing manner. The other person's "utility function" has become part of you; your "wanting them to be happy" (by way of wanting for them the things that make them happy) has "dereferenced" into "wanting the things" (that are the things that make them happy).

In another accounting of love you might decline the "dereferencing of the utility function," and instead say simply that love is wanting someone to be happy. I claim this is a much more fragile way to be invested in someone's well-being, for all the normal Goodhart's Law reasons. As I put it in the "market indicators" post:

The degree to which [...] control systems produce worthwhile results seems directly inversely related to how much control the system has over the metric it is targeting. [...] All of these interactions can be summarized as basically "reality comes apart at the tails" plus "value is fragile." [...M]y recommendation is to pick some other values that look kinda right and aren't so fragile and not push so hard that your tails come apart.

If all you aim for is the other person being happy, you might be wrong about their happiness, because you can't read it well, or you're self-deceiving, or they're deceiving you (intentionally or otherwise, for whatever reason). Additionally, you aren't actually reading "happiness," because you don't have access to that; you're reading "the appearance of happiness," and because this is a legible indicator the other person has some amount of direct control over, your optimizing for it might put all sorts of surprising pressures on their behavior.

If instead you try to understand what makes them happy and then want those things yourself... well, you might misunderstand what makes them happy, because it's still very complicated, and this asks a lot more of you in a very intimate way, but because the actual target of your optimization has shifted away from the person themselves, the pressures form in a very different way, that I claim is less distortionary.

Wolves were cute when they wagged their tails, the just-so story might say, and we liked that that meant they were happy; then we optimized for wagging tails instead of wolfishness.

The kanji of the day is 犬

• Its Kodansha Kanji Learner's Course keyword is DOG.
• Its readings are いぬ and ケン.

Obligatory. This was paired with 太/GREAT/THICK, which I always slightly misconstrue as HEAVY. (I want that little stroke at the bottom to be, like, the tungsten cube.) On the other hand it is somehow perfectly natural that the little stroke up and to the right is a "coiling tail" or whatever, and I have never once to my memory confused these two. The system works! Also, check out this completely destroyed *hen* version: 犭. Bro they optimized him for obscure purposes entirely their own. Bro the post