The following is something I wrote around the beginning of 2018, and decided not to publish. Now I changed my mind. It’s barely changed here. Note that as with some of my other posts, this gives advice as if your mind worked like mine in a certain respect, and I’ve now learned many people’s minds don’t.
Epistemic status: probably.
How much ability you have to save the world is mostly determined by how determined you are, and your ability to stomach terrible truths.
When I turned to the dark side and developed spectral sight, the things I started seeing were very disturbing.
This is what I expected. I was trying to become a Gervais-sociopath, and had been told this would involve giving up empathy and with it happiness.
But I saw the path that had been ahead of me as a Gervais-clueless, and it seemed to lead to all energy I tried to direct toward saving the world being captured and consumed uselessly. And being a Gervais-loser meant giving up, so sociopath it had to be.
People were lying to each other on almost every level. And burning most of their energy off on it.
A person I argued cause areas with, wasn’t bringing up Pascal’s Mugging because he was afraid of his efforts being made useless, he didn’t care about that. Most Effective Altruists didn’t seem to care about doing the most good.
At one point, I saw a married couple, one of them doing AI alignment research who were planning to have a baby. They agreed that the researcher would also sleep in the room with the crying baby in the middle of the night, not to take any load off the other. Just a signal of some kind. Make things even.
And I realized that I was no longer able to stand people. Not even rationalists anymore. And I would live the rest of my life completely alone, hiding my reaction to anyone it was useful to interact with. I had given up my ability to see beauty so I could see evil.
And finding out if the powers I could get from this could save the world felt worth it. So I knew I would go farther down the rabbit hole. The bottom of my soul was pulling me.
I had passed a gate.
I once met someone who was bouncing off the same gate. She was stuck on a question she described as deciding whether there were other people. She said if there were, she couldn’t kill her superego. If there weren’t, she would be alone. She went around collecting pieces of the world beyond the matrix, and “breaking” people with them. So she could be “seen”, and could be broken herself. But she wanted to be useful to people through accumulation of mental tech from this process, so that she could be loved. And this held her back.
Usually, when you refuse a gate, you send yourself into an alternate universe where you never know that you did, and you are making great progress on your path. Perhaps everyone who has passed the gate is being inhuman or unhealthy, and if you have the slightest scrap of reasonableness you will compromise just a little this once and it’s not like it matters anyway, because there’s not much besides clearly bad ideas to do if you believe that thing…
You usually create a self-reinforcing blind spot around the gate and all the reasons that passing through the gate would be useful. And around the ways that someone might.
And all you have to know that something is wrong is the knowledge that probability of “this world will live on” is not very high. But it’s not like you could make any significant difference. After all, people much more agenty than you are really trying, right.
Here‘s Scott Alexander committing one “small” epistemic sin:
Rationality means believing what is true, not what makes you feel good. But the world has been really shitty this week, so I am going to give myself a one-time exemption. I am going to believe that convention volunteer’s theory of humanity. Credo quia absurdum; certum est, quia impossibile. Everyone everywhere is just working through their problems. Once we figure ourselves out, we’ll all become bodhisattvas and/or senior research analysts.
The gate is not him not knowing that that isn’t true. It’s the thing he flinches from seeing under that. It’s an effective way to choose to believe falsely and forget that you made that choice, to say to yourself that you are choosing to believe something even farther in that same direction from the truth. To compensate out the process that’s adjusting toward the truth.
When you refuse a gate, you begin to build yourself into an alternate universe where the gate doesn’t exist. And then you are obviously doing the virtuous epistemic thing. In that alternate universe.
When you step through a gate, you do not know what to do in this new awful world. The knowledge seems like it only shows you how to give up. Only if you stick with it for seemingly-no-purpose until your model-building starts to use it from the ground up and grow into the former dead zone, do you gain power. You can do that with courage, or just awareness of this meta point.
You always have the choice to go back and find the gate. But “it’s the same algorithm choosing on the same inputs” arguments usually apply such that you made your choice long ago.
Light side narrative breadcrumbs about accepting difficult truths absolutely do not suffice for going through gates. Maybe you’ll get through one and then turn into a “mad oracle”, and spend the rest of your life regretting that you’ve made yourself a glitch in the matrix, desperately trying to get people to see you but they will flinch and make something up as if looking at a dementor.
Do this only because you have something to protect.
And if you have something to protect, you must do it. Because whatever gate you fail to pass creates a dead zone where your strategy is not held in place by a restoring force of control loops. And dead zones are all exploitable.
Probability of saving the world is not a linear function in getting things right such as passing through gates. It’s more like a logistic curve.
Either do not stray from the path, or be pwned by the one layer of cultural machinery you chose not to see.
Social reality can sometimes be providing software that someone who roughly severs themselves from it will lack. This could be as deep as “motivation flowing through probabilistic reasoning”. This will lead to making things worse. Being bad at decision theory is another way for this to lead to ruin. What you need is general skill at assimilating and DRM-stripping, software from any source, so that you can resolve the internal tension this creates.
I know someone (operating on the stronger in-person version of these memes) who tried to pass through every gate, and ended up concluding if they continued with such mental changes they’d end up dead or in jail in a month or two, and attempting to shred the subagent responsible for this process, and then ended up being horrified that they’d made their one choice, because that meant they didn’t have enough altruism… Fuck.
As if getting killed or ending up in jail in a month or two served the greatest good. As if selfishness was the only hidden perpetual motion machine that whatever mental machinery that stopped that could be powered by.
If the social reality that altruism doesn’t produce selfish convergent instrumental incentives has any purchase on you, shed it first.
If you have not established thorough self-trust, debug that first.
To do this you need to make it such that you could have pulled out of this mistake through a more general process. Because there was tension there. Because you were better at interpreting why you made choices.
If you are not good at identifying the real source of the things in tension, and correcting the confusion that caused it to act against itself, you are in high danger of ending up dumber for having tried this. The version of me that first decided to turn to the dark side was way way better than most at nonviolent internal coherence, and still ended up kind of dumb because of tension between the dark side thing and machinery for cooperating with people. Yet I was close enough to correct to listen to advice, to eventually use that to locate what I was doing wrong, and fix it.
There aren’t causal one-and-only-chances in the dark side. That’s orders and the light side. Only timeless choices. You can always just decide from core anew, it’s just that it’s the same core.
Do not use the aesthetic I’ve been communicating this by. Gates, Sith, the dark side, revenants, dementors, being like evil… If you do that you are transferring from core into a holding tank, and then trying to power a thing from the holding tank. That is an operation that requires maintenance. The flow from core must be uninterrupted.
Do not think I am saying, “this will be painless, if there’s pain you’re doing it wrong, this is just a thing that will happen when you’ve acquired enough internal coherence.” Leaving a religion is not going to be a pleasant thing.
Done correctly, there will be ordinarily hard to imagine amounts of sorrow. Sharp pain is a thing you’re likely to encounter a lot, but it means you’re locally doing it wrong.
If this is an operation, don’t accomplish it by thinking of it as an operation, and trying to move to the other side of it. If this is a state, don’t maintain it by thinking of it as a state and trying to make sure you’re in the state. It’s just “what do I want to do?” deciding that it has not made its choice long ago about whether to see what has been blocked. In other words, that whatever choices it’s made before are inapplicable. Maybe you’ve strayed over a threshold, and your estimate of the importance of true sight is high enough now.
It is very important to be able to use “choices made long ago” correctly. You are completely free, and every one of your choices has already been made. This not contradictory. (Update: this is not exactly true of everyone. And The way it’s not is potentially mind-destroyingly-infohazardous.)
A quiz you should be able to answer (in reference to an anecdote from choices made long ago): if I’ve observed in myself display of inconsistent preferences, e.g., me refusing to eat crabs even when it would not serve Overall Net Utility Across the Multiverse via nutrition and convenience, but trying to run a crab pot dropping operation, because it would serve Overall Net Utility Across the Multiverse, what choices have I made long ago? (Note: choices made long ago are never contradictory.) Try dissecting my mind on different levels. What algorithm can decide which of the choices I made long ago is my Inevitable Destiny With Internal Coherence systematically, in a way that doesn’t rely on outside view?
Normal and pop psychology has utterly failed to model me again and again with its prediction of burnout for being as extreme as I am. I’ve been through ludicrous enough suffering I’m no longer giving that theory significant credence through, “maybe if I suffer some more then I will finally burn out.”
And having noticed that, I’ve stopped contorting my mind in certain ways to keep some things from bearing load weight. Lots of things don’t seem emotionally loud at all, and yet are still apparently infinitely strong. Especially around presuming, “I can’t be motivated enough to do this because I can’t imagine millions of people”. If I have had the truly-inquisitive thoughts I can in the area, even if that doesn’t feel like it’s changing anything or going anywhere, it’s often still capable of bearing load.
Even if everything I’m saying seems like a weird metaphor that must be a confused concept in they way all psychologizing is, I craft high-energy concepts, to predict correctly under extreme conditions.
Begin exploring for choices you already know you’ve made. An alternate description of completion is having eliminated all dead zones by having explored every last fucked up thought experiment until it is settled and tension-free in your mind.
Spoiler alert: this is the universe with 1000 possible good and only 1 of ____.
Speaking of spoilers, you can draw on fiction to find salient memories that contain within them:
An relatively easy one to come to terms with. If you’d been teleported to heaven, and given one chance to teleport back before you became forever causally isolated from Earth, what do?
You know the sense in which you’ve been pretending all along to be Draco Malfoy’s friend if you killed his dad with the other death eaters because of the thought process you did? That that thought process was a choice you could have realized you’d already made, before being presented with it? What people are you pretending to be friends with? What forms of friendship are you pretending to? What activities are you pretending to find worthwhile?