Ben Pace

I'm an admin of this site; I work full-time on trying to help people on LessWrong refine the art of human rationality.

Longer bio:


AI Alignment Writing Day 2019
Transcript of Eric Weinstein / Peter Thiel Conversation
AI Alignment Writing Day 2018
Share Models, Not Beliefs


Tools for keeping focused

Update after 2 weeks: this has worked out as intended. I now am happy to open Slack to ping individuals, I don't expect to get hijacked by recent channel convo. At the same time, it's no problem to click through all the channels once per day to see what's been going on. We have one time-sensitive channel that I haven't muted, and that's not been abused.

Am way happier with Slack than I've ever been.

Radical Probabilism

Thank you, they were all helpful. I'll write more if I have more questions.

("sadly that's unprobable to work" lol)

Radical Probabilism

Thank you, those points all helped a bunch. 

(I feel most resolved on the calibration one. If I think more about the other two and have more questions, I'll come back and write them.)

Radical Probabilism

I made notes while reading about things that I was confused about or that stood out to me. Here they are:

  • The post says that radical probabilism rejects #3-#5, but also that Jeffrey's updates is derived from having rigidity (#5), which sounds like a contradiction. (I feel most dumb about this bullet, it's probably obvious.)
  • The convergence section blew me away. The dialogue here correctly understood my confusion (why would I only believe either h(1/3) or h(2/3)) and then hit me with the 'embedded world models' point, and that was so persuasive. This felt really powerful, tying together some key arguments in this space.
  • I don’t get why the proof of conservation of expected evidence is relevant. It seems to assume that not only do I know how I will update, but that the bookie does too, which seems like an odd and overpowered assumption, and feels in contrast with all the things you said about rigidity – why does the bookie get to know how I’ll update?
  • "This has some implications for AI alignment, but I won't try to spell them out here." Such temptation! :)
  • I didn’t follow the argument that classical bayesians don’t have calibration. I think it's just saying that classical bayesianism doesn't have any part for self-reference, and that's a big deal? I don't think this means bayesians aren't calibrated, just that they don't have calibration as an explicit part of their model.
  • I do not understand how Jeffrey updates lead to path dependence. Is the trick that my probabilities can change without evidence, therefore I can just update B without observing anything that also updates A, and then use that for hocus pocus? Writing that out, I think that's probably it, but as I was reading the essay I wasn't sure which bit was where the key step was happening.
  • Okay, I got tired and skipped most of the virtual evidence section (it got tough for me). You say "Exchange Virtual Evidence" and I would be interested in a concrete example of what that kind of conversation would look like. I'm imagining it's something like "I thought for ages and changed my mind, let me tell you why".
  • Thanks for the the stuff at the end, about making the meta-bayesian update. I wanted to read you say your thoughts on that, would've been sad if it hadn't been there.
  • The examples of non-bayesian updates I've been making are really valuable. I'll be noticing these more often.
Radical Probabilism

Sh*t. Wow. This is really impressive. 

Speaking for myself, this (combined with your orthodox case against utility functions) feels like the next biggest step for me since Embedded Agency in understanding what's wrong with our models of agency and how to improve them.

If I were to put it into words, I'm getting a strong vibe of "No really, you're starting the game inside the universe, stop assuming you've got all the hypotheses in your head and that you've got clean input-output, you need far fewer assumptions if you're going to get around this space at all." Plus a sense that this isn't 'weird' or 'impossibly confusing', and that actually these things will be able to make good sense.

All the details though are in the things you say about convergence and not knowing your updates and so on, which I don't have anything to add to.

Covid 8/27: The Fall of the CDC

Gotcha, you saw the primary source.

What a sad state of affairs...

Are We Right about How Effective Mockery Is?

I also remember this! (And also cannot recall the link.)

Covid 8/27: The Fall of the CDC

See this tweet from my friend Andrew Rettek, noting that the head of the FDA is far from alone in failing statistics forever. 

I look at these links for about 5 mins, and I want to register that my impression is consistent with the FDA person having correct beliefs about the facts on the ground and just being imprecise when trying to explain it in simple terms to the public. If 100 people were going to die of covid, then the plasma thing would have saved 35 of them.

But I'm also having difficulty finding primary sources, so perhaps they did just say the straightforward absolute-risk thing.

Forecasting Thread: AI Timelines

Let's do it. I'm super duper busy, please ping me if I've not replied here within a week.

Forecasting Thread: AI Timelines

I so want to see a bet come out of this.

Load More