Coding day in and out on LessWrong 2.0
Promoted to curated: This post is answering (of course not fully, but in parts) what seems to me one of the most important open questions in theoretical rationality, and I think does so in a really thorough and engaging way. It also draws connections to a substantial number of other parts of your and Scott's work in a way that has helped me understand those much more thoroughly.
I am really excited about this post. I kind of wish I could curate it two or three times because I do really want a lot of people to have read this, and expect that it will change how I think about a substantial number of topics.
Hey! Send me an email at email@example.com and I can probably put you in touch :)
Promoted to curated: I have only skimmed Atlas Shrugged (which is something I probably want to fix at some point), but even without having read it I still got a lot of value out of this post. In general I have a hypothesis that something in the space of "understanding and iterating on what broader attitudes I am using to interface with the world" is one of the key bottlenecks for many people in both their development for epistemic and instrumental rationality, and this post felt like it helped me make some progress on that. I also liked the commentary and advice on how to read and learn from both non-fiction and fiction books.
I added the above notice about licensing requirements, which I think should make this fine.
This sounds fun! I probably won't have enough time to participate, but I do wish I had enough time.
Yeah, I do think it's likely that AGI would be monitored during training, but the specific instance of Open AI staff being asleep while we train the AI is a clear instance of us not monitoring the AI during the most crucial periods (which, to be clear, I think is fine since I think the risks were indeed quite low, and I don't see this as providing super much evidence about Open AI's future practices)
At least with current technologies, I expect serious risks to start occuring during training, not deployment. That's ultimately when you will the greatest learning happening, when you have the greatest access to compute, and when you will first cross the threshold of intelligence that will make the system actually dangerous. So I don't think that just checking things after they are trained is safe.
The LessWrong team maintains a WPEngine account that we've taken over from Trike Apps that already has a bunch of sites like SSC, 80k, and OB on it, and I think it would totally make sense for us to also host your site on it. This would come with all the benefits of self-hosting, plus some more benefits that come with WPEngine. My guess is that the migration should be totally seamless for you. I know for a fact that this also enables you to activate the old editor again, since some of the sites we help maintain have that activated.
I think the marginal financial cost to us is 0, since I think (80%) we have a plan with just a fixed number of sites we get to build, and we are pretty far below that number (11/30).
Now, many years later, for future observers, it seems important to note that the linked study probably didn't survive the replication crisis, which makes this paragraph as it stands sadly one of the worst in the essay. I am not confident that this particular study didn't replicate, since that would require substantially more time investment, but the vast majority of the studies around priming and associative reasoning were really badly flawed, and I expect the above to be no different, and I would advise great caution before taking any studies in its reference class to be worth any more than random anecdotes (and to be substantially less valuable than your own personal anecdotes or unbiased samples of anecdotes from your friends that you've bothered to vet and follow up on).
Random piece of feedback that made me hesitant: When I went to Naval Gazing the site took 16 seconds to respond, and this happened consistently on reload.