All of Chris_Leong's Comments + Replies

Absolutely fascinating! Thanks for posting! This gave me a new lens onto Nietzsche.

You apparently completely misunderstood the point we were making with the white box thing.

I think you need to taboo the term white box and come up with a new term that will result in less confusion/less people talking past each other.

Some thoughts since you ask for feedback elsewhere:

1) Part of the reason why this post is likely being downvoted is the clickbait title. This is not looked upon favorable on Less Wrong.
2) You make some pretty good points in this post, but you state it far too confidently, almost like a mathematical proof. If you want your posts to do well on Less Wrong, try not to make strong claims without correspondingly strong evidence.

Gabriel makes a very good point: there is something of a tension between allowing reign of terror moderation and considering it a norm violation to request the deletion of comments for low quality.

(TBH, I was convinced that reign of terror would be a disaster, but it seems to be working out okay so far).

What do you think the 101 space should look like for Less Wrong?

E/acc seems to be really fired up about this:

https://twitter.com/ctjlewis/status/1725745699046948996

I read their other comments and I'm skeptical. The tone is wrong.

AI alignment certification and peacebuilding seem like two very different and distinct projects. I'd strongly suggest picking one.

I also notice that I am just afraid of what would happen if I were to e.g. write a post that's just like "an overview over the EA-ish/X-risk-ish policy landscape" that names specific people and explains various historical plans. Like I expect it would make me a lot of enemies.

This seems like a bad idea.

Transparency is important, but ideally, we would find ways to increase this without blowing up a bunch of trust within the community. I guess I'd question whether this is really the bottleneck in terms of transparency/public trust.

I'm worried that as a response to FTX we might end up turning this into a much more adversarial space.

For the record, I updated on ChatGPT. I think that the classic example of imagining telling an AI to get a coffee and it pushes a kid out of the way isn't so much of a concern any more. So the remaining concerns seem to be inner alignment + outer alignment far outside normal human experience + value lock-in.

Thanks for highlighting the issue with the discourse here. People use the word evidence in two different ways which often results in people talking past one another.

I'm using your broader definition, where I imagine that Stella is excluding things that don't meet a more stringent standard.

And my claim is that reasoning under uncertainty sometimes means making decisions based on evidence^[1] that is weaker than we'd like.

1. ^{^}

   Broad definition

Oh, I don't think it actually would end up being temporary, because I expect with high probability that the empirical results of more robust evaluations would confirm that open-source AI is indeed dangerous. I meant temporary in the sense that the initial restrictions might either a) have a time-limit b) be subjective to re-evaluation at a specified point.

Define evidence.

I'm not asking this just to be pedantic, but because I think it'll make the answer to your objection clearer.

1. Thanks for sharing your concerns and helping us be more calibrated on the value of this study.
2. I agree that a control group is vital for good science. Nonetheless, I think that such an experiment is valuable and informative, even if it doesn't meet the high standards required by many professional science disciplines.
3. I believe in the necessity of acting under uncertainty. Even with its flaws, this study is sufficient evidence for us to want to enact temporary regulation at the same time as we work to provide more robust evaluations.
4. The biggest critique for m

Any chance that you could make this more concrete by specifying such a proposal? I expect it'd be possible to make up an example if you want to avoid criticising any specific project.

This is a good point. I neglected to address this possibility.

“If you then say we can split A into A1 and A2, you have added information to the problem. Like the Monty Hall problem, information can change the odds in unexpected ways!” - It’s not clear which is the baseline.

I haven't listened to the whole interview, but it sounds like you might be reading more into it than is there.

Shane talked about the importance of checking the reasoning process given that reinforcement learning can lead to phenomena like deceptive alignment, but he didn't explain exactly how he hopes to deal with this other than saying that the reasoning process has to be checked very carefully.

This could potentially tie to some proposals such as approval-based agents, interpretability or externalized reasoning, but it wasn't clear to me how exactly he wa... (read more)

Thanks for the examples. The third example was good, the second was okay and the first and fourth didn't seem very good. Interested to see how this develops.

BTW, I was curious to see a concrete example where we applied the example to two different contexts.

Our inner game policy efforts are already bearing fruit. We can't precisely define exactly what will happen, but we certainly can push for nuance via this route than we would be able to through the public outreach route.

I can see why you would be a lot more positive on advocacy if you thought that crippling advances is a way out of our current crisis. Unfortunately, I fear that will just result in AI being built by whichever country/actor cares the least about safety. So I think we need more nuance than this.

I agree that there is a trade-off here, however:

a) Dumbing down the message will cost us support from ML engineers and researchers.
b) If the message is dumbed down too much, then the public is unlikely to create pressure towards the kinds of actions that will actually help as opposed to pressuring politicians to engage in shallow, signaling-driven responses.

I strongly agree with Steven Byrnes suggestion that we consider things on a case-by-case basis. I think it's both clear that some people in the alignment community have already taken actions that have significantly shortened timelines and also that it's possible to go too far. I suspect that the earlier someone is in their career, the less they need to worry about accidentally causing capability externalities lest they end up paralysed, but the further people go, the more careful they have to be.

She thought it was spelling, not naming

Sorry, I'm still confused. She was pointing to objects and tapping to receive a name, so presumably she already knew that these words referred to objects.

I think this is a fascinating idea, although I have to be honest that I don’t find the examples you’ve provided very compelling. In order to be persuaded of the usefulness of these techniques, I’d want to see more concrete examples, as when the examples are abstract it is very hard (and subjective) to evaluate how well it is doing at decoding a latent representation in a new context.

In case anyone finds it helpful, the short version of this post seems to be:

1. Train a model to encode and decode text to and from a latent space
2. Train a model to predict the

I would really like him to do that. However I suspect that he feels that these arguments actually aren’t that strong when you look into it, so that might not make sense from his perspective.

I’m still confused by the Helen Keller example. It sounds like she already knew that she could ask for the names of objects, so I’m struggling to see what the realisation was that led her to excitedly ask about the names of a bunch of objects.

One thing that me more comfortable with making statements that are less nuanced in some circumstances is Wittgenstein's idea of language games. Rationalists have a tendency of taking words literally, whilst Wittgenstein views statements as moves in a language games where there are a host of different language games for different situations and people can generally figure it out. Specifically, there seems to be some distinct language games associated with protests where people understand that your sign or slogan doesn't cover everything in complete nuance. At the same time, I think we should be trying to raise the bar in terms of the epistemics/openness of our advocacy work and I do see risks in people taking this reasoning too far.

I suspect that MIRI was prioritising alignment research over the communication of that research when they were optimistic about their alignment directions panning out. It feels like that was a reasonable bet to make, even though I do wish they'd communicated their perspective earlier (happily, they've been publishing a lot more recently).

The POC || GTFO article was very interesting.

I do worry though that it is mixing together pragmatics and epistemics (even though it does try to distinguish the two). Like there's a distinction between when it's reasonable to believe something and when it's reasonable to act upon something.

For example, when I was working as a web developer, there's lots of potential bugs where it would have made sense to believe that there was a decent chance we were vulnerable, but pragmatically we couldn't spare the time to fix every potential security issue. It doesn't m... (read more)

Regarding security mindset, I think that where it really kicks in is when you have a system utilising its intelligence to work around any limitations such that you're no longer looking at a "broad, reasonable" distribution of space, but now a "very, specific" scenario that a powerful optimiser has pushed you towards. In that case, doing things like doubling the size may make your safety schemes if the AI now has the intelligence to get around it.

How so: "There’s a lot more we hope to say about our new (and still evolving) strategy, and about our general thinking on the world’s (generally very dire) situation. But I don’t want those announcements to further delay sharing the above updates, so I’ve factored our 2023 strategy updates into multiple posts, beginning with this one."

Interesting idea. Looking forward to seeing how this goes!

After skimming, I'm still confused. How do you actually want us to use these? GPT4 is undoubtedly useful, including for people trying to save the world, but it's not clear what we should want to do with it. So to for goal agnostic systems.

I don’t know, but would love to find out.

I claim that you also need a could_C "Asher counterfactually could drive off the cliff" unless you want to be eliminativist about counterfactuals.

I've written about this here. Eliezer seems to have found the same solution that I did for the student and exam problem: there's a distinction between a) being fixed independently of your action b) being fixed given your action.

I wrote in one of my footnotes:

You may object that RLHF is mostly capabilities. I also tend to think about it as being primarily a capabilities advance, but it is an advance in alignment as well

ie. it belongs in the reference class when figuring out the difficulty of making progress on alignment.

Regarding scalability, I wrote:

I suspect that all of these approaches are still very far away from where we need to be. I consider them substantial advances nonetheless for two key reasons: having a baseline helps people choose an appropriate level of ambition, and

I'm working on an adversarial collaboration with Abram Demski atm, but after that's finished, I'd be open to having a dialog here on decision theory.

I'm skeptical because in addition to logical reasoning, intuitive reasoning seems pretty important. And I'm not sure if there's a simpler representation of intuitive reasoning that a bunch of weights from a bunch of concepts to another concept.

I'm leaving that open to interpretation. Feel free to add answers relevant to your interpretation.