Modeling Transformative AI Risk (MTAIR)

Wiki Contributions


So the code that wires a 100-trillion-synapse human brain is about 7.5 megabytes. Now an adult human contains a lot more information than this.


Minor quibble which seems to have implications - "There is a consensus that there are roughly about 100 billion neurons total in the human brain. Each of these neurons can have up to 15,000 connections with other neurons via synapses"

My rough understanding is that babies' brains greatly increase how many synapses there are until age 2 or 3, then these are eliminated or become silent in older children and adults. But this implies that there's a ton of connections, and most of the conditioning and construction of the structure is environmental, not build into the structure via genetics.

There's nothing fundamental about polyamory that precludes guarantees of availability. You can certainly have 1 (or more) primary partners who you agree to be more closely bound to. And in fact, many poly people have exactly this, a primary "marriage" along with polyamory.

Interesting post, I wish I had seen it initially. 

It makes a point closely related to something I wrote on Ribbonfarm about corporate structure (and polyamory) a few years earlier:

As an aside, a question that initially bothered me about polyamory was: why isn’t polyamory more widespread, especially among people who aren’t religious or traditional? Yes, there are some scale limits. At the very least, there is a tradeoff between the frequency you can see someone and the number of people involved, but I’m sure there are people who would be happy to juggle 5 or 10 partners. Why isn’t it more common? Why don’t adults keep pivoting, and why is polygamy now relatively rare? Traditional marriage was a good tradeoff for social designers who wanted legible structures, but it’s less obvious why it’s useful for the people. Given that, it’s confusing why so many people nowadays think there is a single “correct” family structure.

I’ll leave that as a question for now, because it should answer itself later, once we figure out why companies don’t stay agile as they scale. The parallel to companies, though, is clear; what social structures work, for what purposes, and why?
<snip half of the (very long) post>
(This also finally answers the questions about polyamory; typical structures are comfortable, and the simplest structure that allows for a relationship is a dyad.) 

I assumed, evidently incorrectly, that the point was to prompt government planners and policymakers with clear ideas now, and say that they will be relevant once X happens - and I don't think that there is an X such that they will be convinced, short of actual catastrophe. 

It now sounds like you looking to do conditional planning for future governance interventions. I'm not sure if that makes sense - it seems pretty clear that groundwork and planning on governance splits between near-term / fast takeoff, and later / slow takeoff, and we've been getting clear indications that we're nearer to the former than the latter - but we aren't going to develop the interventions materially differently based on specific metrics, since the worlds where almost any of the interventions are effective are not going to be sensitive to that level of detail.

Yeah, that seems correct, especially when you look at how likely similar answers for "Are people real?" are (It does much better, with a ~20% chance of starting with "Yes" - but there's a lot of weight on stupid nuance and hedging.)

Interestingly, however, "bananas," "mammals," and "cows" are unamibiguously real.

Thinking about this a bit, (not a huge amount,) I think the specific example "are bugs real" ends up looking interesting in part because the word "bugs" in the prompt has incredibly low likelihood. (As does the following word, "real") 

So the model is conditioning on very low likelihood inputs, which seems like part of the reason for the behavior.

I think humans are complex, and don't have coherent desires. At the same time, most things people want they enjoy. More enjoyment and liking things is from more wealth.

The fact that we've invented weird corner cases by optimizing too hard on engagement (facebook), or on taste (empty calories), doesn't change the fact that there are lots of things we like and benefit from.

There is a lot of OSS that I use but wouldn't pay for if it costed money... if it wasn't free I would just use underscore instead... you get value from something but wouldn't pay for it because there happen to be similarly valuable products available for free.


Imagine if the other OSS package(s) you'd use instead also didn't exist. How much would you pay for one of them to exist or be available to you? Maybe not very much, but whatever amount that is, is a benefit you have from the item(s) existing, and that benefit is part of "wealth" the way he is using the term.

Load More