Daniel Kokotajlo

Comments

Forecasting Thread: AI Timelines

This is my all things considered forecast, though I'm open to the idea that I should weight other people's opinions more than I do. It's not that different from my inside view forecast, i.e. I haven't modified it much in light of the opinions of others. I haven't tried to graph my inside view, it would probably look similar to this only with a higher peak and a bit less probability mass in the "never" category and in the "more than 20 years" region.

How good is humanity at coordination?

To be clear, after I made it, I thought more about it and I'm not sure it's correct. I think I'd have to actually do the math, my intuitions aren't coming in loud and clear here. The reason I'm unsure is that even if for some reason post-apocalyptic worlds rarely get simulated (and thus it's very unsurprising that we find ourself in a world that didn't suffer an apocalypse, because we're probably in a simulation) it may be that we ought to ignore this, since we are trying to act as if we are not simulated anyway, since that's how we have the most influence or something.

Forecasting Thread: AI Timelines

Eyeballing the graph in light of the fact that the 50th percentile is 2034.8, it looks like P(AGI | no AGI before 2040) is about 30%. Maybe that's too low, but it feels about right to me, unfortunately. 20 years from now, science in general (or at least in AI research) may have stagnated, with Moore's Law etc. ended and a plateau in new AI researchers. Or maybe a world war or other disaster will have derailed everything. Etc. Meanwhile 20 years is plenty of time for powerful new technologies to appear that accelerate AI research.

Forecasting Thread: AI Timelines

The main issue for me is that if I win this bet I either won't be around to collect on it, or I'll be around but have much less need for money. So for me the bet you propose is basically "61% chance I pay SDM $22 in 10 years, 39% chance I get nothing."

Jonas Vollmer helped sponsor my other bet on this matter, to get around this problem. He agreed to give me a loan for my possible winnings up front, which I would pay back (with interest) in 2030, unless I win in which case the person I bet against would pay it. Meanwhile the person I bet against would get his winnings from me in 2030, with interest, assuming I lose. It's still not great because from my perspective it amounts to a loan with a higher interest rate basically, so it would be better for me to just take out a long-term loan. (The chance of never having to pay it back is nice, but I only never have to pay it back in worlds where I won't care about money anyway.) Still though it was better than nothing so I took it.

Forecasting Thread: AI Timelines

OK, thanks for the explanation. Yeah life insurance seems marginally useful to me anyway (It costs money in expectation, but makes your risk profile better) so adding in a 30-50% chance that it'll never pay off makes it clearly not worth it I think. To answer your question, well, it would depend on how good the returns are, but they'd have to be unusually high for me to recommend it.

Forecasting Thread: AI Timelines

I don't know what that means, sorry. But my guess is that the answer is no; why pay now for something which will only maybe be worth it 20years from now? 

Predictions for Neuralink's Friday Announcement

I was thinking they'll probably show off a monkey with the device doing something. Last year they were talking about how they were working on getting it safe enough to put in humans without degrading quickly and/or causing damage IIRC; thus I'd be surprised if they already have it in a human, surely there hasn't been enough time to test its safety... IDK. Your credences seem reasonable.

Forecasting Thread: AI Timelines

This is also because I tend to expect progress to be continuous, though potentially quite fast, and going from current AI to AGI in less than 5 years requires a very sharp discontinuity.

I object! I think your argument from extrapolating when milestones have been crossed is good, but it's just one argument among many. There are other trends which, if extrapolated, get to AGI in less than five years. For example if you extrapolate the AI-compute trend and the GPT-scaling trends you get something like "GPT-5 will appear 3 years from now and be 3 orders of magnitude bigger and will be human-level at almost all text-based tasks." No discontinuity required.

Predictions for Neuralink's Friday Announcement

Gosh, I know very little about this. But to get the ball rolling, I'll make some guesses:
--The device will be implanted in a human (20%)

--The device will be allegedly capable of functioning for more than ten years in the brain without causing any significant damage (30%)

--The demonstration will involve controlling a mouse/cursor of some sort on a screen (30%)

--The demonstration will involve controlling a virtual character of some sort (30%)

Forecasting Thread: AI Timelines

Sounds good. Also, check out the new image I added to my answer! This image summarizes the weightiest model in my mind.

Load More