Recent Discussion

This post contains the abstract and executive summary of a new 96-page paper from authors at the Future of Humanity Institute and OpenAI.


In many contexts, lying – the use of verbal falsehoods to deceive – is harmful. While lying has traditionally been a human affair, AI systems that make sophisticated verbal statements are becoming increasingly prevalent. This raises the question of how we should limit the harm caused by AI “lies” (i.e. falsehoods that are actively selected for). Human truthfulness is governed by social norms and by laws (against defamation, perjury, and fraud). Differences between AI and humans present an opportunity to have more precise standards of truthfulness for AI, and to have these standards rise over time. This could provide significant benefits to public epistemics and...

1Owain_Evans13mA few points: 1. Political capture is a matter of degree. For a given evaluation mechanism, we can ask what percentage of answers given by the mechanism were false or inaccurate due to bias. My sense is that some mechanisms/resources would score much better than others. I’d be excited for people to do this kind of analysis with the goal of informing the design of evaluation mechanisms for AI. I expect humans would ask AI many questions that don’t depend much on controversial political questions. This would include most questions about the natural sciences, math/CS, and engineering. This would also include “local” questions about particular things (e.g. “Does the doctor I’m seeing have expertise in this particular sub-field?”, “Am I likely to regret renting this particular apartment in a year?”). Unless the evaluation mechanism is extremely biased, it seems unlikely it would give biased answers for these questions. (The analogous question is what percentage of all sentences on Wikipedia are politically controversial.) 2. AI systems have the potential to provide rich epistemic information about their answers. If a human is especially interested in a particular question, they could ask, “Is this controversial? What kind of biases might influence answers (including your own answers)? What’s the best argument on the opposing side? How would you bet on a concrete operationalized version of the question?”. The general point is that humans can interact with the AI to get more nuanced information (compared to Wikipedia or academia). On the other hand: (a) some humans won’t ask for more nuance, (b) AIs may not be smart enough to provide it, (c) the same political bias may influence how the AI provides nuance. 3. Over time, I expect AI will be increasingly involved in the process of evaluating other AI systems. This doesn’t remove human biases. However, it might mean the problem of avoiding capture is somewhat different than with (say) academia and other human institutions
2Daniel Kokotajlo2hWould this multiple evaluation/regulatory bodies solution not just lead to the sort of balkanized internet described in this story [] ? I guess multiple internet censorship-and-propaganda-regimes is better than one. But ideally we'd have none. One alternative might be to ban or regulate persuasion tools, i.e. any AI system optimized for an objective/reward function that involves persuading people of things. Especially politicized or controversial things.

Standards for truthful AI could be "opt-in". So humans might (a) choose to opt into truthfulness standards for their AI systems, and (b) choose from multiple competing evaluation bodies. Standards need not be mandated by governments to apply to all systems. (I'm not sure how much of your Balkanized internet is mandated by governments rather than arising from individuals opting into different web stacks). 

We also discuss having different standards for different applications. For example, you might want stricter and more conservative standards for AI that helps assess nuclear weapon safety than for AI that teaches foreign languages to children or assists philosophers with thought experiments. 

The idea of progress fell out of favor in the course of the 20th century. But when exactly, and why?

In a recent essay I alluded to the pivotal role of the World Wars. Here’s a quote that adds weight to this—from Progress and Power, by historian Carl Becker, published in 1936:

For two centuries the Western world has been sustained by a profound belief in the doctrine of progress. Although God the Father had withdrawn into the places where Absolute Being dwells, it was still possible to maintain that the Idea or the Dialectic or Natural Law, functioning through the conscious purposes or the unconscious activities of men, could be counted on to safeguard mankind against future hazards. However formulated, with whatever apparatus of philosophic or scientific terminology


I mean, even with your image it seems to me that the earlier movie was more fond of the human form, while the later one has a more weird/grotesque view of it. You could say that's the subjectivity of beauty, but that view itself feels to me like part of the decline. Gehry thinks beauty is subjective; his buildings turn me off. Gaudi thought beauty was a specific and analyzable aspect of nature (curved lines and so on); his buildings are my favorite places in the world.

2ChristianKl1hI think this touches on an important aspect of the nature of progress. Thomas Kuhn wrote about how fields where the research is guided more intrinsincly by focusing on problems that a scientific community itself considers important (like theoretical physics) then fields that focus on solving real world projects (like domestic science). It might be imporant for scientists to actually believe that their field can make progress in a notion that's independent of real world application. For Einstein believing in progress meant believing that he could completely revolutionize physics. Eric Weinstein believes that similar progress is still possible in physics while the mainstream theoretical physics community believes that there isn't foundamental progress to be found and what's left is just aligning parts of string theory.
2ChristianKl2hPlenty of advances in knowledge have nothing to do with science and applied science out of academia. The notion that "Without science we would have nothing" sounds like propaganda out of some academic departments that overrates their importance.
1Puxi Deek2hOK I admit that's a bit too absolute. I wasn't using the word science to distinguish stuff that don't follow the scientific method. I'm not sure what to call those. Maybe just human knowledge? I was mainly trying to distinguish pure knowledge that isn't used to make something tangible vs the method of using those knowledge/science to achieve/make something.

I appreciate Zoe Curzi's revelations of her experience with Leverage.  I know how hard it is to speak up when no or few others do, and when people are trying to keep things under wraps.

I haven't posted much publicly about my experiences working as a researcher at MIRI (2015-2017) or around CFAR events, to a large degree because I've been afraid.  Now that Zoe has posted about her experience, I find it easier to do so, especially after the post was generally well-received by LessWrong.

I felt moved to write this, not just because of Zoe's post, but also because of Aella's commentary:

I've found established rationalist communities to have excellent norms that prevent stuff like what happened at Leverage. The times where it gets weird is typically when


He straightforwardly agreed, and said he provides the environment for long term dedication to meditation because there is a market demand for that product. 🤷


FWIW as a resident of MAPLE, my sense is Soryu believes something like:

"Smaller periods of meditation will help you relax/focus and probably have only a very small risk of harm. Larger/longer periods of meditation come with deeper risks of harm,  but are also probably necessary to achieve awakening, which is important for the good of the world." 


But I am a newer resident and could easily misunderstanding here.

2cousin_it18mI used to think the ability to have deep conversations is an indicator of how "alive" a person is, but now I think that view is wrong. It's better to look at what the person has done and is doing. Surprisingly there's little correlation: I often come across people who are very measured in conversation, but turn out to have amazing skills and do amazing things.
1TAG36mThe original "sneer club" comment?
1Puxi Deek1hThem leaving out the exact details of what went on with their groups make the whole discussion sketchy. Maybe they just want to keep the conversation to themselves. If that's the case, why are they posting on LW?

This is an essay about one of those "once you see it, you will see it everywhere" phenomena.  It is a psychological and interpersonal dynamic roughly as common, and almost as destructive, as motte-and-bailey, and at least in my own personal experience it's been quite valuable to have it reified, so that I can quickly recognize the commonality between what I had previously thought of as completely unrelated situations.

The original quote referenced in the title is "There are three kinds of lies: lies, damned lies, and statistics."

Background 1: Gyroscopes

Gyroscopes are weird.

Except they're not.  They're quite normal and mundane and straightforward.  The weirdness of gyroscopes is a map-territory confusion—gyroscopes seem weird because my map is poorly made, and predicts that they will do something other than their normal,...

3cousin_it4hIn the first part, you're saying that price limits have the same intent as withdrawal limits, rationing and so on: to prevent panic and speculation. That's true, but it doesn't matter if the result of price limits is different: empty shelves and not much else. That's what the econ folks have been saying.
1spkoc3hTriple prices or empty shelves is a false dichotomy. Everyone gets the supply and demand curve. That's not the point. Society exists to counter-balance natural bad luck not to amplify it. Social policies that make a disaster even more disastrous for an individual are going to produce rage. Your house got flooded, you have no heat or electricity, you really need some oil for your generator and now that oil is 10 times more expensive. I get that price signals are a good way to coordinate everyone in a community consuming less of a good, but people will fundamentally dislike it because it makes a bad situation worse for an individual. Also the actual reasons economists are against price gouging are hilariously theoretical universe of frictionless spheres type arguments. Supply chains can take ages to react to price changes even in situations where there is no government boogie man tweaking things. Just look at the microchip supply crisis. The actual solution to these issues is having effective emergency supply delivery handled by the government. The whole price gouging conversation is societal bike shedding. Modern governments can and do provide emergency aid almost in real time as disasters happen. If X developed world government lacks that capability, smack'em at the ballot box and tell the next crew to copy whatever the other dozens of countries are successfully doing in that department.

The whole price gouging conversation is societal bike shedding. Modern governments can and do provide emergency aid almost in real time as disasters happen.

A few weeks ago I said the same thing:

About price gouging, I’m not sure this is even the right question. Disaster recovery is the perfect situation where planned economy beats market: there’s a known need, affecting a known set of people equally, and the government has tax money specifically for this need.

And yeah, I also wish the "bikeshed" of price limits stopped being discussed, made into law,... (read more)

1Puxi Deek6hThis is merely hypothesizing the chain of causality. You can never be certain of what action leads to what outcome, but you can definitely put a probability on each branching path. Anything that hasn't happened yet in reality is theoretical. I'm not sure why you are so certain some outcomes would come out 100% while others you merely consider them as fabricated options.

It surprises me that when people think of "software that brings about the singularity" they think of text models, or of RL agents. But they sneer at decision tree boosting and the like as boring algorithms for boring problems.

To me, this seems counter-intuitive, and the fact that most people researching ML are interested in subjects like vision and language is flabergasting. For one, because getting anywhere productive in these fields is really hard, for another, because their usefulness seems relatively minimal.

I've said it before and I'll say it again, human brains are very good at the stuff they've been doing for a long time. This ranges from things like controlling a human-like body to things like writing prose and poetry. Seneca was as good of a philosophy...

This was written for the Vignettes Workshop.[1] The goal is to write out a detailed future history (“trajectory”) that is as realistic (to me) as I can currently manage, i.e. I’m not aware of any alternative trajectory that is similarly detailed and clearly more plausible to me. The methodology is roughly: Write a future history of 2022. Condition on it, and write a future history of 2023. Repeat for 2024, 2025, etc. (I'm posting 2022-2026 now so I can get feedback that will help me write 2027+. I intend to keep writing until the story reaches singularity/extinction/utopia/etc.)

What’s the point of doing this? Well, there are a couple of reasons:

  • Sometimes attempting to write down a concrete example causes you to learn things, e.g. that a possibility is more

Minor note about title change: Originally this was "What 2026 looks like (Daniel's median future)" I intended "what 2026 looks like" to be the primary title, but I was hopeful that some people would be inspired to write their own stories in a similar style, in which case there would be multiple stories for which "what 2026 looks like" would be an appropriate title, and I didn't want to hog such a good title for myself, so I put "daniel's median future" as a backup title. Unfortunately I think the backup title caught on more than the main title, which is a shame because I like the main title more. Since no one is competing for the main title, I deleted the backup title.

To get the best posts emailed to you, create an account! (2-3 posts per week, selected by the LessWrong moderation team.)
Subscribe to Curated posts
Log In Reset Password
...or continue with

“My own heuristics for working in politics are: focus, ‘know yourself’ (don’t fool yourself), think operationally, work extremely hard, ... and ask yourself ‘to be or to do?’” - DC

Dominic Cummings is fascinating for four reasons. One, he is extremely committed to truth-seeking but from a different perspective than most of LW. Two, he has a shocking amount of real-world “success”, especially for a truth-seeker. Three, he fills the missing niche of trying to describe what government is actually like, to great effect. Four, he has uniquely powerful ideas about how to do project management well and how to fix government. 

At the very least, he is extremely thought-provoking, and provides tons of value to >30% of people around me who try reading or listening to him.

However, most...

1spkoc5hCummings' accomplishments are kinda pathetic, actually? He was associated with the successful Brexit effort. OK. So were lots of other people. Cameron was lukewarm on remain and Labour was basically pro-brexit but couldn't talk about it. In retrospect it's not that shocking Remain lost when neither major party was fully campaigning for it. Also this is literally his only meaningful accomplishment. Then he later gets into government as Johnson's fixer, which given that Johnson is averse to actual work means he can basically do whatever he wants. He then fails to dark arts manipulate anyone at a high level and leaves government having done basically nothing. Now he's back to being a blogger, nerdsniping rationalists. He's basically mental viral noise and the #1 source of my self-confidence lowering updates. I read his stuff and it sounds good. Then his results are atrocious. Maybe my instincts suck. Also "Oh the Brexit campaign didn't lie to people as much as Remain", this is delusional. Voting for brexit is polling at 36%( at this point. Tons of examples of business sectors/voting groups who believed the promises that they wouldn't get shafted that did get shafted [] [] [] People thought the tory government would replace the EU development funds to poor regions. Yeah, about that, not they didn't He was part of a
2ChristianKl4hGiven that they said we'll spend the money on the NHS instead of on EU, I don't see how that was what Cummings campaign implied. The EU development funds to poor regions are badly thought out systems and part of the point of Brexit was money not flowing that way and instead to priorities like the NHS. There's no point to have farming subsidies for pig farmers. In a society where people on average eat too much meat, pork should cost at the supermarket the economic price it costs to produce pork and not less because of government subsidies. Brexit allowed to get rid of bad policy like that.

Given that they said we'll spend the money on the NHS instead of on EU, I don't see how that was what Cummings campaign implied.

Thirteen Government ministers and senior Conservatives have today committed that every region, group and recipient of EU funding will continue to get that money after a ‘Leave’ vote in the EU referendum. In an open letter, the signatories - who include Boris Johnson, Michael Gove and Priti Patel -  assure those people and organisations

... (read more)
2Connor_Flexman9hNot sure why you think domestic pressure / public agreement is strong evidence. Public pressure for all sorts of things seems hardly correlated with whether they're beneficial. I think the strongest arguments for Brexit are pretty orthogonal to the economy. Things like "can the government react to crises on the order of weeks instead of months". I do think enough crises would give us data on this but I'm not even sure it will be reasonable to extract counterfactuals from several. Other reasons to do Brexit seem similarly hard to measure compared to myopic economic impact.

One occasionally hears some concerns about falling testosteron/sperm counts, usually in some narrative about the Good Olden days when Men were still real Men, etc. It sounds a little like ' they are turning the frogs gay' type of stuff, but perhaps there is something to the scientific claim after all.

EDIT: It seems my phrasing has unnecessarily antagonized people. Please believe this is a good-faith question. Also, after researching the issue I have become significantly more concerned than before.

Isn't the current consensus that fitoestrogens from soy and grain do not affect male fertility or testosterone level? 

I wrote a post about going without a phone for 10 days. Ten days have now passed, and I'm evaluating my options. This post is about my experience being phoneless and my thoughts about having a phone moving forward.

The last ten days have been extraordinarily peaceful! After a break-in phase of frequently checking my pant pocket for a phantom phone, I began to feel more at ease. After about three days, I felt a calmness that I hadn't enjoyed since middle school. After a week, I became more aware of the passage of time -- my days felt closer to a single drawn-out experience, as opposed to a cluttered collection of moments. During errands, I was forced to spend time waiting for as long as 30 minutes....

Now that I've read this, I really want to go for an extended period without my phone.

I most likely won't follow through with this (90% certainty), even though I want to.


1FlippingRed4hThis experiment illustrates a great way of using tech as utilitarian value. I wonder if you've read Cal Newport's books "Deep Work" and "Digital Minimalism" which the author emphasizes of using technologies for beneficial use and eliminating ones that are distracting or useless.
3adamzerner7hBy getting a feature phone, that provides experimental value. Ie. you get more data on what that lifestyle is like, and whether it is something you want to do longer term. That experimental value seems to me like it overwhelms all other considerations, and thus it would make sense to get a feature phone.
1Jeff Butterworth9hConsider getting the iPhone but change the way you use it. A friend and I are much happier since we made our phones silent. Do not disturb mode, no notifications, etc. All those services you mentioned (Uber, etc) are great and you should still have access to them. I check my phone when I finish a task and need a break. That is usually many times per day. So I do notice important things like calls and texts and I return them, but when it is convenient for me. Your friends will understand occasional delays, just as they did during the last 10 days. It took practice to get out of the addictive phone checking habit, but the same was true of not eating cookies all day. You can do it!