Comments on “Limits to reasoning, reduction, and simulation”

Add new comment


Bill Benzon 2023-02-13

I agree with pretty much all of this.

FWIW, Kevin Kelly coined the term “thinkism” for some of this:

Many proponents of an explosion of intelligence expect it will produce an explosion of progress. I call this mythical belief “thinkism.” It’s the fallacy that future levels of progress are only hindered by a lack of thinking power, or intelligence. (I might also note that the belief that thinking is the magic super ingredient to a cure-all is held by a lot of guys who like to think.)


Daniel Filan 2023-02-14

In footnote 4:

Andreas Bender an Isidro Cortes-Ciriano

“an” should be “and”

Thinkism & typo

David Chapman 2023-02-14

Bill — thank you very much for that; I’ve read it and it was interesting and yes similar.

Daniel — thanks, I always appreciate corrections! Fixed now.

The simulation "box"

S 2023-02-15

[pasted from elsewhere]

I’m pretty surprised by the reasoning in this post.

On the one hand, it is naming hard-to-reduce parts very well, as Chapman often does.

But it also seems to be making a point about what is and isn’t possible with “simulation”. I’d have thought, surely Chapman doesn’t believe that simulations are literally isolated from reality—he’s in a lineage that repeatedly emphasizes that “you are already enlightened”, that dualism is an illusion, that “thought” also takes place in reality.

At least on a quick read, Chapman seems to say “experiment” is something different from “pure thinking” but this a very naive and anthropocentric definition of “experiment”! Whatever has intelligence will be situated in reality, and so will have “access” to reality to perform its experiments in, whether or not it looks like a typical xperiment to us.

Simulation is just that part of reality where experiments are cheap (to paraphrase VI Arnol’d).

Also from this same (very short) post:

It seems unlikely that a superintelligent AI could develop biological weapons enormously faster than people can. There’s no substitute for killing a lot of monkeys.

Credit for being very falsifiable, but what an absurd claim!

There’s no substitute for killing a lot of monkeys.

Hal Morris 2023-02-18

There’s no substitute for killing a lot of monkeys.

Or Uygers, or any prisoners abandoned by the world.

Another duplicate

Hal Morris 2023-02-18

Another duplicate. In this case, maybe related to the bug you suspect. I switched away from the page after I’d posted, then came back, and it said it had to resend my requests in order to reconstruct the page.

The previous time, however, one of the 2 identical articles was dated yesterday, the 2nd was dated today. That was likely just my doing.

AI can already discover biological weapons

Ondrej Kubu 2023-03-27

AI can already discover [biological weapons]
(, but so far not produce them.

Add new comment:

You can use some Markdown and/or HTML formatting here.

Optional, but required if you want follow-up notifications. Used to show your Gravatar if you have one. Address will not be shown publicly.

If you check this box, you will get an email whenever there’s a new comment on this page. The emails include a link to unsubscribe.