Comments on “Limits to reasoning, reduction, and simulation”

I agree with pretty much all of this.

FWIW, Kevin Kelly coined the term “thinkism” for some of this:

Many proponents of an explosion of intelligence expect it will produce an explosion of progress. I call this mythical belief “thinkism.” It’s the fallacy that future levels of progress are only hindered by a lack of thinking power, or intelligence. (I might also note that the belief that thinking is the magic super ingredient to a cure-all is held by a lot of guys who like to think.)

In footnote 4:

Andreas Bender an Isidro Cortes-Ciriano

“an” should be “and”

[pasted from elsewhere]

I’m pretty surprised by the reasoning in this post.

On the one hand, it is naming hard-to-reduce parts very well, as Chapman often does.

But it also seems to be making a point about what is and isn’t possible with “simulation”. I’d have thought, surely Chapman doesn’t believe that simulations are literally isolated from reality—he’s in a lineage that repeatedly emphasizes that “you are already enlightened”, that dualism is an illusion, that “thought” also takes place in reality.

At least on a quick read, Chapman seems to say “experiment” is something different from “pure thinking” but this a very naive and anthropocentric definition of “experiment”! Whatever has intelligence will be situated in reality, and so will have “access” to reality to perform its experiments in, whether or not it looks like a typical xperiment to us.

Simulation is just that part of reality where experiments are cheap (to paraphrase VI Arnol’d).

Also from this same (very short) post:

It seems unlikely that a superintelligent AI could develop biological weapons enormously faster than people can. There’s no substitute for killing a lot of monkeys.

Credit for being very falsifiable, but what an absurd claim!

There’s no substitute for killing a lot of monkeys.

Or Uygers, or any prisoners abandoned by the world.

Another duplicate. In this case, maybe related to the bug you suspect. I switched away from the page after I’d posted, then came back, and it said it had to resend my requests in order to reconstruct the page.

The previous time, however, one of the 2 identical articles was dated yesterday, the 2nd was dated today. That was likely just my doing.

AI can already discover [biological weapons]
(https://futureoflife.org/podcast/sean-ekins-on-the-dangers-of-ai-drug-discovery/), but so far not produce them.