Comments on “What is the Scary kind of AI?”

It’s an interesting and new approach to look at possible AIs through the properties that make them scary (and powerful). I think it’s a great idea and I’m looking forward to read the follow-up chapters. (I’m already starting to imagine a Venn diagram of properties and scenarios…)

To give you constructive feedback: The list here still feels a bit loose, though. Is AGI a distinct property or a group of other properties? What differentiates super intelligent from mind-like? Are motivations and morals another scary property or more of a lens to look at artificial intelligence and understand its agency and hence scariness? And where are the social aspects of the AI, the way it changes social interaction, enables surveillance and oppression etc. - is that all in the power category?