Mostly agree with your points, but not sure the analogies at the end hold.
In the cases of the A-bomb aand hydrofluorocarbons, it was almost impossible to deny the damage they could cause. The chemistry didn't leave much to the imagination.
In the case of superintelligent AGI, almost everything is left to the imagination. We have no solid intuitions for what something orders of magnitude smarter than Von Neumann looks like. On the one hand, that means we have no evidence that it wont kill us all. On the other hand. that also means we don't have evidence it *does* pose a threat to anyone, let alone an existential one.
Mostly agree with your points, but not sure the analogies at the end hold.
In the cases of the A-bomb aand hydrofluorocarbons, it was almost impossible to deny the damage they could cause. The chemistry didn't leave much to the imagination.
In the case of superintelligent AGI, almost everything is left to the imagination. We have no solid intuitions for what something orders of magnitude smarter than Von Neumann looks like. On the one hand, that means we have no evidence that it wont kill us all. On the other hand. that also means we don't have evidence it *does* pose a threat to anyone, let alone an existential one.