I think the majority of the population at large either doesn't care about what happened or wish that the guy had actually managed to kill Altman. Not even necessarily because Altman is involved with AI, but just because he is extremely rich. I don't imagine any increased resistance from the population at large - the population at large either doesn't mind when rich people are killed or loves it. The exceptions would be people like entertainers who develop a parasocial relationship with the public and provide direct joy to people, but AI company leaders don't fall into that category.

That said, it is true that killing Altman would almost certainly achieve nothing. See my other post in this thread.

The problem with trying to stop it is, how? Even if you killed every single AI company leader and every single top AI engineer, it would almost certainly just slow down the rate of progress in the technology, not stop it. The technology is so vital to national security that in the face of such actions, state security forces would just bring development of the tech under their direct protection Manhattan Project-style. Even if you killed literally every single AI engineer on the planet, it's pretty likely that this would just delay the development of the technology by a decade or so instead of actually preventing it.

The technology is pushed forward by a simple psychological logic: every key global actor knows that if they don't build the technology, they will be outcompeted by other actors who do build the technology. No key actor thinks that they have the luxury of not building the technology even if they wanted to not build it. It's very similar to nuclear weapons in that regard. You can talk about nuclear disarmament all you want but at the end of the day, having nuclear weapons is vital to having sovereignty. If you don't have nuclear weapons, you will always be in danger of becoming just the prison bitch of countries that do have them. AI seems that it is growing toward a similar position in the calculus of states' notional security.

I can think of no example in history of the entire world deciding to just forsake the development of a technology because it seemed like it could prove to be too dangerous. The same psychological logic always applies.

This is the rhetorical trick that LessWrongers (Yudkowsky's site) have settled on for decades: They have justified everything around the premise that there's a chance, however small, that the world will end. You can't argue that the world ending is a bad thing, so they have their opening for the rest of their arguments, which is that we need to follow their advice to prevent the world maybe ending. They rebut any counterarguments by trying to turn it into a P(doom) debate where we're fighting over how likely this outcome is, but by the time the discussion gets there you've already been forced to accept their argument. Then they push the P(doom) argument aside and try to argue that it doesn't matter how unlikely it is, we have a morally duty to act.

A common advantageous strategy is to take the randomly-selected topic, however unrelated, and invent a chain of logic that claims that taking a given side/action leads to an infinitesimal risk of nuclear extinction/massive harms. This results in people arguing that e.g. "building more mass transit networks" is a bad idea because it leads to a tiny increase in the risk of extinction--via chains as silly as "mass transit expansion needs energy, increased energy production leads to more EM radiation, evil aliens--if they exist--are very marginally more likely to notice us due to increased radiation and wipe out the human race". That's not a made-up example.

The strategy is just like the LessWrongers' one: if you can put your opponent in the position of trying to reduce P(doom), you can argue that unless it's reduced to actual zero, the magnitude of the potential negative consequence is so severe as to overwhelm any consideration of its probability.

In competitive debate, this is a strong strategy. Not a cheat-code--there are plenty of ways around it--but common and enduring for many years.

As an aside: "debate", as practiced competitively, often bears little relation to "debate" as understood by the general public. There are two main families of competitive debate: one is more outward-facing and oriented towards rhetorical/communication/persuasion practice; the other is more ingrown and oriented towards persuading other debaters, in debate-community-specific terms, of which side should win. There's overlap, but the two tend to be practiced/judged by separate groups, according to different rubrics, and/or in different spaces or events. That second family is what I'm referring to above.

No one to my knowledge has ever argued that AI research should be prohibited because of a very small probability of its turning out extremely badly. This is entirely a straw man set up by people who want AI research to continue. Yudkowsky argues that if AI research is allowed to continue, then the natural expected outcome will be very bad (probably human extinction, but more exotic terrible outcomes are also possible) [1]. There are others who argue that no team or organization anywhere should engage in any program of development that has a 10% or more chance of ending the human race without there first being an extensive public debate followed by a vote in which everyone can participate, and this is their objection to any continuance of AI research.

[1] But don't take my word for it: here is Yudkowsky writing in Apr 2022 in https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/: "When Earth’s prospects are that far underwater in the basement of the logistic success curve, it may be hard to feel motivated about continuing to fight, since doubling our chances of survival will only take them from 0% to 0%. That's why I would suggest reframing the problem - especially on an emotional level - to helping humanity die with dignity, or rather, since even this goal is realistically unattainable at this point, die with slightly more dignity than would otherwise be counterfactually obtained."

That kind of idea might have held water in the 90's, but that's not the world we live in any longer.

This misses the point. He specifically said the entire world because the point is that someone will develop AGI (theoretically; I’m not making a statement about how close we are to this).

9 nations have nuclear weapons despite non proliferation agreements and supposed disarmament. It’s not enough for most countries to agree not to build nuclear weapons if the goal is to have no nuclear weapons. Same for AGI. If it can be developed, you need all nations to agree not to develop it if it don’t want it to exist. Otherwise it will simply be developed by nations that don’t agree with you.

(Also arguably the only reason most nations don’t have nuclear weapons is the threat of destruction from nations that already have them if they try.)

I don't remember who, but someone made an interesting point about this around the time GPT-4 was released: If the major nuclear powers all understand this, doesn't that make nuclear war more likely the closer any of them get to AGI/ASI? After all, if the other side getting there first guarantees the complete and total defeat of one's own side, a leader may conclude that they don't have anything to lose anymore and launch a nuclear first strike. There are a few arguments for why this would be irrational (e.g. total defeat may, in expectation, be less bad than mutual genocide), but I think it's worth keeping in mind as a possibility.

Nuclear weapon proliferation seems to have plateaued recently, but I think that this appearance is partly deceptive. The main reasons it has plateaued is that: 1) building and maintaining nuclear weapons is expensive, 2) there are powerful countries that are willing to use military force to stop some other countries from developing nukes, and 3) many countries have reached nuclear latency (the ability to build nuclear weapons very quickly once the political order is given to do it) and are only avoiding actually giving the order to build nukes because they don't see a current important-enough reason to do it.

The prohibitions aren't perfect, of course (and not without their own negative externalities in some cases). But all of those things are much more accessible to people than nuclear weapons, and we've still had successes in banning/reducing them. So maybe there's hope yet.

Your analysis seems to assume that people will remain more afraid of being "outcompeted" than of being murdered, even after a campaign of terrorism that would make 9/11 look minor.

>it often also creates new [problems], often surprising ones

Let's reframe this to remove the negative bias: murder has the obvious direct first-order effect of removing the target from existence, but also a host of non-obvious higher-order effects resulting from people's response to that violence. These can be counterproductive to the goals of the murderer, but they can also work in favor of it. That is why "terrorism" is a real thing - the higher-order effects are essentially a force multiplier, and if you have nothing to lose then the calculus of causing a major disruption begins to look favorable; any disruption, because regression to the mean is good if you're at the shitty end of the bell curve.

AI is such an important technology that in the face of such a campaign of terrorism, governments would bring the development of the technology directly under the protection of the state security forces, largely outside the reach of terrorists. If not in the US, then in China or other places. At that point the terrorists would have to attain a level of power where they could feasibly overthrow the government in order to stop the development of the technology. Now, some scientists would be uncomfortable in such conditions and would stop working on the technology, but enough would remain that the technology would continue to progress, albeit more slowly.

>and if you have nothing to lose then the calculus of causing a major disruption begins to look favorable; any disruption, because regression to the mean is good if you're at the shitty end of the bell curve.

Very true, if the status quo feels shitty enough one becomes extremely willing to just roll the dice.

Sure, but keep in mind that Hitler is already pretty bad. So while yes, killing him might open the door to someone worse stepping in, it may also open the door to someone more level headed.

You know. In theory.

Shock collars / implanted brain bombs would be my evil plan, but he's got smarter people than me on this so who knows?

I think the general population is much more likely to feel joy about it than want a police crackdown.

If we're talking about attacks against average software engineers and obscure founders, fewer people would be happy about it, but a great number still would be. There is a lot of envy toward software engineers and founders.

Someone blindly shooting at Altman’s house is going to kill a neighbour or the housekeeper. Not Sam Altman. Probably not even his family.

The internet may be happy. But the locals will get scared. This happens every time these lone-wolf escalations occur.

But then, jazz is sometimes spoken of as expressing the rhythms, sounds, and emotions of the modern city.

Questionable and violent US foreign policy is much much older than the current Trump administration.

For example, consider the word "remove" and the many different associations it can have.

In this case, the comment (assuming I didn't misread it, which is always possible) seemed obviously to be endorsing specific violence against a specific person and indeed wishing for it to escalate. That's a kind of violence in its own right, and a poison that we don't want here.

Responding to the firebombing of someone's house with the sentence "Removing him is active harm reduction for the world" has an obvious meaning. If you didn't mean it the obvious way, you should have made your point very differently.