No credible AI expert thinks LLMs will lead to AGI. They are hilariously bad at most things and heavily dependent on questionable data sources like reddit.
AI don't need to be competent to cause harm, just like humans. Just ask law enforcement arresting people because their face recognition AI told them to.
Not that I trust governments trying to oversee them either...