XHTML was not backwards compatible and was harder to write for essentially no benefit to most people. "It's XML" was basically the entire list of selling points, and that was just not a compelling enough proposition — in fact, for many people, that was negative value.
I think it was HTML4 that was not forwards compatible.
XML is a subset of SGML, and HTML4 was SGML. XHTML was / is XML. So HTML4 parsers (as being SGML parsers) shouldn't have issues with XHTML.
It's the other way around: XML parsers don't like HTML. So the attempt failed to change the default parsers to XML; as old webpages had than issues, and the web crowd didn't like to fix that. People wanted to continue to use the horrible SGML mess—only because nobody wanted to touch their HTML4.
So Google created their own web "standards council" and ignored the W3C henceforth. The rest is history.
