They did; perhaps not in ways that were immediately obvious to the general public. Back in 2014, they created attention to improve how RNNs handled long sequences, an effort driven by google translate. Same goes for when they created the transformer architecture. The first product to see an improvement was also translate. They simply never exposed the bare language models to the public a la OpenAI style with GPT3