"In recent months, we’ve seen further improvements to the state of the art in RNN language modeling. The current state of the art results are held by two recent papers by Melis et al. and Merity et al.. These models make use of most, if not all, of the methods shown above, and extend them by using better optimizations techniques, new regularization methods, and by finding better hyperparameters for existing models. Some of these methods will be presented in part two of this guide."
Am I right in saying that the recently publicised Google Transformer [1] Neural Network is actually the state of the art now, over RNNs?
The Transformer network is solving a different problem: translating a given sentence into another with the same meaning.
The problem discussed here is about completing the next word in a partial sentence, where AFAIK some variety of RNN is still best. It might be possible to adapt the Transformer architecture to that task, but that would make it a different model.
Am I right in saying that the recently publicised Google Transformer [1] Neural Network is actually the state of the art now, over RNNs?
[1] https://research.googleblog.com/2017/08/transformer-novel-ne...