It's not a conclusive test because the net is too lightweight. Such networks don't utilise the compute at maximum, so it's not showing what it would do under heavy load. At least try a ResNet-50.
Interestingly, it's only utilizing the neural engine and no GPU. It'd be interesting to see what happens when M1 Tensorflow updates to utilize both GPU + NE