> And Keras was far more annoying because it wants scipy, which wants BLAS, and I don't use Anaconda. I haven't gotten around to that one yet...

Christoph Gohlke to the rescue: http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy

I also don't think that the numpy installed via pip is using mkl, so his version of numpy should be faster too: http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy