> GPUs are still Single Instruction Multiple Data (SIMD), for every vector operation you are doing operation: adding vectors, taking a dot production you are only executing a single op at a time.

Sort of. It's both, really. On nvidia at least, the threads in the warp are simd, but between warps it's mimd. And that's before we get into SMs.