Seems like webgpu is not supported by mobile safari yet.
And possibly has coverage in only 65% of the desktop browser market. [1] Does that roughly conform to how you understand the current penetration of this browser api?
Presuming coverage for a given user, I don’t have a good answer for why to consider remote.
It seems like it would be worth testing compatibility for webgpu and attempting to run on the client if possible, but then have a remote instance available otherwise.
Does that make sense to you?
Can you tell me another reason why someone would want a remote instance of whisper given 20x realtime potential at client in your project?
Indeed WebGPU is basically only supported on Chromium based browsers.
This means that the primary usecase for whisper-turbo and my upcoming libraries is Electron/Tauri apps. For users that don't have WebGPU supported for whatever reason, we will still hit OAI/other server deployment. In the ideal case there should be a 90% cost reduction and same or improved UX.
Someone will still want a remote instance today as there is still engineering to be done. I need more aggressive quantization, better developer experience and more features in order to get people off of the OAI API.
I see WebGPU is available in safari tech preview 92, but still an "experimental" feature there.
It looks like Webgpu is kind of been around the block for a while now. I wonder what the hold up is at firefox and safari. It would be much preferable to run more ops on the client. (Complete speculation but I could see this, and the battery use implications possibly making Apple hesitant)
My goal is to provide a browser-based experience first. This is to get at the largest potential user base and there's no friction from install. So, at least for now a electron app is not in plan.
(Disclaimer: I am the author)