The creator of that repo has a blog with some posts on building a wasm version of FFmpeg, I mostly followed those: https://jeromewus.medium.com/
I also used some of their scripts for building the 3rd-party libraries.
It took a bit of trial and error to see which versions of the different libraries and build tools work together.
We have identified the issues related to AVI and HEVC and will fix them as soon as possible.
Regarding the MKV file, the rendering process hangs a little bit until the demuxing is complete. However, it should continue smoothly once demuxing is finished.
Would be amazing if you could share the mp4 files you're having problems with. Thanks!
You don't need to keep the video files in memory, it's enough to have the javascript File object and then you can read the packets you need on the fly. This way the app's memory usage should be okay.
This is what I did in my similar project (vidmix.app) and seems to be what OP's project is doing, except that they also copy the file to indexedDB to keep it around after the browser tab is closed.
It seems to me that the demonstration is calculated in sRGB space, so with non-linear brightness values and I suspect that most of the unnaturalness of the smear is due to that. To simulate the physics this would need to be done with linear brightness values and only at the end converted to sRGB.
(unless some non-linear effects in human visual perception cancel all that out, but it should at least be mentioned if so)
Thanks for pointing that out. I've done color space conversion in other graphics applications but clearly haven't learned my lesson. I'll double check and update the interactive figures (and text) if it makes sense. The main "torusphere" shader should be fine because the motion blur is non-realistic and hand tuned, but the first couple of interactive figures are a direct application of theory so what you're saying applies to those. Overall I don't think it invalidates the main ideas in the text though.
Yes I noticed that the gamma is all wrong, it actually defeats the purpose of the article since you don't get smooth perceived motion blur with the figures. This could have been so much better.
It's wild to me the author would overlook such a crucial thing (ignoring linear space). But then again, even Adobe barely supports linear and it's 2024.
I've been having the same issue with an app. Random rejections with a single line copy-pasted from the (genuine) feature list in the app description. In this case the lines just start with a regular dash ("-").
This together with the sometimes 10+ day review times makes it impossible to maintain an app on the Play Store now.
For people who don't want to sign up but still like the idea of a video editor running in a browser (that doesn't upload anything), I am working on an alternative:
Feels like we're burying the lede here focusing on firefox/chrome or having to login when it's really damned impressive that you have an entire video editor in the browser.
So it seems like codec support allows viewing individual frames, and you've got some kind of ffmpeg script, and some kind of totally canvas based ui. There's a lot of cool tech in here showing some really interesting possibilities. Congrats on that!
What kind of ui library are you using for this btw?
I've built the UI library myself, it's written in C++ and is drawing with OpenGL. (translated to WebGL by emscripten) Javascript is only used for the browser-related parts.
The point of this is that the javascript layer can be easily replaced and the project built for other platforms. (for example it runs on iOS natively: https://apple.co/3QVAYAq)
Stuff like that and late AV1[0] are one reason I moved to Edge after 19 years on Firefox. More so just that I was impressed with Edge when I tried it. Brave also stood out to me when I tested every browser. It’s fast but has the distinction of being the only semi-major browser that’s fully open source other than Firefox. Edge is still very good, I would say it’s the best browser going but I’d be tarred and feathered for that.
I think using Chrome or Edge for PWAs and Firefox for general browsing is an acceptable compromise. I don't need to use the same program for video editing and reading news articles etc, although it would be nice if I could.
For the technologist that wants that single program, and values privacy and transparency like most of us do, Brave fits the bill nicely. It’s currently my second favorite browser after Edge.
That said, I had Clipchamp installed from the MS Store before I went back to MS Video Editor.
As I understand it, Web Codecs is quite a ways away from being a web standard - it is currently just a draft[1] for a recommendation for a possible future standard.
Just beware of any major API changes or any indication that it might be dropped in its entirety!
I use a WebAssembly build of FFmpeg to extract the streams and packets from the video files and as a fallback for some codecs. But most of the decoding/encoding is done with WebCodecs which in most cases has access to hardware acceleration so can be much faster than ffmpeg.
In Chrome and Edge you can use WebCodecs to decode and display video frames one-by-one. (this is what I used for the video editor at https://vidmix.app )
In other browsers you could build FFmpeg with webassembly and use that for frame-by-frame decoding but it's not going to be nearly as performant.
Decoding the video manually seems so much overkill when all that's needed would essentially be a currentFrame property. But it seems I have to explore this avenue as well. I only aim to support Chrome and Firefox but as much as I hate to admit it, Firefox is somewhat lacking on the video side anyway (as there is a bug that lets it display different video frames for the same currentTime than other browsers or FFmpeg [1]). Therefore I already discourage using Firefox for video annotation.
Yes, the shortcomings of the html video tag are quite unfortunate. Just some very small improvements would make it a lot more useful. One other issue I found at the time I was looking at it (not sure if it's still the case) was that stepping one frame forward wasn't any faster than seeking from a random location. Seemed like the video was decoded from the last keyframe every time, making it inefficient to iterate through the frames.
Couple years ago there was a similar site with the same idea about listing all private keys. If you "found" an active address it was immediately emptied. (I am not sure if it had a search bar or you had to edit the URL)
