It's disingenuous to suggest that "Yjs will completely destroy and re-create the entire document on every single keystroke" and that this is "by design" of Yjs. This is a design limitation of the official y-Prosemirror bindings that are integrating two distinct (and complex) projects. The post is implying that this is a flaw in the core Yjs library and an issue with CRDTs as a whole. This is not the case.
It is very true that there are nuances you have to deal with when using CRDT toolkits like Yjs and Automerge - the merged state is "correct" as a structure, but may not match your scheme. You have to deal with that into your application (Prosemirror does this for you, if you want it, and can live with the invalid nodes being removed)
You can't have your cake and eat it with CRDTs, just as you can't with OT. Both come with compromises and complexities. Your job as a developer is to weigh them for the use case you are designing for.
One area in particular that I feel CRDTs may really shine is in agentic systems. The ability to fork+merge at will is incredibly important for async long running tasks. You can validate the state after an agent has worked, and then decide to merge to main or not. Long running forks are more complex to achieve with OT.
There is some good content in this post, but it's leaning a little too far towards drama creation for my tast.
You can split CRDT libs and compose them however you want, but most teams never get past the blessed bindings, because stitching two moving targets together by hand is miserable even if you know both codebases. Then you're chasing a perf cliff and weird state glitches every time one side revs.
In theory you can write better bindings yourself. In practice, if the official path falls over under normal editing, telling people to just do more integration work sounds a lot like moving the goalposts.
Author here, sorry if this was not clear: that specific point was not supposed to be an indictment of all CRDTs, it was supposed to be much more narrow. Specifically, the Yjs authors clearly state that they purposefully designed its interface to ProseMirror to delete and recreate the entire document on every collab keystroke, and the fact that it stayed open for 6 YEARS before they started to try to fix it, does in my opinion indicate a fundamental misunderstanding of what modern text editors need to behave well in any situation. Not even a collaborative one. Just any situation at all.
I think it's defensible to say that this point in particular is not indicting CRDTs in general because I do say the authors are trying to fix it, and then I link to the (unpublicized) first PR in that chain of work (which very few people know about!), and I specifically spend a whole paragraph saying I hope that I a forced to write an article in a year about how they figured it all out! If I was trying to be disingenuous, why do any of that?
To be clear, we ARE arguing CRDTs needlessly complex for the centralized server use case. What I am describing in the "delete and replace all on every keystroke" problem is the point at which it became clear to me that the project did not understand what modern text editors need to perform well in any circumstance, let alone a collab one.
I think this is still reasonable to say because the final paragraph in that section is 100% about how they might fix the delete-all problem, and I hope they do, so that I can write about that, too. But also, that the rest of the article is going to be about how you have to swim upstream against their architecture to accomplish things that are either table stakes or trivial in other solutions.
> To be clear, we ARE arguing CRDTs needlessly complex for the centralized server use case.
I've been working in the OT / CRDT space for ~15 years or so at this point. I go back and forth on this. I don't think its as clear cut as you're making it out to be.
- I agree that OT based systems are simpler to program and usually simpler to reason about.
- Naive OT algorithms perform better "out of the box". CRDTs often need more optimisation work to achieve the same performance.
- But with some optimisation work, CRDTs perform better than OT based systems.
- CRDTs can be used in a client/server model or p2p. OT based systems generally only work well in a centralised context. Because of this, CRDTs let you scale your backend. OT (usually) requires server affinity. CRDT based systems are way more flexible. Personally I'd rather complex code and simpler networking than the other way around.
- Operation based CRDTs can do a lot more with timelines - eg, replaying time, rebasing, merging, conflicts, branches, etc. OT is much more limited. As a result, CRDT based systems can be used for both realtime editing or for offline asyncronous editing. OT only really works for online (realtime) editing.
(For anyone who's read the papers, I'm conflating OT == the old Jupitor based OT algorithm that's popular in google docs and others.)
CRDTs are more complex but more capable. They can be used everywhere, and they can do everything OT based systems can do - at a cost of more code.
You can also combine them. Use a CRDT between servers and use OT client-to-server. I made a prototype of this. It works great. But given you can make even the most complex text based CRDT in a few hundred lines anyway[1], I don't think there's any point.
The algorithm in prosemirror-collab-commit is inspired by Google Wave, and implemented as a slight tweak to the prosemirror-collab system. The tweak is in the name.
I'm not sure about classical OT, and it's been a really long time since I wrote prosemirror-collab-commit, but.. On the authority it's more like nth triangle where n is the number of concurrent commits being processed based off the same document version for mapping. So 50 clients sending in commits at the same based off the same doc version would be (50 * 51) / 2. Applying has a different, potentially larger, cost and that's O(n).
You don't have to have server affinity, but I'd be cooler if you did. Locks you need.
It works offline, sure. For some definition of "works". The further histories diverge the greater the chance of losing user intent. But that really depends on the nature of the divergence. Some inspection and heuristics could probably be used to green-light a LOT of "offline" scenarios before falling back on an interactive conflict resolution strategy.
I'm not sure what magic CRDTs exist today, but in the case of Yjs and ProseMirror allowing histories to drift too far will absolutely risk stomping all over user intent when they are brought back together.
The magic of CRDTs does not prevent this. They are in exactly the same boat as OT, prosemirror-collab and prosemirror-collab-commit. It can't be prevented. The problem is worse with CRDTs because they instantly destroy user intent in the conversion to/from their underlying representation, which is the XML document. See discussion with Marijn about, e.g., splitting blocks above.
Heya, cheers. I'm actually intimately familiar with the node splitting issue. I've created y-prosemirror backends(complete with edit history, snapshots, etc) and "rewrote" y-prosemirror in TypeScript heavily refactoring and modifying it for some crazy use cases.
Those use cases hit a wall with, and I'm a bit fuzzy, the Yjs data structure and y-promirror diffing algorithm destroying and creating new XML nodes; black-holing anything else that occurred in them or duplicating content.
Actually, I think I agree with most of this... except the part where you think it's not clear-cut, ha ha. I meant that not as a comparison between OT and CRDTs, but as a comparison to prosemirror-collab. My opinion is that in the centralized server case, it is (unfortunately) basically better in every dimension.
> But with some optimisation work, CRDTs perform better than OT based systems.
I read your paper and I think this is a mistake. You assume that OT has quadratic complexity because you're considering classic operation-based OT. But OT can be id-based, in which case operations are transformed directly on the document, not on other operations. This is essentially CRDT without the problems of supporting P2P, and therefore the best CRDT will never perform better than the best OT.
> CRDTs let you scale your backend. OT (usually) requires server affinity. CRDT based systems are way more flexible. Personally I'd rather complex code and simpler networking than the other way around.
All productivity apps that use these tools in any way shard by workspace or user, so OT can scale very well.
If you don't scale CRDT that way, by the way, you'd be relying too much on "eventual consistency" instead of "consistency as quickly as possible."
> (For anyone who's read the papers, I'm conflating OT == the old Jupiter-based OT algorithm that's popular in Google Docs and others.)
Similar to what I said before. I think limiting OT to an implementation that’s over three decades old doesn’t do OT justice.
> I think limiting OT to an implementation that’s over three decades old doesn’t do OT justice.
I haven't kept up with the OT literature after a string of papers turned out to "prove correctness" in systems which later turned out to have bugs. And so many of these algorithms have abysmally bad performance. I think I implemented an O(n^4) algorithm once to see if it was correct, but it was so slow that I couldn't even fuzz test it properly.
> You assume that OT has quadratic complexity because you're considering classic operation-based OT. But OT can be id-based, in which case operations are transformed directly on the document, not on other operations.
If you go down that road, we can make systems which are both OT and CRDT based at the same time. Arguably my eg-walker algorithm is exactly this. In eg-walker, we transform operations just like you say - using an in memory document model. And we get most of the benefits of OT - including being able to separately store unadorned document snapshots and historical operation logs.
Eg-walker is only a CRDT in the sense that it uses a grow-only CRDT of operations, shared between peers, to get the full set of operations. The real work is an OT system, that gets run on each peer to materialise the actual document.
> This is essentially CRDT without the problems of supporting P2P, and therefore the best CRDT will never perform better than the best OT.
Citation needed. I've published plenty of benchmarks over the years from real experiments. If you think I'm wrong, do the work and show data.
My contention is that the parts of a CRDT which make them correct in P2P settings don't cost performance. What actually matters for performance is using the right data structures and algorithms.
It seems to me the burden of proof is on you. You were the one who claimed that “CRDTs perform better than OT-based systems.”
I’m simply denying it. My reasoning is that CRDTs require idempotence and commutativity, while OTs do not. What requirement does OT have that CRDT does not? Because if there isn’t one, then by definition your claim can’t be correct. And if there is one, that would be new to me, although I suspect you might be using a very particular definition of OT.
> It seems to me the burden of proof is on you. You were the one who claimed that “CRDTs perform better than OT-based systems.”
Ah, I assumed we were talking about Jupiter based OT systems - which are outperformed by their newer cousins (like eg-walker). Like you say, these use a different data structure to transform changes and that's why they're faster.
> My reasoning is that CRDTs require idempotence and commutativity, while OTs do not.
The only property not required by a centralized OT system is the OT TP2 property. Ie, T(op3, op1 + T(op2, op1) == T(op3, op2 + T(op1, op2)). Central servers also give you a single global ordering.
If you discard TP2 and add global ordering, does that open the door to new optimisations? I don't know, and I certainly can't prove the absence of any such optimisations. So I think the burden of proof is on you.
The root of our misunderstanding or debate is clear: although CRDT is fairly well defined, I don’t think the same is true for OT.
What I have in mind is what I mentioned earlier:
> OT can be id-based, in which case operations are transformed directly on the document, not on other operations.
This is exactly what I do in my library DocNode[1], which I describe as “id-based OT”.
With this model, it’s not even necessary to satisfy TP1. In fact, the concept T(o1, o2) doesn’t exist, because operations aren’t “transformed” against other operations, but against the document. Maybe the word “transform” is a bit misleading, and “apply” would be more appropriate. The problem is that there is still a slight transformation. For example, if a client inserts a node between nodes A and B, but by the time it reaches the server B has been deleted, the effective operation might become “insert between A and C”.
The server is append-only. The client has several options to synchronize with the server: rebasing, undo-do-redo, or overwriting the document.
Maybe I’m the one who shouldn’t describe DocNode as “[id-based] OT” and should instead coin a new term. Operational Application (OA)? Operations Without Transformation (OWT)? Operations Directly Transformed (ODT)? Operational Rebasing (OR)? Not sure. What would you recommend?
Just to be extremely clear: we pay for a lot of OSS software. We pay one individual project more than $10,000 a year. We would have paid for Yjs too, if we thought it was a good use of resources!
Have you considered doing property tests with Mathematica as an oracle?
An ai based development workflow with a concrete oracle works very well. You still need the research and planing to solve things in a scalable way, but it solves the "are the tests correct" issue.
What we've done is pull out failing property tests as a unit tests, makes regression testing during the agentic coding loop much more efficient.
I'm convinced that LLMs results in all software needing to be open source (or at the very least source available).
In future everyone will expect to be able to customise an application, if the source is not available they will not chose your application as a base. It's that simple.
The future is highly customisable software, and that is best built on open source. How this looks from a business perspective I think we will have to find out, but it's going to be fun!
Why do you think customization can only viably done via changing the code of the application itself.
I think there is room for closed source platforms that are built on top of using LLMs via some sort of API that it exposes. For example, iOS can be closed source and LLMs can develop apps for it to expand the capabilities of one's phone.
Allowing total customization by a business can allow them to mess up the app itself or make other mistakes. I don't think it's the best interface for allowing others to extend the app.
> In future everyone will expect to be able to customise an application, if the source is not available they will not chose your application as a base. It's that simple.
This seems unlikely. It's not the norm today for closed-source software. Why would it be different tomorrow?
Because we now have LLMs that can read the code for us.
I'm feeling this already.
Just the other day I was messing around with Fly's new Sprites.dev system and I found myself confused as to how one of the "sprite" CLI features worked.
So I went to clone the git repo and have Claude Code figure out the answer... and was surprised to find that the "sprite" CLI tool itself (unlike Fly's flycli tool, which I answer questions about like this pretty often) wasn't open source!
That was a genuine blocker for me because it prevented me from answering my question.
It reminded me that the most frustrating thing about using macOS these days is that so much of it is closed source.
I'd love to have Claude write me proper documentation for the sandbox-exec command for example, but that thing is pretty much a black hole.
I'm not convinced that lowering the barrier to entry to software changes will result in this kind of change of norms. The reasons for closed-source commercial software not supporting customisation largely remain the same. Here are the ones that spring to mind:
• Increased upfront software complexity
• Increased maintenance burden (to not break officially supported plugins/customizations)
• Increased support burden
• Possible security/regulatory/liability issues
• The company may want to deliberately block functionality that users want (e.g. data migration, integration with competing services, or removing ads and content recommendations)
> That was a genuine blocker for me because it prevented me from answering my question.
It's always been this way. From the user's point of view there has always been value in having access to the source, especially under the terms of a proper Free and Open Source licence.
Manifold works on solid triangle meshes, OpenCascade is a true BREP kernel that represents solids as edges (straight and curved) and surfaces (not meshed) computed from those edges. There is no triangulation in the root model in OpenCascade.
Yes, exactly. The other reason Cloudflare workers runtime is secure is that they are incredibly active at keeping it patched and up to date with V8 main. It's often ahead of Chrome in adopting V8 releases.
I didn’t know this, but there are also security downsides to being ahead of chrome — namely, all chrome releases take dependencies on “known good” v8 release versions which have at least passed normal tests and minimal fuzzing, but also v8 releases go through much more public review and fuzzing by the time they reach chrome stable channel. I expect if you want to be as secure as possible, you’d want to stay aligned with “whatever v8 is in chrome stable.”
Cloudflare Workers often rolls out V8 security patches to production before Chrome itself does. That's different from beta vs. stable channel. When there is a security patch, generally all branches receive the patch at about the same time.
As for beta vs. stable, Cloudflare Workers is generally somewhere in between. Every 6 weeks, Chrome and V8's dev branch is promoted to beta, beta branch to stable, and stable becomes obsolete. Somewhere during the six weeks between verisons, Cloudflare Workers moves from stable to beta. This has to happen before the stable version becomes obsolete, otherwise Workers would stop receiving security updates. Generally there is some work involved in doing the upgrade, so it's not good to leave it to the last moment. Typically Workers will update from stable to beta somewhere mid-to-late in the cycle, and then that beta version subsequently becomes stable shortly thereafter.
Thanks for the clarification on CF's V8 patching strategy, that 24h turnaround is impressive and exactly why I point people to Cloudflare when they need production-grade multi-tenant security.
OpenWorkers is really aimed at a different use case: running your own code on your own infra, where the threat model is simpler. Think internal tools, compliance-constrained environments, or developers who just want the Workers DX without the vendor dependency.
Appreciate the work you and the team have done on Workers, it's been the inspiration for this project for years.
This is one of the main use cases we are building "Durable Streams" for, it's an open source spec for a resumable and durable stream protocol. It's essentially an append only log with a http api.
When we built ElectricSQL we needed a resumable and durable stream of messages for sync and developed a highly robust and scalable protocol for it. We have now taken that experience and are extracting the underlying transport as an open protocol. This is something the industry needs, and it's essential that it's a standard that portable between provider, libraries and SDKs.
The idea is that a stream is a url addressable entity that can be read and tailed, using very simple http protocol (long polling and a SSE-like mode). But it's fully resumable from a known offset.
We've been using the previous iteration of this as the transport part of the electric sync protocol for the last 18 months. It's very well tested, both on servers, in the browser, but importantly in combination with CDNs. It's possible to scale this to essential unlimited connections (we've tested to 1 million) by request collapsing in the CDN, and as it's so cacheable it lifts a lot of load of your origin when a client reconnect from the start.
For the LLM use case you will be able to append messages/tokens directly to a stream via a http post (we're working on specifying a websocket write path) and the client just tails it. If the user refreshes the page it will just read back from the start and continue tailing the live session. Avoids appending tokens to a database in order to provide durability.
This is what I'm most interested in. I have an application which has a smaller trimmed down client version but it shares a lot of code with the larger full version of itself. Part of that code is query logic and it's very dependent on multiple connections and even the simplest transactions on it will deadlock without multiple connections. Right now if one wants to use the Postgres option, it needs Postgres manually installed and connected to it which is a mess. It would be the dream to have a way to easily ship Postgres in a small to medium sized app in a enterprise-Windows-sysadmin-friendly way and be able to use the same Postgres queries.
It is very true that there are nuances you have to deal with when using CRDT toolkits like Yjs and Automerge - the merged state is "correct" as a structure, but may not match your scheme. You have to deal with that into your application (Prosemirror does this for you, if you want it, and can live with the invalid nodes being removed)
You can't have your cake and eat it with CRDTs, just as you can't with OT. Both come with compromises and complexities. Your job as a developer is to weigh them for the use case you are designing for.
One area in particular that I feel CRDTs may really shine is in agentic systems. The ability to fork+merge at will is incredibly important for async long running tasks. You can validate the state after an agent has worked, and then decide to merge to main or not. Long running forks are more complex to achieve with OT.
There is some good content in this post, but it's leaning a little too far towards drama creation for my tast.
reply