This is perhaps only tangentially related...
but I'm trying to do something to support sharding of websocket connections based on a user_id....
I want the connections to be distributed to separate instances of a WS server based on modulo-based or consistent hashing algorithm... above that in the stack we have a message broker or queue partitioned using the same sharding algorithm for the number of websocket servers running. This queue/message broker sends publishes messages to each web socket service which in-turn pushes the message to the client. This is how I want to handle horizontally scalable web socket connections...
The question I have is the best way to do this and the best way to migrate connections. From one service to another when/if the replication factor of web socket servers changes (we add/remove servers)...
One way I would think would be to send a disconnect message to the websocket service when this event happens and it would disconnect it's clients and then have a random delay for when re-connection happens... so as to avoid trying to reinitialize all of those connections at once. This seems sub-optimal as with consistent-hashing many of the connections will stay on the same server, and terminating those connections would be unnecessary.
What is the best way to migrate an open TCP from one server to another? Or if the connection must be closed... how can we minimize the number of connections that are to be closed.
BTW: I know I can broadcasts to all instances of the websocket server, and then unless that server holds the connection it is no-op... if the server does have a connection then send the message back... This also seems it would have a limit to how far this scaling-strategy would grow before running into limitations.
Linux has TCP connection repair, which can be used to transparently migrate an active TCP session between two Linux boxes. The TCP traffic will be paused for the duration of the migration, but the other endpoint will not notice as long as the migration is complete before TCP timeouts kick in.
I am one of the co-founders of https://ably.com. We have built a Serverless websockets platform that is near infinitely scalable and designed to handle the complexity you’re describing such as cluster resizing, connection failures, automatic sharding and reallocations using hash rings across our global edge service etc. I’d be really interested to hear why you wouldn’t consider offloading this to a service like ours which powers the likes of HubSpot, Expedia, Spotify etc. Feel free to contact me directly @mattheworiordan if you’d prefer not to comment here. I’m really keen to hear different perspectives on what we’re doing right and where we can improve!
Can you just store the shard id for each user in a database? Each server could check if it is authorative for each incoming connection, and redirect if not.
That way, you can migrate users one by one (sending a 'redirectUserConnections(userId)' to the original server, and even pick a shard that is geographically close to the user. (If you care about race conditions during migrations, you need to be really careful though.)
race conditions during migrations are the biggest concern.... You do not want a client with a connection to server they will never receive a message on.
Could be anything (redis pub/sub with a channel for each shard, sqs with separate queues for each shard, separate partitions in kafka, etc)...
haven't committed to using one particular technology. queues that support retries could be useful in the event that a disconnect happens messages wouldn't be necessarily lost. Something custom built on top of Redis could accomplish the same perhaps.
Yes... but this seems to be then a timing issue to me... what happens if the replication factor changes again while they are reconnecting to the wrong server... then they will miss the second 'reconnect' message... and will stay connected to this server where it will never receive messages... perhaps an edge-case scenario.
I recommend storing the server topology somewhere that's super highly available, like Cloudflare. Then having the client choose from that. If their connection drops (use heartbeats), retry a different one you have and refresh the server topology in parallel. If the second connection fails, retry again with the new topology. This is convergent and avoids putting load on your database to connect, so you don't suffer from thundering herd problems.
This is not about the syntax (indeed that's C-like), but much more about the style of API. This style of API was basically invented in Node.js land and then carried over to other languages. And lots of JS-style syntax sugar has been added to the other languages to make it possible.
If someone made a web app in C++, it used to be OOP heavy. There's no class declaration in sight here.
> This style of API was basically invented in Node.js land and then carried over to other languages.
You mean Lisp, and no surprise there, since JavaScript was supposed to be Scheme, except the commercial interests forced it to have Java-like syntax :).
I write some Common Lisp and while it's alike, this C++ API is much more similar to the JS API than to anything I've written in CL or Clojure. But yeah, I guess the basic idea is from Lisp.
µWebSockets is a simple to use yet thoroughly optimized, standards compliant and secure implementation of WebSockets (and HTTP). It comes with built-in pub/sub support, URL routing, TLS 1.3, SNI, IPv6, permessage-deflate
Header-only libraries are a common thing in C++. They allow pretty easy consumption, because the consumer only needs to specify the include path. That's fairly easy with any build system (and C++ has a lot of them).
Any binary library would also require building the library itself using the same toolchain and then linking to it - which requires a lot more configuration on the consuming side. It might be easy if both projects use the same build system (e.g. cmake), but that's far from guaranteed in C++.
I think things have changed, but back when I was doing C++, templates (generics) were implemented in header files, so it was not unusual to have entire programs written in .h files.
Having dealt only a tiny bit with Node, GYP (it's garbage C++ interop layer) and the inexplicable dependency on Python (???) being installed on the devs machine in order to build the C++ code that got copied into node_modules, I know better than to ever try another library making use of that.
I don't think you have less dep when building other c++ projects. People also use automake, autoconf or whatever to generate makefiles. It is just that the author choose python here. I never see anybody writes makefile manually and build moderate or big size project. There are always tons of tools used.
That's epsilon, not ypsilon (it just occurred to me that it's more common to see it called "upsilon" in English, though I don't know why based on how the letter is pronounced - "eee-psilon")
Cool little library, I'm using it, but author has an ego the size of Wolfram's. This is just one snippet, there are many if you rummage through the docs and issues:
> In the 1970s, programming was an elite's task. Today programming is done by uneducated "farmers" and as a result, the care for smart algorithms, memory usage, CPU-time usage and the like has dwindled in comparison.
I used to vehemently agree with stuff like this, but now I just cringe. Yeah software today sucks, but comparing it with the 70s/80s/90s is just stupid. We don't ship software for one piece of incredibly simple hardware and a few thousand customers anymore. Elitism isn't going to make the situation any better and frankly unproductive whining like this is just getting exhausting.
I "vehemently agree" but I don't insult farmers or other blue collar workers because that's a shitty thing to do. The author is just an asshole and should be punched in the face by a farmer.
Oh wow I need to see that series. I hope it's hilarious.
I think farming is misunderstood because historically it's what 95% of people did, and it wasn't all that high tech. Not that pre-modern farming was easy: there's endless nuances about how to actually grow plants/animals optimally. But modern farming is ridiculously high tech. A modern farmer is a business owner, a logistics engineer, a mechanic, a programmer/hacker, and a labourer. I'm probably missing a dozen sub-professions of farmer. I bet "calories generated per farmer per year" would be a fascinating timeseries graph to view.
also farmers are incredibly resourceful. Every farmer i have ever met can fix any just about equipment they have because its so difficult and expensive to get someone to come out and fix it.
I would call players who expend time in lieu of skill "grinders." "Farming," on the other hand, is often a late-game playstyle which gets the best results (farming for rares to fully equip a build).
I mean the necessity to do this can be seen as bad game design, and maybe the players are dummies for picking bad games (although the Diablo and Destiny series are pretty well received), but this sort of thing is required to play at the highest levels...
> Some of you need to learn how to read, or learn how to not get so emotional.
How is anyone supposed to infer that from the quotes? I am not a gamer so I would never know this. Its domain specific jargon that overlaps with long standing definitions.
Insulting an entire group of people, by directly implying that farmers are not skilful or educated, is worse than insulting the person who made the insult in the first place.
But he didn't insult crop farmers, he placed the word in quotes.
A "farmer" (note the double quotes) is a gaming/internet culture term that denotes someone who does repetitive, boring tasks in order to accumulate some kind of value.
I'm pretty certain that intelligent people who comment here are aware of it, but are using mental gymnastics in order to justify the insults they'll direct towards the author of microwebsockets.
a) For what it's worth, I did not immediately think of that meaning of the term, despite being familiar with it. I suspect that despite your claims most people would not. Using quotes just means that the author is not claiming that programming is done by literal farmers.
b) Even if that's what the author meant, it is still a derogatory term when you consider its origin. Why do you think gaming culture picked "farmer" as the word used to describe someone who performs mindless repetitive tasks?
> I'm pretty certain that intelligent people who comment here are aware of it,
It has nothing to do with intelligence. I don't game. I don't consume media that would constantly expose me to gaming memes either. Mind you, I have heard of gold farmers in passing. Though when I read the word 'farmers' the definition I am familiar with is older than all of us put together. Nice try at an insult though.
> A "farmer" (note the double quotes) is a gaming/internet culture term that denotes someone who does repetitive, boring tasks in order to accumulate some kind of value.
Question for you: where did that come from? Oh, that's right, by implication that actual real life farmers do repetitive boring tasks in order to accumulate some kind of value.
I’m clearly saying, like others are in this thread, that just because you associate calling someone a “farmer” with an anachronistic gaming term doesn’t mean that almost everyone will read it in a different way, nor does it mean that your interpretation is correct.
‘What do you do for a job?’
‘I’m a farmer’
‘Oh right what’s the best quest to farm on in WOW?’
Because they are not actually farmers, but have similar negative stereotypical traits?
Why are you so eager to accuse everyone of blind rage? I read the whole thing. It’s bog standard “these kids on my lawn” stuff which isn’t worth any brain power discussing.
However, you not being able to understand why people didn’t assign some convoluted meaning to the word farmer is quite interesting.
> In the 1970s, programming was an elite's task. Today programming is done by uneducated "farmers" and as a result, the care for smart algorithms, memory usage, CPU-time usage and the like has dwindled in comparison.
What's wrong with that statement? In 1970's, the number of people who dealt with programming was significantly smaller than today. It's a matter of fact. And people who worked with computers back then were highly educated and pioneers of the technology, there was no room for half-hearted effort or "let's copy the first result from stackoverflow.com".
Today, programmers who are meticulous, detail-oriented and who genuinely recognize the problem first are in minority and that's how I read the above.
The author might have stated it abruptly, but it does not mean he's wrong. You're using the "cool little library", yet you have this kind of resentment for the person who made it, have you even tried to look at the situation from their perspective before you passed the judgement?
I'm asking out of curiosity, because I can relate to having to work with under-skilled individuals who spend more time sorting out their CV and online presence than their work.
You can be both right and an asshole. These things are not exclusive. There is a way to say what he is saying without calling all of today's programmers "farmers", which also has some undertones that being a farmer is somehow shameful.
> under-skilled individuals who spend more time sorting out their CV and online presence than their work
That sounds more like a organizational/hiring problem.
In gaming culture, a farmer is someone who repeats the same actions for extended periods of time.
It's not an insult in gaming context, and "farming" in the context of our craft does not relate to actual farmers. It sounds like taking things out of context to gain moral upper hand which allows for insulting the author.
I have to be devil's advocate, this whole shaming because someone else might have gotten offended is seriously tiresome.
Library is crafted well, guy's annoyed with self-proclaimed crappy devs, yet the freedom to say so is revoked by the very community he helps.
But it's not an insult towards actual farmers, it's an insult towards an impostor that pretends they're doing valuable work while all they do is the opposite and they merely throw time at a problem, creating illusion of value.
I'm saying this because I'm raised on a farm, I was always proud of it, I still do it as a hobby (and I love my crops) and I never, ever considered it an insult when I heard a term "farmer" or "peasant".
Nowadays, I read about people being offended in my stead, giving themselves the right to insult other people, like author of microwebsockets.
Is there a world in which haters could hate without using excuses such as "it insults farmers"?
I'm one of the farmers, I am not insulted, I do have the right to say this and point the attention at the *real* problem - and it's fake developers.
"Farmer" is even more degrading in Swedish ("bonde"), to the point where you avoid using it even for actual farmers and say "agriculturalist" ("jordbrukare") instead.
It's a poorly translated word written in a haste. A better translation would be "red neck" (I think?). This is translated from my inner Swedish thoughts and we don't have Californian wokeness here (especially not down here in Skåne). But instead of finding a new word I reformulated that section everyone is so upset about. It's fascinating how easy it is for internet people to assume worst intent in everything, and look for things to be offended by, rather than to read between the lines and adjust ones interpretation based on context.
If you read the section, applying context from what the project is about and where it is used, you should pretty easily grasp the overall idea that is: "compression sucks and is incredibly inefficient". It would be a much more interesting thread if people took that message and argued against/for it, technically. But now we are here arguing about some word usage that might be offensive if interpreted in a very literal way. Congratulations; we have reached peak cancel culture. I hope you enjoy it while it lasts.
> In gaming culture, a farmer is someone who repeats the same actions for extended periods of time.
This isn't gaming. This is a programming library.
> It's not an insult in gaming context
This is not a "gaming context."
> and "farming" in the context of our craft does not relate to actual farmers
I've never heard anyone refer to "farming" in the context of programming. Please share with me more of the use of farming in the context of programming (not gaming).
> I have to be devil's advocate
This is a lie. You do not "have" to be devil's advocate. This was a choice you made.
> yet the freedom to say so is revoked by the very community he helps.
This is a lie.
Stop being a liar.
Finally, do not "revoke" my "freedom" to say this to you by disagreeing or arguing against me.
> This isn't gaming. This is a programming library.
And humans can be gamers and programmers. Welcome to real life, tutorial is now over.
> This is not a "gaming context."
When a word is placed in quotes, convention is that it has alternate meaning. I gave perspective, gaming is something that's socially well-known, along with its terms. Don't pretend you're unaware. Don't be a liar.
> I've never heard anyone refer to "farming" in the context of programming. Please share with me more of the use of farming in the context of programming (not gaming).
I won't.
> This is a lie. You do not "have" to be devil's advocate. This was a choice you made.
And these are mental gymnastics that you allow yourself so you can justify being rude. Don't be a liar.
> This is a lie.
> Stop being a liar.
Feel free to stop being a liar.
> Finally, do not "revoke" my "freedom" to say this to you by disagreeing or arguing against me.
I can assume you're accustomed to being invasive, rude and impolite but please, keep it for your inner circle of people. Or keep giving out orders to strangers from internet who will never interact with you after exchanging these words.
Speaking for myself, how I interpreted it is that there is some relationship between how unskilled developers do their job and what takes place during actual farming.
When I first heard of the term farmer/farming in-game, I didn't think anyone meant to point to any stereotypes either, just that the actual act had similarities with farming. Granted, in this sentence it's easier to interpret it the other way, but isn't it always better to give someone the benefit of the doubt? I can see how the process of programming lazily without understanding can resemble farming -and there might be a more intuitive metaphor if the author is not a native english speaker.
He follows through with an entire rant about people using "2022-10-26 14:59:42" to represent dates instead of 1666789167 Unix timestamps, and JSON in general. If that's really your best example then meh... I use JSON and string time formats because it's just easier to inspect things with curl or whatnot and gives a nice balance between human-readable and machine-readable, which I rather like for APIs. I do the same with database stuff, so "select * from foo" can be easily read. It can be quite helpful in debugging and the performance difference (even with compression) is just negligible in most cases.
But what do I know ... I'm just a farmer. Now excuse me while I go shove my computer down a shredder and go do something else.
> In 1970's, the number of people who dealt with programming was significantly smaller than today. It's a matter of fact. And people who worked with computers back then were highly educated and pioneers of the technology, there was no room for half-hearted effort or "let's copy the first result from stackoverflow.com".
And yet probably the same percentage of what they wrote was utter crap [0]. The illusion comes from having more people writing more code for more reasons now. Sturgeon's law still applies.
On top of that, the industry rewards those who:
* "Produce results" insetad of those who pre-empted a bug 5 years in advance by building something to be robust [1].
* Job hop every year or two, and thereby disproportionately rewarding those who work on their CV.
* Are personal friends and family of higher ups [2].
To claim the major reason why software is in its current state is largely/solely because of all the "farmers" is harmful ignorance, at best.
> programmers who are meticulous, detail-oriented and who genuinely recognize the problem first are in minority
Perhaps the biggest problems in(/with) technology today aren't of a technical nature. If even somewhat true, then those who are "meticulously and genuinely recognizing the problems" aren't necessarily even programmers.
[0]: This is hard/impossible to know one way or the other due to survivorship bias. A lot of software from the 70s is just gone.
[1]: Instances of this are hard to even find, naturally. Squeaky wheel gets the grease and all that.
[2]: Hardly exclusive to IT, but still a significant factor.
As someone who once was part of an outsourced team maintaining a COBOL system, I can attest to the fact that people wrote shit then, and will continue to write shit. I was trying to debug and figure what the hell was going on in some especially complex logic and found the author's comment at the top of the file with the date "1983". I put my resignation papers 2 weeks later and moved on to JS, Python and systems design
That wasn't my understanding of the past. I had the perspective that early programmers were basically cowboys in the wild west, people from a variety of backgrounds that were just in the right place and took on the projects.
As an uneducated farmer this gave me a giggle, it suggests if I filed a ticket about a performance problem in the project it'd likely be fixed instantaneously. That quote is basically an advert, if the author wishes to establish such a high bar for themselves it'd be foolish not to avail of it.
To be fair to the author, that quote is part of a longer passage about how one should not optimize for free form ascii first when fixed size buffers are an option. At least that how I read it, it's clearly opinionated writing.
Gatekeeping implies that this author actually has something he can keep from you, like universities gatekeeping a certificate for intellectual credibility, or OpenAI saying that they have information that is too dangerous for the world. The personal refusal to acknowledge anyone for anything is only gatekeeping insofar as the public ought have some moral prerogative to that person's acknowledgment.
Why is this one author's acknowledgment so interesting that we should have a debate over it? In the face of all priorities, does this really bubble up to the top?
I find it interesting that everybody in this thread seems to take exception to the comparison to "farmers". I mean, farming is brutal hard work but it also does not require a high degree of thinking, so I can see his point even if he has made it in an offensive way. Personally, I'm much happier sitting in a warm comfortable place thinking hard about code instead of being out in a cold field 20 hours a day, but meh.
Much more offensive (to me) is this gem from later on:
> Designing clever, binary and minimally repetitive protocols saves enormous amounts of CPU-time otherwise lost to compression.
Well yes, but also, and more importantly no. This is technically true - not compressing the data is more efficient in terms of computation per bit on the wire. But it is also wrong in a much more subtle way.
The advantage of using JSON (or an equivalent) and then compressing it down is that we inject a large amount of redundancy into the format. The CPU is taking that redundancy back out for us to save bandwidth, but we get to exploit it in much more valuable ways. The encoder/decoder are generic, simple and highly optimized. We can use them everywhere and trust them to work. This has value in itself, and it means that we can spend our limited budget of programmer time in optimizing other code where the payoff is higher. It means less time tracking down bugs in "super 10x programmer's one-off encoder that was only written for this specific application". And lastly, but my favourite - it means that when something goes wrong we have a hope of repairing the damage to the data. Because redundancy gives us options for error checking and recovery, while optimizing the binary format generally does not.
Also, if you want the format to really be optimal it should not be designed as byte streams (as low-level binary formats generally are) and instead should be a stream of variable-sized tokens that are fed through an arithmetic encoder.
The first half of this week I spent repairing data corruption caused by some of my code. Luckily there is a lot of redundancy in how I encoded the data (in a structure that is described in JSON inside the records) so I could write a tool that repaired the damage. I guess I'm lucky that I'm just a "farmer" :)
I mean you can be critical about performance and lack of optimization without punching down. People like this have never worked on applications in a normal corporation, I'm sure.
hehe yeah, though it doesn't have to be a bad thing. There are quite a few brilliant mavericks that never have and never would be able to fair well doing that -it would crush their soul or drive them mad. They just do what they love and everyone's better off for it. I think in time they tend to stop being permanently irritated/critical/disgusted about these things, and learn to accept and ignore -for their own sanity.
> We have had standardized, time-zone neutral representation of time in binary, efficient, 4-byte representation since the 1970s. [...] This is just an example of how we have regressed in our algorithmic thinking.
Now I'm wondering if uWebSockets will fail spectacularly in 2038.
"uneducated farmer" are those who can survive on a piece of land dependency-free. They know weather, season, plant, animal, chemistry, machinery and many more. Us engineers without 30mins of internet are less useful than a sprouting potato. I'm not sure what the author meant.
Programming wasn't an elites task back then lol. I guess I can't disprove his statement about being peasants now, but they are completely wrong about the 1970s.
It sure does not sound like an elite career to me. It was just regular people working with the constraints they had at the time, not because of some super intelligence or leet genius focus on perfection.
If anything, projecting yourself (not you in particular!) into some mythical lost elite, as opposed to the current stupid pleb sounds like pure cope imo.
It is pure cope, whilst software has definitely diluted in quality with the lower barrier to entry... that barrier was never some IQ gate it was mostly socio-economic.
There is a very valid case to be made for the current state of performance but back then if it wasn't optimized for the resource constraints it didn't run -- nowadays software is optimized for shipping quickly so of course it sucks once the buzzword paint falls off.
We're comparing buy-once bespoke boots vs. assembly line fad-of-the-month trainers. The idea that either was the result of some 'giga brain' creators and not market influences is delusional.
Lol, easy to nitpick but if you'll read the whole document this guy really knows his stuff. It's not a coincidence that uWebSockets is continuously being used by most trading platforms.
"It's not bragging if you can back it up." - Muhammad Ali
I want the connections to be distributed to separate instances of a WS server based on modulo-based or consistent hashing algorithm... above that in the stack we have a message broker or queue partitioned using the same sharding algorithm for the number of websocket servers running. This queue/message broker sends publishes messages to each web socket service which in-turn pushes the message to the client. This is how I want to handle horizontally scalable web socket connections...
The question I have is the best way to do this and the best way to migrate connections. From one service to another when/if the replication factor of web socket servers changes (we add/remove servers)...
One way I would think would be to send a disconnect message to the websocket service when this event happens and it would disconnect it's clients and then have a random delay for when re-connection happens... so as to avoid trying to reinitialize all of those connections at once. This seems sub-optimal as with consistent-hashing many of the connections will stay on the same server, and terminating those connections would be unnecessary.
What is the best way to migrate an open TCP from one server to another? Or if the connection must be closed... how can we minimize the number of connections that are to be closed.
BTW: I know I can broadcasts to all instances of the websocket server, and then unless that server holds the connection it is no-op... if the server does have a connection then send the message back... This also seems it would have a limit to how far this scaling-strategy would grow before running into limitations.