Nomad is awesome and works at scale. The engineers continue to battle harden it and it’s a joy to work with. You do have to manage things like service discovery (usually with consul) and traffic routing separately - but the integration with vault is sublime.

About the only real negative of Nomad is that it doesn’t have the mindshare that k8s does, so you don’t see the amount of developer engagement in extending it the way you do in the k8s SIGs. Also, being an expert in Nomad doesn’t give you the same number of career opportunities, and on the other side - there aren’t umpteen thousand nomad SREs the way there are with k8s - so getting someone up to speed can take a couple months (but this system is very well defined, well documented, and small enough that any half talented engineer can master it very quickly)

Nomad does have the very important advantage that Hashicorp stands behind the product - so if anything goes awry, you’ve got a support team and escalation that will jump on and root cause/resolve any issue, usually within a matter of hours and even in the really squirrelly cases (that you are only likely to see when when you are managing many, many thousands of nodes in a cluster) within days.

> You do have to manage things like service discovery (usually with consul)

Since the last couple of versions there has been native Service Discovery in Nomad which works pretty well.

My personal experience is that Consul and vault are too complicated to fiddle it. I can spin a kubernetes cluster in minutes but I gave up on trying a Consul+Vault+Nomad lab around all the setup steps for replication, RBAC and whatnot.

Less career opportunities need not be bad if they are better payed and the companies are more accommodating.

Swarm isn't dead: Mirantis still puts engineering resources into it and has plenty of businesses relying on it. Bunch of new features just released https://www.mirantis.com/blog/announcing-the-23-0-major-rele...

k8s is good, I don't get the hate against k8s.

k8s is good, but complex, and thus not fit for everyone and everything, but gets overused to death due to being the most popular choice, thus you have a lot of people that dislike it due to bad experience.

There's an absolutely massive ecosystem of tooling around it to make a developer's life easier and abstract away the confusing parts. No one needs to write K8s manifests directly if they don't want to.

The abstractions work very well, until they don't and you need to dive in 5 abstractions deep to debug what's happening, or an upgrade needs to be done and you have conflicting dependencies a few levels deep.

Even during development I’ve had to learn far more than I’d like about k8s internals as things broke left and right.

ECS, nomad or even autoscaling VMs are much easier to deal with when they fail.

But isn't this the case with everything? I feel there is a necessary level of complexity to everything. Things can't be made simpler beyond a level.

YAML spaghetti, plus a complexity that makes Java application servers look like toys.