> What is the general pattern of usage with Docker containers? Are they supposed to isolate just one application and do IO via a networks socket?
The maintainers of Docker appear to strongly believe that 1 application per container is the way to go.
At sufficient scale, they are pretty much correct. You grab Y bare metal hosts and spin up X docker containers for X processes.
> Isn't this just pushing the problems into managing dependencies between more little VMs while also constraining the architecture? It increases the difficulty of synchronizing 10 or 20 separate isolated applications start,upgrade,fail. Maybe all that living on yet another VM guest machine (so conceptually having 2 virtualization levels). Handling more complicated network setup (bridging, firewall rules at multiple levels). Handling effects of disk and other subsystem interacting with each other in strange, sometimes sub-optimal ways.
A handful of people advocate for immutable role-based containers [e.g. https://devopsu.com/blog/docker-misconceptions/] for that reason. In that use case, its really replacing running something like Xen + Chef, KVM + Ansible, or whatever.
You grab a host machine, you stand up your X containers with Z processes per container on Y hosts.
I think this is really equivalent to the Microservices vs. Normal SOA vs. Monolithic argument. Given sufficient scale/requirements, each makes sense. However, none of them are optimal for all situations.
If you really do have a container per process, I'd suggest you check out OSv from Cloudius Systems. If a single address space is all you need and you're the only process using the network space, you can make a lot of optimizations and assumptions that aren't being made in a Docker instance. They were able to get better performance running memcached inside OSv inside KVM on Linux on bare metal, then they did running memcached in Linux on bare metal.
> At sufficient scale, they are pretty much correct. You grab Y bare metal hosts and spin up X docker containers for X processes.
Can you elaborate why they are correct with this view? What is inherently superior about 1 process per container? What is wrong with having two cooperating processes per container? And why should Docker containers only run on bare metal?
> And why should Docker containers only run on bare metal?
The "point" of Docker is to be a lightweight replacement for virtualization in some cases. Layering Docker on top of virtualization, while fine, adds layers rather than removing/replacing them.
The overall idea is to apply the principle of least privilege. If each process run in its own set of namespaces, and only can only access exactly what it is meant to, then the container configuration is explicit documentation of dependencies, and you also reduce attack surfaces.
As for running on bare metal, I don't think the implication is they always better, but that if your goal is huge scale minimizing the number of layers is better for efficiency (though not necessarily better for other things - e.g. you may want to group/sandbox some containers in VMs for increased security)
> Can you elaborate why they are correct with this view? What is inherently superior about 1 process per container?
At sufficient scale, horizontally scaling is complicated and involves implementing the things I avoid until I have no choice [Dynamic Service Discovery on a per-process basis, etc.]. I'm mostly deferring to stuff I've read about it since I haven't really worked at companies that are larger than $XX million a year.
It generally is things like principle of least privilege, the fact you already have to bite the bullet on managing your service discovery through yet another tool, etc.
Personally, I've never dealt with anything larger than 2-3 datacenter locations with a rack of equipment in each.
> And why should Docker containers only run on bare metal?
That you can just ignore.
I really just meant don't do:
bare metal -> Spin up Xen VM -> Spin up docker containers
The maintainers of Docker appear to strongly believe that 1 application per container is the way to go.
At sufficient scale, they are pretty much correct. You grab Y bare metal hosts and spin up X docker containers for X processes.
> Isn't this just pushing the problems into managing dependencies between more little VMs while also constraining the architecture? It increases the difficulty of synchronizing 10 or 20 separate isolated applications start,upgrade,fail. Maybe all that living on yet another VM guest machine (so conceptually having 2 virtualization levels). Handling more complicated network setup (bridging, firewall rules at multiple levels). Handling effects of disk and other subsystem interacting with each other in strange, sometimes sub-optimal ways.
A handful of people advocate for immutable role-based containers [e.g. https://devopsu.com/blog/docker-misconceptions/] for that reason. In that use case, its really replacing running something like Xen + Chef, KVM + Ansible, or whatever.
You grab a host machine, you stand up your X containers with Z processes per container on Y hosts.
I think this is really equivalent to the Microservices vs. Normal SOA vs. Monolithic argument. Given sufficient scale/requirements, each makes sense. However, none of them are optimal for all situations.