It's not enough to intercept `set search_path`, in fact any arbitrary function could change it, but Postgres doesn't report it back, unlike some other gucs.
Hmm, yes. ParameterStatus is not sent when search_path is changed. But I think it would be trivial to patch PostgreSQL to send it.
I bet one even can push such patch through commitfest.
Patroni was openly developed from day 0. The repo was created in early July 2015, but before that it was living as the fork of Compose Governor.
The Spilo docker container which packages PostgreSQL + Patroni was also always publicly available from the first day. I agree, the Spilo is a bit opinionated due to the way how it is used at Zalando.
These two projects have quite a long history and originally weren't even targeted to be deployed on K8s because back in 2015 K8s wasn't absolutely suited for running stateful workloads.
The most opinionated one is Zalando Postgres-Operator, and yes, first it was the internal tooling and was solving our specific problems. Now I would also argue that with such an amount of external contributors it already became a way more general solution.
After all, Patroni is so general, that we call it a template for PostgreSQL HA. You can take it and build something that you need/want without relying on Spilo and Zalando Postgres-Operator. Speaking of Patroni@K8s, there are already two very nice examples: Crunchy Data PostgreSQL Operator and StackGres are both relying on Patroni for running PostgreSQL HA on K8s.
Hi Cyber - yes I know it has a very active community, and you and I have interacted on github before. But I find this response very similar to github: it’s a template, so build whatever you like with it. That’s not what most people want, and that kind of a reply reinforces my comment. I don’t use React because I want a template for for building a reactive application. I use it because it does all the things I don’t want to have to be concerned with. Like I said, I don’t think any project needs defense. I thank you all for the work you’ve done. But it is not the easiest bit of kit nor the most helpful/inclusive community.
It looks like you are talking here about postgres-operator by Crunchy
Yeah, crunchy was a little faster with releasing it than Zalando, but try you look closer how they deploy postgres on kubernetes.
Somehow it feels that they are trying to map 1 to 1 the same approach how folks used to run postgres on bare metal. That is: deploy master pod, wait until it's up and running, deploy a replica pod, and so on...
It doesn't really look cloud-k8s-native.
In my opinion such deployment should look absolutely different. You just need to deploy k8s manifest, which will create Secrets, StatefulSet and Service which will be used to connect to the master.
The rest should happen automatically:
* StatefulSet will start N pods with postgres
* pods (Patroni) will elect leader.
* elected leader will initialize (initdb) a new cluster
* all other pods will get basebackup from the leader and become replicas
* if the master(leader) pod die - other pods will elect a new leader
* StatefulSet will start a replacement of failing pod and it will join the cluster as a new replica
And more important all it should happen without connection to Etcd, ZooKeeper or Consul. It should just use Kubernetes API.
> I currently prefer Stolon over Patroni because statically typed languages make it easier to have less bugs (Stolon is Go, Patroni is Python)
Sounds like a holy-war topic :)
But lets be serious. How statically typed language helps you to avoid bugs in algorithms you implement? The rest is about proper testing.
> and because the proxy it brings out of the box makes it convenient: On any machine I connect to localhost:5432 to get to postgres
It seems like you are running a single database cluster. When you'll have to run and support hundreds of them you will change your mind.
> if the Postgres fails over, it ensures to disconnect me so that I'm not accidentally connected to a replica.
HAProxy will do absolutely the same.
> Bugs. While Stolon works great most of the time, every couple months I get some weird failure. In one case it was that a stolon-keeper would refuse to come back up with an error message, in another that a failover didn't happen, in a third that Consul stopped working (I suspect a Consul bug, the create-session endpoint hung even when used via plain curl) and as a result some stale Stolon state accidentally accumulated in the Consul KV store, with entries existing that should not be there and thus Stolon refusing to start correctly.
Yeah, it proves one more time:
* don't reinvent wheel: HAProxy vs stolon-proxy
* using statically typed language doesn't really help you to have less bugs
> I suspect that, as with other distributed systems that are intrinsically hard to get right, the best way to get rid of these bugs is if more people use Stolon.
As I've already told before. We are running a few hundred Patroni clusters with etcd and a few dozen with ZooKeeper. Never had such strange problems.
> > if the Postgres fails over, it ensures to disconnect me so that I'm not accidentally connected to a replica.
> HAProxy will do absolutely the same.
well I think that is not the same what stolon-proxy actually provides.
(actually I use patroni)
but if your network gets split and you end up with two masters (one application writes to the old master) there would be a problem if one application would still be connected to the splitted master.
however I do not get the point, because etcd / consul would not allow to still hold the master role which means that the splitted master would lose the master role and thus either die, because it can not connect to the new master or just be a read slave and the application would than probably throw errors if users are still connected to the splitted application.
highly depends how big your etcd/consul is and how good your application detects failures.
(since we are highly dependent on our database we actually kill hikaricp (java) in case of too many master write failures and just restart it after a special amount of time.
well we also look in creating a small lightweight async driver based on akka, where we do this in a little bit more automated fashion.)
> well I think that is not the same what stolon-proxy actually provides. (actually I use patroni) but if your network gets split and you end up with two masters (one application writes to the old master) there would be a problem if one application would still be connected to the splitted master.
On network partition Patroni will not be able to update leader key in Etcd and therefore restart postgres in read-only mode (create recovery.conf and restart). No writes will be possible.
Patroni has REST API providing a health-checks for load-balancers. patroni:8008/replica will respond with http status code 200 only if node is running as replica.
In the HAProxy config file you just need to list all Patroni nodes, specify a health-check and it will do a load-balancing for you.
Correct. Writes need to be directed to master. May be not only writes but some reads as well. Only application can know which statement can be executed on replicas and which must be executed on master.
Compose guys are kind of happy with existing functionality of Governor and didn't really wanted to add something new. That was the main reason of making a fork.
If you read a further comments to this news you can get familiar with the part of new functionality of Patroni comparing to Governor.
> Stolon is quite comparable to Patroni and has more features than Patron.
Let me as a question, what feature Stolon has and Patroni doesn't? Can you give an example?
From my side I can provide list of features available in Patroni, but not in Stolon.
For example:
* there is no way to do a controlled failover (switchover) in Stolon.
* in Patroni it's possible to exclude some nodes from a leader race.
* Patroni supports cascading replication
* Patroni can take basebackup from replicas if they are marked with a special tag
* it's even possible to configure Patroni to use a custom backup/recovery solution instead of pg_basebackup
* Patroni can give you a hint that postgres must be restarted to apply some configuration changes
* with Patroni it's even possible to schedule switchover or restart restart of postges on some specific time (for example 04:00 am, when traffic is minimal)
* with Patroni you can control High-Availability / Durability ratio. I.e.: what to do if master postgres has crashed? Either you will try to start it back or failover to a replica. But start of a crashed postgres may take a long time. With Patroni you can configure that if postgres didn't started in some amount of seconds - please do failover.
Stolon provides cloud-native deployment and Patroni doesn't?
This is not really true. The main idea of Patroni is to be not dependent on some specific technologies. It can work as on bare-metal with the same success as on Kubernetes.
It's very easy to build a custom solution with Patroni for your use-case.
For example there is a Spilo project: https://github.com/zalando/spilo, a docker image build with Patroni and wal-e for a cloud deployments.
Thanks for this. As I mentioned in previous comment, I couldn't find much about Stolon.
One of things I likee in stolon is the proxy, it enforce connections to the right PostgreSQL master and forcibly closes connections to unelected masters. Which is not present in Patroni, but I guess it should be doable using consul's service discovery and dynamic DNS. (I am not sure if this can be done using etcd and zookeper).
Yes, one can reimplement it in Python, but don't forget:
// This implementation does not guarantee that only one client is acting as a
// leader (a.k.a. fencing). A client observes timestamps captured locally to
// infer the state of the leader election. Thus the implementation is tolerant
// to arbitrary clock skew, but is not tolerant to arbitrary clock skew rate.
