Folks, this is a masterclass in how not to give an apology.

1. Apology (great, I hope they end it here) 2. Shift the blame slightly (uh oh) 3. Attempt to make him feel bad (yikes) 4. Question the initial incentive (big yikes) 5. Full conspiracy (big big yikes)

Extremely manipulative and weak feeling.

From this/other work, Hamilton seems like an all-or-nothing -- E.G. you really have to buy into the paradigm. Do you see value in partial migration? How might one get started without boiling the ocean at their company?

In short. It's not all-or-nothing.

Hamilton is just a python library at its core, unlike something like Airflow or Prefect, there is no system to set up. All it requires is some python code to be written which would replace some procedural code. The scope of which, is up to you. But yes, you can piece-meal migration quite easily to get your toes wet and gain conviction/win over hearts and minds that way (if that's a concern).

Also here's the ticket we have on lineage -- would love input if you have any! https://github.com/stitchfix/hamilton/issues/42

Thanks!

1) Not limited to pd.Series -- any type works. So if you have a dataset or a way of expressing types it should just work. That said, pd.Series/DataFrames are convenient, and we have some special tooling around them. We're looking at tooling to add more expressiveness to datasets. Something like koalas type-hinting (there are a few approaches) might be helpful in this: https://koalas.readthedocs.io/en/latest/user_guide/typehints.... With the type-equivalency check we added this should be fairly extensible.

2) Yes! Although not too extensively. I think OpenLineage would be a pretty simple thing to add -- just have to get the abstractions right.