As a previous comment said, this sounds like a generalization of regression testing. This article seems to focus on the small scale and omits a couple points relevant to larger scale systems:
- data dependencies cause regressions, so you. need some way to factor this in
- production changes cause regressions
- debugging at the source code level can be laborious past a certain point
The way I approached doing this at another (maybe same) T$ compnay was tooling that could take anything observable -- e.g. logs, program traces, signals from services, service definitions, data, stack traces from debuggers for side-by-side running -- convert it to a normal form (e.g. protobuf or JSON) and then diff that to look for regressions.
- data dependencies cause regressions, so you. need some way to factor this in - production changes cause regressions - debugging at the source code level can be laborious past a certain point
The way I approached doing this at another (maybe same) T$ compnay was tooling that could take anything observable -- e.g. logs, program traces, signals from services, service definitions, data, stack traces from debuggers for side-by-side running -- convert it to a normal form (e.g. protobuf or JSON) and then diff that to look for regressions.