It seems like this would do well on Python code where type errors are relatively rare, and the developers' attitude towards types has been basically sane for the history of the project. Furthermore, this looks like it's going to work best if you have a lot of pure functions. If these things are true you're already doing pretty well! I'd say this is a tool to make already-exemplary Python code even better.
However, my main use for this so far has been for old python 1.5-2 programs (some 15 years old) with not much typing at all. Adding limited type information for methods seems useful for helping the static checkers as well in these cases. The division errors in the blog post were based on real bugs it found inside class methods for example. I bet there are lots of corner cases I haven't tripped over yet however. I wouldn't be surprised with dynamic python that it could cause quite a lot of problems. Python can get pretty fancy. I told it to ignore all 'self' variables for example.
I think basing it on the coverage tool helped avoid a lot of problems. Because they have already done the hard parts of getting the tracing to only happen on user code. If you don't trace into libraries, and only do user code the task becomes 7 million times easier.
We found that auto-documentation via these kinds of run-time traces is really powerful. If you add in a bit of centralization (connecting traces across many users), this can also be a useful way to capture bugs or unforeseen behavior.
Very interesting! I don't think the paper mentions type tracing combined with static analysis, property testing, or AOT compilers however. Do you have another link with more info? I can't seem to find it anywhere, and am keen to have play with it :)
It does seem like there's a lot more very useful tools which could be made from the data we already have. Bug databases combined with commits are also filled with useful information. It does seem you can train a classifier for certain bugs from just those features.
I think the "Auto Complete from stack overflow" and related stuff from early 2016 was also pretty neat, in that it didn't use trace info, but more normal text search methods. But was surprisingly useful none the less.
