The training data was so large that I don’t think a single epoch was used. I.e. it’s not a guarantee that any specific site is known by GPT-3 just because it’s in the data.

But, with the nature of random samples, it’s probably likely that it saw at least a little bit of SO.