Is there a corresponding result that gives the number of examples needed to provide a sufficient training set for a given physical phenomenon? I’m imagining a high-dimensional equivalent of Nyquist’s sampling theorem.

Coupled with this result, we’d then have a reasonable estimator of the network size required for particular tasks before even starting the data collection.