I'm new in this community and I don't know whether my question is proper for this community. I will delete this post if it is not proper.
I'm interested in deep learning network models and have a question about this.
Suppose we have a feedforward neural network models with different hyperparameters (for example, the number of layers, neurons and so on), say $H_1,..., H_n$. Also assume all the samples are selected from a particular distribution.
First we got $100$ samples. If we find that a typical hyperparameter set, $H_i$, performs best. Now I want to increase the number of samples to $1000, 10000,$ even millions. Is $H_i$ still the best hyperparameter set to perform among $H_i$'s? I want to know whether there are references about such topics. The best is a mathematical proof about this fact.
Again, please tell me if this question is not proper to the community. Thanks a lot!