Imagine a neural network, whose parameters (like number of layers, epochs of training, numbers of neurons, ...) can be specified as arguments. You don't know where the optimum is (say, the point where the least errors are made, which you can calculate from the training and testdata).

I'm working on a program that tries that randomly at first and then only in the areas that look like they may be where the minimum is. This works Ok, but doesn't guarantee the best results.

Is it possible to "convert" a program into a large function like $$ f(x_1, x_2, x_3, ...) = \mathrm{loss} $$ and find the minimal-loss (the minimum of the function) computationally?

The number of arguments would be huge (all parameters for the net and all training and testdata), but still, would that be an easier-to-calculate way than just trial-and-error? The number of basic operations is clearly very limited (AND, XOR, ...).

If so, how to do that?

  • It can be converted and that's basically what you did. The question is whether the exact global minimum of this function can be found computationally. Well, the answer is negative in general as long as you have continuous arguments. For functions satisfying certain properties like being convex and smooth for all its parameters this global minimum can be approximated, but this is unlikely your case. – Mikhail Berlinkov Dec 3 at 21:34

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.