# Find optimum of a neural network computationally

Imagine a neural network, whose parameters (like number of layers, epochs of training, numbers of neurons, ...) can be specified as arguments. You don't know where the optimum is (say, the point where the least errors are made, which you can calculate from the training and testdata).

I'm working on a program that tries that randomly at first and then only in the areas that look like they may be where the minimum is. This works Ok, but doesn't guarantee the best results.

Is it possible to "convert" a program into a large function like $$f(x_1, x_2, x_3, ...) = \mathrm{loss}$$ and find the minimal-loss (the minimum of the function) computationally?

The number of arguments would be huge (all parameters for the net and all training and testdata), but still, would that be an easier-to-calculate way than just trial-and-error? The number of basic operations is clearly very limited (AND, XOR, ...).

If so, how to do that?

• It can be converted and that's basically what you did. The question is whether the exact global minimum of this function can be found computationally. Well, the answer is negative in general as long as you have continuous arguments. For functions satisfying certain properties like being convex and smooth for all its parameters this global minimum can be approximated, but this is unlikely your case. – Mikhail Berlinkov Dec 3 at 21:34