9
$\begingroup$

Following up on this question...

I am attempting to learn how to use and create neural networks for my research, and one point is somewhat escaping me. I realize that hidden layers are a somewhat necessary portion of this, however I am stuck on two points which my references aren't explaining to my satisfaction:

  1. What is the exact purpose of the hidden layer?
  2. How does one determine how many hidden layers to use?

From what I gather, it is to "model" real world functionality, but if possible I'd like a bit more of an explanation.

Improve this question
$\endgroup$
1
  • $\begingroup$ but some time i face that if you increasde hidden layers then the overfitting problem comes so its very difficult to say how many hidden layers we can use ? may be it is hit and trial method. $\endgroup$
    – user20835
    Dec 25, 2013 at 11:18

2 Answers 2

Reset to default
9
$\begingroup$

A hidden layer is used to increase the expressiveness of the network. It allows the network to represent more complex models than possible without the hidden layer.

Choosing the number of hidden layers, or more generally choosing your network architecture including the number of hidden units in hidden layers as well, are decisions that should be based on your training and cross-validation data. You should train the network with a set amount of nodes (to start, try one hidden layer, with one unit per input unit) and test the model.

See this link for more help: http://www.faqs.org/faqs/ai-faq/neural-nets/part3/section-10.html

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ that link is quite helpful $\endgroup$
    – erik
    Jan 3, 2012 at 21:01
6
$\begingroup$

I assume we're discussing simple feed-forward neural nets, i.e. multi-layer perceptrons.

  1. The hidden layer is necessary to capture non-linear dependencies between your data's features and the variable you're trying to predict. If you don't use a hidden layer, you might as well use linear regression (for regression) or logistic regression (for classification).
  2. By trying various numbers of hidden layers and evaluating how well they work, e.g. in a cross-validation setting. Commonly, one hidden layer will be enough and NN performance is optimized by varying its size and the regularization.

Note that with more than two hidden layers, you're in deep learning land and you probably need custom algorithms to train your net. The reason is that vanilla backpropagation suffers from the "vanishing gradient" problem in deep nets: the gradient of the error function dies down at the layers close to the input, and those layers will hardly be trained.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.