2
$\begingroup$

(Feel free to suggest a better title) (same with tags, none of relevant tags exist yet, and I only have 101)

Let's say I have a perceptron with an optimal number of layers and optimal number of neurons, which I have taught to recognize an alphabet which was printed in a certain font, with the same settings and resolution. Now if I feed it a symbol from that set, but say, printed in another (but similar) font, or with the same font, but rotated 5 degrees, or shifted 2 pixels to the left, or scaled a 5 percent, and so on, will this system recognize this modified symbol, or at least have an easier time learning to recognize this symbol? (than a system without any prior learning)

Basically, what is the point of back-propagating neural networks? Is it 1) a way to recognize shapes, and effectively filter noise in images of those shapes, in form of different rotations, scales and other disortions or 2) just a relatively simple and fast way to recognize a limited number of bitmaps of those shapes, and not being able to recognize the shape in another bitmap if that particular bitmap was never taught to the network.

$\endgroup$
  • $\begingroup$ The question is probably more suitable for MetaOptimize Q&A, you can find a link in the FAQ. $\endgroup$ – Kaveh Jun 26 '11 at 0:47
1
$\begingroup$

I would be very surprised if a backpropagation-trained multilayer FFN spontaneously learned concepts of shape. It's not that the notion of "shape" isn't representable by a FFN (given enough layers, anything is). It may even be the case that the globally optimal configuration does capture something about shape. But there must be a strong local minimum about the exact-image replicator, meaning that typical backpropagation (i.e., gradient descent) algorithms will get stuch in such local minima. Unless you've specifically primed your FFN to learn shape, I would be very surprised if it did.

$\endgroup$
  • $\begingroup$ So it is possible to teach it shape by feeding different images of the same shape? (I know, in my original setup I only fed it 1 image per shape) $\endgroup$ – Cray Jun 26 '11 at 14:31
  • $\begingroup$ In the connectionist (as opposed to symbolic or rule-based) framework, it's hard to say whether or not the system has "learned shape". The only test is tautological: see if it manages to recognize novel shapes of previously seen letters. Your idea of training on different images of the same letter sounds like it should work (I believe that's how they trained the zipcode reader FFN). $\endgroup$ – Aryeh Jun 27 '11 at 6:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.