A cellular neural network is a kind of recurrent neural network that could be thought of as a hybrid between neural nets and cellular automata.

As I understand it, the classic Chua-Yang CNN is limited to a fairly small set of functions, but addition of a second layer or a different choice of nonlinearity expands the function set to all computable. This sounds very similar to traditional neural networks, where Kolmogorov's 1957 theorem says that a two-layer net can represent any continuous function arbitrariliy close.

However, there is also the significant caveat that traditional shallow neural nets are very inefficient at representing complicated functions. Do universal CNNs have a similar weakness? Do they require excessively large network sizes or take too long to converge when used for complicated function computation?

  • $\begingroup$ This isn't a complete answer, but if you think of neural networks in terms of information theory; there will always be functions that require very little information to emulate and functions that require lots of information to emulate. The task is to make the functions that you are likely to see require as little information as possible. $\endgroup$ – dan_waterworth Feb 24 '12 at 7:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.