2
$\begingroup$

Has any research been done about / are people interested in incorporating prior knowledge of good high-level features into a deep neural net?

I know this somewhat defeats the purpose of feature learning but consider the following applied scenario: image classification task, 20 classes, 'small' training set (10k), expert human knowledge of high-level features.

Ideal would be to compensate for lack of training cases by incorporating knowledge of these features into the architecture (or via other means). How?

$\endgroup$
2
$\begingroup$

This 2013 paper integrates deep learning models with structured hierarchical Bayesian models, for the net to learn novel concepts from very few training examples. It shows encouraging results.

Another approach is to import a general-purpose top-performing neural net, and replace the top fully connected layers with initialised ones (including an output layer that corresponds to the number of classes in your specific task). OverFeat is an example, for image classification - it was trained on ImageNet.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.