Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [ne.neural-evol]

Theoretical questions in Neural and Evolutionary Computing

0
votes
0answers
10 views

Back-propagation for computing derivative of certain line integral

Consider a function F (think of neural networks) with two sets of parameters: (1) model parameters $\mathbf{w}$, and (2) input data ${\bf x} \in {\mathbb R}^d$. Fix $i \in [d]$, consider the following ...
-1
votes
0answers
45 views

Find optimum of a neural network computationally

Imagine a neural network, whose parameters (like number of layers, epochs of training, numbers of neurons, ...) can be specified as arguments. You don't know where the optimum is (say, the point where ...
9
votes
5answers
634 views

Can neural networks be used to devise algorithms?

After the newer and newer successes of neural networks in playing board games, one feels that the next goal we set could be something more useful than beating humans in Starcraft. More precisely, I ...
0
votes
0answers
53 views

Rules of thumb for universal approximation

A neural network can function as an universal approximator for a function $f$, according to the universal approximation theorem (original paper). In this answer, I read that "As a rule of thumb, each ...
4
votes
0answers
941 views

Universal Approximation Theorem for non-sigmoidal activation functions

The most cited Universal Approximation Theories for multi-layer feedforward neural networks by Cybenko (1989) and Hornik (1991) assume the activation functions of the network to be sigmoidal. However, ...
4
votes
0answers
114 views

Justifying the state of virtual memory as a vector space

First, I'm mostly experienced with Math, which I hope won't be too inconvenient. I saw Operational Calculus on Programming Spaces by Sajovic and Vuk, which seemed very interesting to me (for a "short ...
2
votes
0answers
80 views

How does white noise on the output channels influence the training of a neural network?

Let $\rho(\sigma): \mathbb{R} \rightarrow \mathbb{R}$ be a probability density that is parametrized by a parameter vector $\sigma \in \mathbb{R}^s$ (for example the normal distribution where $s = 1$ ...
0
votes
1answer
290 views

Serial and parallel neural network [closed]

The question is: can exist a parallel or serial neural network or someone talked about this? For explanation, in the network a single record in a data set enter in the input layer as a value between ...
1
vote
1answer
147 views

Are biases necessary to make neural networks universal approximators when using sigmoid activations?

In a neural network, a bias is a constant term that is added to the weighted input in a neuron/unit: output = activation_function( input1*weight1 + ... + inputn*weightn + bias) I can see that the ...
4
votes
1answer
78 views

Data Mining of self-replicators

My current (very limited) understanding of the creative process that leads to the design of self-replicators is that any particular self-replicator, like Universal Constructor, Langton's loop or ...
1
vote
0answers
35 views

Why does there always need to be a direct crossover between parents and children in real valued GAs?

I have just been thinking about the simulated binary crossover (SBX) operator used in the NSGA-II algorithm and other real-valued genetic algorithms; and i am wondering if there is any reason that ...
2
votes
0answers
47 views

second order regularisation on a neurofuzzy network with Bernstein basis functions

We're trying to build a neural network that uses a neurofuzzy approach. Our reference is the book Adaptive Modelling, Estimation and Fusion from Data: A Neurofuzzy Approach by Chris Harris, Xia Hong, ...
4
votes
2answers
831 views

Are single hidden-layered neural networks at least as good as multi hidden-layered neural networks?

If I have a multi hidden-layered neural network that is getting a better approximation for a function than a single one, does that mean that there is something "fishy" about my multi layered one ...
1
vote
0answers
148 views

How to define deep learning? [closed]

Ive read some articles about deep learning but I found its hard to provide a clear definition of deep learning. For me its like an intelligent feature selection method. But it seems that its not ...
1
vote
2answers
152 views

Theoretically, can everyday computing tasks be broken down into ones solvable by a neural network?

MIT Review recently published this article about a chip from IBM, which is more or less a Artificial neural network. Why IBM’s New Brainlike Chip May Be “Historic” | MIT Technology Review The article ...
2
votes
1answer
70 views

Neural Networks: Incorporate feature-specific knowledge

Has any research been done about / are people interested in incorporating prior knowledge of good high-level features into a deep neural net? I know this somewhat defeats the purpose of feature ...
4
votes
0answers
95 views

Greedy Backpropagation: has anyone researched it?

Backpropagation is computationally expensive. Has any research been done on a partially greedy implementation of it? Intuition: at the beginning of training, big rough learning steps can be taken, so ...
6
votes
3answers
694 views

Neural Networks: what's the point of learning features that don't linearly separate?

Unless I'm mistaken, deep neural networks are good for learning functions that are nonlinear in the input. In such cases, the input set is linearly inseparable, so the optimisation problem that ...
2
votes
3answers
425 views

Which algorithms have been proposed to learn the architecture of a deep neural network?

Yoshua Benhgio's Learning Deep Architectures for AI book mentions that we should [...] strive to develop learning algorithms that use the data to determine the depth of the final architecture. ...
3
votes
1answer
544 views

Using Indicator Functions as Transfer Functions for Neural Networks

Does there exist any theory (other than Cybenko's proof of the Universal Approximation Theorem with sigmoids) advocating the use of indicator functions as transfer functions for machine learning with ...
4
votes
0answers
464 views

Practical Implications of Kolmogorov's Result on the Universal Approximation Theorem with Neural Networks

After having read matus's beautiful answer in this thread explaining (among other things) Kolmogorov's result regarding the Universal Approximation Theorem with Neural Networks, I wonder: if just $\...
1
vote
0answers
197 views

Generalization Issues with Practical Suggestions from Universal Approximation Theorem with Neural Networks

After having read matus's beautiful answer in this thread explaining (among other things) Cybenko's proof of the Universal Approximation Theorem for Neural Networks, I wonder: if we use a piecewise ...
1
vote
0answers
98 views

Why do people like using evolution computing techniques like GA on multi-objective optimisation?

I am new to the field of multi-objective optimisation and I try to find some books to read on this topic. Yet when I search around my library I found there are many books on using evolution computing ...
1
vote
0answers
54 views

Bound on the number of hidden nodes for multilayer perceptron with a single hidden layer

The universal approximation theorem states that: Let $\phi(·)$ be a nonconstant, bounded, and monotonically-increasing continuous function. Let $I_m$ denote the m-dimensional unit hypercube $[0,1]^m$....
4
votes
1answer
512 views

Parameters of energy function for TSP

[This question was initially asked here. It went unanswered so I thought I should ask it in a different community.] I am reading this paper by Hopfield et al. On page six, the authors defined the ...
21
votes
2answers
12k views

Universal Approximation Theorem — Neural Networks

I posted this earlier on MSE, but it was suggested that here may be a better place to ask. Universal approximation theorem states that "the standard multilayer feed-forward network with a single ...
0
votes
1answer
108 views

When to apply soft constraints evaluation functions in genetic algorithms?

As stated in the question, say i created a random population of 100 timetables. 1 % of these timetables are valid. Then, if I apply the soft constraints evaluation functions on the 1% valid timetables,...
0
votes
1answer
630 views

To what extent is it possible to use genetic algorithms to make wind mill turbine blades more efficient?

I recently watched this video on youtube. It featured someone explaining how he used genetic algorithms to improve the efficiency of wind mill turbines by finding the optimal shape for the blades. ...
4
votes
1answer
1k views

Neural Networks to classify accelerometer double taps

I'm building an application for Android devices that requires it to recognize, by accelerometer data, the difference between walking noise and double tapping it. I'm trying to solve this problem using ...
4
votes
2answers
503 views

Selection in a genetic algorithm

I have a working genetic algorithm which uses a few genetic operators on pairs of individuals. These two individuals are selected by fitness-proportional (roulette-wheel) selection, that is, an ...
2
votes
2answers
489 views

Accuracy rate in neural networks

I've implemented a neural network (using the back-propagation algorithm) in a digital marketing application, where the algorithm classifies words according to quality, price and another ...
2
votes
1answer
182 views

Linear Function Representation of Neural Network

I have trained a neural network using MATLAB and am ready to deploy it into my software. Right now, I include it in my software by programming the structure of the neural network and the connection ...
2
votes
0answers
187 views

Computational power of cellular neural networks

A cellular neural network is a kind of recurrent neural network that could be thought of as a hybrid between neural nets and cellular automata. As I understand it, the classic Chua-Yang CNN is ...
3
votes
1answer
123 views

What is the correct name for the space of genotypes and fitness?

I'm looking for a formal definition of the space that consists of the dimensions gene 1-N and the fitness. In literature there is often the search space mentioned, but it only contains all genes. If ...
9
votes
2answers
3k views

Purpose and determining when to use hidden layers

Following up on this question... I am attempting to learn how to use and create neural networks for my research, and one point is somewhat escaping me. I realize that hidden layers are a somewhat ...
0
votes
1answer
2k views

What are some examples of sequential-decision tasks?

I ask this because I'm currently learning about Neural Networks as a subset of the machine learning algorithms Just trying to get some intuition on what sort of problems out there are categorized as "...
3
votes
0answers
73 views

Can we benefit the linear relation between $\sigma$ and best convergence speed in $(\mu,\lambda)$-ES and $(\mu+\lambda)$-ES?

$(\mu,\lambda)$-ES and $(\mu+\lambda)$-ES are two well-known Evolution Strategies. One variant of these algorithms is when we set a constant value for $\sigma$ and generate off-springs for $x$ as: $$ ...
2
votes
1answer
310 views

GA puzzle solver stuck at local maximum

I have a jigsaw type problem with 192 pieces which I am trying to find solutions to. I have written a GA which starts from a random allocation then 'crosses' by taking rectangular blocks from one ...
6
votes
2answers
369 views

What limits the performance of evolutionary computing techniques?

What limits the current performance of genetic algorithms and neural networks? The principles underlying these techniques, at least the popular science presentation of these principles, suggests that ...
0
votes
2answers
405 views

Genetic Algorithm convergence test functions

I was working on parallel implementation of Genetic Algorithm with MapReduce. I have found that in many papers authors are referencing OneMAX as problem they used to test scalability and convergence ...
4
votes
2answers
2k views

Dealing with duplicates in Genetic Algorithm

I'm using Genetic Algorithm to create a rota for home-care organisation. All the groundwork is complete, I'm getting the results, but results are not as good as expected. If I calculate fitness for ...
-6
votes
2answers
407 views

Genetic algorithms

Is it theoretically possible to reconstruct the contents of a file from its id using evolutionary computing? A file in this case can be a text, image, video or audio file. The 'id' in this case, ...
15
votes
2answers
570 views

Quantum PAC learning

Background Functions in $AC^0$ are PAC learnable in quasipolynomial time with a classical algorithm that requires $O(2^{log(n)^{O(d)}})$ randomly chosen queries to learn a circuit of depth d [1]. If ...
22
votes
1answer
766 views

How much computational power fits into a cubic centimeter?

This question is a followup on the question about DNA algorithms asked by Aadita Mehra. In comments there, Joe Fitzsimmons said, in part: [T]he radius of the system must scale proportionately to ...
13
votes
1answer
1k views

Computational Power of Neural Networks?

Let's say we have a single-layer feed forward neural network with k inputs and one output. It calculates a function from $\lbrace 0,1\rbrace ^{n}\rightarrow\lbrace 0,1\rbrace $, it's fairly easy to ...
0
votes
1answer
135 views

Analysis of variables of varying numbers

i work with amino acid sequences and i want to use a selfmade model to tell me something about it, lets call it f(seq). Now i want to know the contribution of every position in the sequence onto the ...
7
votes
3answers
600 views

What subjects, topics does a computer science graduate need to learn to apply available machine learning frameworks, esp. SVMs

I want to teach myself enough machine learning so that I can, to begin with, understand enough to put to use available open source ML frameworks that will allow me to do things like: Go through the ...
0
votes
1answer
507 views

What does “number of inputs to each neuron” mean in Neural Network terms?

I am reading about a Neural Networks project that has some data like this I am new to this, and though I think I understand what a 3:1 network mean, I do not understand what number of inputs (to each ...
55
votes
10answers
3k views

Provable statements about genetic algorithms

Genetic algorithms don't get much traction in the world of theory, but they are a reasonably well-used metaheuristic method (by metaheuristic I mean a technique that applies generically across many ...
4
votes
2answers
2k views

What is schema theory within Genetic Algorithms

I understand (some) of the workings behind schema theory in genetic algorithms, for example: *****0 would match the genome ...