Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [machine-learning]

Theoretical questions about Machine learning, especially Computational Learning Theory, including Algorithmic Learning Theory, PAC learning, and Bayesian Inference

2
votes
0answers
748 views

Ultimate Jedi Challenge - Multiarmed Bandit / Reinforcment Learning / advanced AI problem

(This is an attempt to reformulate this question more concisely.) Context You are a Jedi master who wants to prepare a training program (online-algorithm) for his apprentice, "Luke". Luke needs to ...
7
votes
1answer
146 views

Is it possible to create a machine learning classifier to generate Mock interfaces for systems testing?

I'm investigating whether it is feasible to be able to learn a system interface by watching network traffic (assuming the usual problems are solved e.g. encryption etc) I haven't been able to find ...
7
votes
0answers
184 views

Sample complexity of PAC learning all k-DNFs over the uniform distribution

Is sample complexity of PAC learning all $k$-DNFs over the uniform distribution known (that is all DNFs with all terms of size at most $k$ and without restriction on the number of terms)? The only ...
9
votes
1answer
286 views

Learning with (Signed) Errors

$\underline{\bf Background}$ In 2005, Regev [1] introduced the Learning with Errors (LWE) problem, a generalization of the Learning Parity with Error problem. The assumption of this problem's ...
10
votes
2answers
1k views

Introductory resources on Computational Learning Theory

Recently I've been reading a decent number of CoLT papers. Although I don't struggle with the individual papers (at least not more than I usually struggle with other theory papers), I don't feel I ...
4
votes
1answer
1k views

Neural Networks to classify accelerometer double taps

I'm building an application for Android devices that requires it to recognize, by accelerometer data, the difference between walking noise and double tapping it. I'm trying to solve this problem using ...
13
votes
2answers
458 views

Learning triangles in the plane

I assigned my students the problem of finding a triangle consistent with a collection of $m$ points in $\mathbb{R}^2$, labeled with $\pm1$. (A triangle $T$ is consistent with the labeled sample if $T$ ...
3
votes
0answers
80 views

Efficiently Detecting “edges” in the time frequency plane

Given a signal $y(t)\in\mathbb{R}$ I wish to detect edge patterns. $s(f,t)$ is a time-frequency decomposition of $y(t)$ in some window $(t-n,t+n)$ so that $f$ loosely corresponds to a local frequency....
3
votes
1answer
208 views

Agnostic Learning of K-Juntas over “small” distribution

I have two questions related to agnostic learning, one specific and one more general, specifically when the distribution relative to which the learner must operate is given explicitly as part of the ...
3
votes
2answers
236 views

Discerning the best model for a problem

This is a vague question. I will do my best, I think it has definite answers. I am hoping for answers of the form "Read book x, learn this specific topic, read this paper/s". What is bothering me is ...
0
votes
2answers
608 views

Advantages of ANN classifiers over the AdaBoost

So what are the advantages of ANN classifiers over the AdaBoost or Boosting algorithm?
6
votes
3answers
251 views

Asymmetry in Property Testing Definition

Property Testing refers to the problem of making a small number of queries to determine whether $x$ is in some language $L$ or whether it is far away from being in $L$. More precisely we want to ...
0
votes
1answer
192 views

Estimating graphs using random cuts

How easy is it to estimate a graph by observing only a few random cuts? Is there prior work related to this? I did google but could not find anything concrete. Any help would be appreciated. Thanks.
9
votes
2answers
2k views

VC-dimension of spheres in 3 dimension

I am searching for the VC-dimension of the following set system. Universe $U=\{p_1,p_2,\ldots,p_m\}$ such that $U\subseteq \mathbb{R}^3$. In the set system $\mathcal{R}$ each set $S\in \mathcal{R}$ ...
1
vote
1answer
267 views

Theoretical proof of convergence of sequential weight update procedure (Neural Networks and Machine Learning)

My question is at the bottom. (Most of the descriptive words come from Chris. Bishop's Neural Networks for Pattern Recognition) let $w$ be the weight vector of the neural network and $E$ the error ...
11
votes
1answer
479 views

Lower bounds for learning in the membership query and counterexample model

Dana Angluin (1987; pdf) defines a learning model with membership queries and theory queries (counterexamples to a proposed function). She shows that a regular language that is represented by a ...
1
vote
0answers
93 views

Belief Propagation on MRF with complex cliques

Is there a belief propagation algorithm for exact inference on a MRF with complex clique structures (i.e. ones involving more than 2 neighbours)? For MRF's with cliques that only involve pairwise ...
0
votes
0answers
132 views

Tractability of mutual information-augmented ensemble classification algorithms

I am seeking to augment random forest classification using Shannon-Weaver mutual information as a metaheuristic to partition candidate datasets. Specifically, I am trying to determine if such an ...
0
votes
0answers
321 views

Divide-and-conquer approach for hierarchical clustering

I have a huge data set (33K), each represented as a bit-vector of 275-dimensions. Basically my data set can be represented as a $33000 \times 275$ matrix. I want to cluster these bit-vectors. I have ...
7
votes
1answer
147 views

Separation result for proper learning under the uniform vs. adversarial distributions?

Does anyone know of a concept class known to be (1) efficiently learnable under the uniform distribution but (1) NP-hard to learn under arbitrary [adversarial] distributions? I mean "learning" in the ...
8
votes
1answer
3k views

machine learning for code and compiler optimization?

I am looking into ML for generating more efficient code (i.e. compile time and run time heuristics). I have a phd (compilers, hpc), but very little ML experience. I would appreciate any references ...
11
votes
5answers
6k views

Is there any gradient descent based technique for searching absolute minimum (maximum) of a function in multidimensional space?

I'm familiar with gradient descent algorithm which can find local minimum (maximum) of a given function. Is there any modification of gradient descent which allows to find absolute minimum (maximum),...
1
vote
1answer
1k views

What is the significance of abstract linear algebra in machine learning/computer vision research?

I am a computer science research student working in application of Machine Learning to solve Computer Vision problems. Since, lot of linear algebra(eigen-values, SVD etc.) comes up when reading ...
4
votes
1answer
872 views

Stochastic Gradient Descent with integer arithmetics

Most implementations of stochastic gradient descent (SGD) rely on floating points. Is there implementations using infinite or finite precision integer arithmetics ?
-3
votes
1answer
132 views

Semi-supervised learning on graphs

What is semi-supervised learning on graphs? We have been told that if we just have a function which has an input graph, or a given graph with labeled nodes, we should be able to predict labels on ...
0
votes
2answers
386 views

What kind of machine learning is this?

The objective is to build a classifier that produces M correct outputs when given N inputs. Let a "sample" be M outputs and N inputs. Each output is some function some of the N inputs but the ...
1
vote
0answers
118 views

Derivation of Expected-Maximization training equations for a noisy-OR Bayesian network

How do I derive the EM training equations for a noisy-OR Bayesian network? I am familiar with one solution to these equations. If I remember correctly, the result is: Given a model with link ...
10
votes
3answers
934 views

Resource / book for recent advances in statistical learning theory

I'm quite familiar with the theory behind VC-Dimension, but I'm now looking at the recent (last 10 years) advances in statistical learning theory: (local) Rademacher averages, Massart's Finite Class ...
6
votes
1answer
215 views

Is there an accepted name for Ross Quinlan's adaptation of the ID3 decision algorithm to use a Pearson's chi-squared test for independence?

In Ross Quinlan's seminal paper Induction of Decision Trees, Quinlan summarizes the current state of machine learning in 1985 and loudly introduces the ID3 decision algorithm in the context of its ...
0
votes
0answers
383 views

Is it possible to add quantum physics theory to traditional machine learning algorithms to get more accurate results

Lets suppose your data set includes locations in several dimensions and the kind of entity which is the class of the data set, is there other kind of information you need to add (such as times, ...
5
votes
2answers
2k views

Why is Bayesian filtering better than Neural Networks when classifying spam?

According to several people on StackOverflow Bayesian filtering is better than Neural Networks for detecting spam. According to the literature I've read that shouldn't be the case. Please explain!
1
vote
1answer
787 views

Linear Genetic Programming

I have a few questions about linear genetic programming. I'm struggling to find much information on them, hopefully someone here can help me, it would be much appreciated. 1) Initialisation: When ...
0
votes
0answers
258 views

Fuzzy K-modes clustering how to find the cluster centers

I'm trying to understand [fuzzy k-modes][1] algorithm (look mainly at page 3) in order to implement it. I'm stuck at the calculation of cluster centers they said as shown in the link https://...
0
votes
0answers
94 views

Designing an appropriate training set for CART classification using imbalanced data

I'm experimenting with using CART (or maybe Random Forest) to classify genomic data. There are essentially two classes, whereof one is the 'normal' state and the other is the 'exceptional' state. Now,...
0
votes
1answer
162 views

what is the equivalence class of a given polytree representing a directed graphical model?

Let $T$ be a binary tree, where the internal nodes are $Y_1,...,Y_n$ and the leafs are $X_1,...,X_m$. (The $Y$s and the $X$s will eventuall represent a random variable in a Bayesian network.) We can ...
0
votes
1answer
206 views

K-means with centres outside the data?

Say we want to split a cube in $\mathbb{R}^{64}$ into 10 pieces. NN, nearest-neighbor or Voronoi splits, take 10 cluster centres $c_0, \ldots, c_9$ in the cube, e.g. from K-means, then classify a new ...
4
votes
1answer
193 views

Active learning for inferring a convex optimization formulation

I was wondering if anybody knows of any relevant references on the general topic of active learning for gradually inferring/updating a convex opt. formulation. As a specific example, I am thinking ...
11
votes
1answer
3k views

Universal Function approximation

It is known via the universal approximation theorem that a neural network with even a single hidden layer and an arbitrary activation function can approximate any continuous function. What other ...
1
vote
1answer
144 views

Reinforcement Learning and Optimistic Decisions

Sutton recommended using an optimistic heuristic on decision making by considering the upper bound of the confidence range of an action-value. From my testing, it seems to work. I find this ...
4
votes
1answer
254 views

Higher-order and black-box clustering

As far as I understand a large number of clustering problems can be formulated as: $\underset{\textbf{P}}{ \text{argmin}} \; \sum_{i,j} f \left(x_i, x_j\right)$ where $\textbf{P}$ is a partitioning ...
5
votes
4answers
1k views

Genetic Algorithm to Draw a Graph? Position assignment problem

I have an assignment problem at hand and am wondering how suitable it would be to apply local search techniques to reach a desirable solution (the search space is quite large). I have a directed ...
4
votes
2answers
378 views

Can you make a different learner?

My questions are: [Solved by Dave] Given a learner N, can you design a learner M that behaves differently from N? No. and [Solved by Dave] Given a learner N, can you design a learner M that is ...
2
votes
1answer
174 views

Does a multilayer perceptron taught by simple backprop learn the shape of a character or the exact image of a character?

(Feel free to suggest a better title) (same with tags, none of relevant tags exist yet, and I only have 101) Let's say I have a perceptron with an optimal number of layers and optimal number of ...
10
votes
1answer
366 views

Agnostic PAC sampling lower bound

It is well-known that for classical PAC learning, $\Omega(d/\varepsilon)$ examples are necessary in order to acheive an error bound of $\varepsilon$ w.h.p., where $d$ is the VC-dimension of the ...
15
votes
2answers
581 views

Quantum PAC learning

Background Functions in $AC^0$ are PAC learnable in quasipolynomial time with a classical algorithm that requires $O(2^{log(n)^{O(d)}})$ randomly chosen queries to learn a circuit of depth d [1]. If ...
7
votes
1answer
141 views

Optimal payoff from sampling from a collection of Bernoulli random variables?

Suppose I have several Bernoulli random variables, $\{X_1, \ldots, X_k\}$, each of which has a fixed probability $p_i$ of equaling 1 for each sample. Further suppose each $p_i$ is randomly distributed ...
4
votes
2answers
419 views

Support Vector Machines and privacy-preservation

suppose we have data matrix A m-by-n (m observations and n features) which I want to Apply SVM on it achieving privacy (Privacy-Preserving SVM) the questions are:- 1 - Is applying kernel trick ...
1
vote
0answers
280 views

Using Graphs/Graph Similarity as Features for a Learner

I'm working to construct a learner than can recognize whether two vertices in a property graph (digraph, vertices and edges can have arbitrary keys/values) modelling a social network in fact represent ...
1
vote
1answer
320 views

Clustering of letters - what approach would give the best results?

I am working on letter recognition program. I have a text and divide it into letters, every single letter is written to separate file. Now I want to apply a clustering algorithm to these images to ...
13
votes
1answer
2k views

What is the tradeoff between population size and the number of generations in genetic algorithms

Genetic algorithms evolve in fewer generations with a larger population, but also take longer to compute a generation. Are there some guide lines for balancing those two factors, in order to arrive at ...