I am reading Cybenko's "Approximation by Superpositions of a Sigmoidal Function". The paper defines a discriminatory function as:

$\sigma$ is discriminatory if for a measure $\mu$, \begin{align} \int \sigma (y^\top x + \theta) d \mu(x) = 0 \end{align} for all $y \in R^n$ and $\theta \in R$, then $\mu=0$.

What is an example (with proof) of a non-discriminatory function? (besides 0)

  • $\begingroup$ What about $\sigma$=the sign function? Then for $\mu$ which is a product of normal gaussians, the integral is zero. $\endgroup$ – Aryeh Aug 22 '18 at 2:29
  • $\begingroup$ well in that case if we let $\theta$ go to infinity, then wouldnt the output of $\sigma$ be 1 for most of the mass, so the integral would be positive? $\endgroup$ – LYH Aug 22 '18 at 3:08
  • $\begingroup$ Can you spell out the exact quantifiers over $y$, $\theta$, and $\mu$? $\endgroup$ – Aryeh Aug 22 '18 at 3:11
  • 1
    $\begingroup$ $\mu$ is some finite signed regular Borel measure on $[0, 1]^n$. (you only have to find one $\mu$ that works) The condition has to be satisfied for any $y \in R^n$ and any $\theta \in R$. I'm confused how a nontrivial function can make that integral $0$ for all $y$ and all $\theta$. $\endgroup$ – LYH Aug 22 '18 at 3:25
  • $\begingroup$ I agree, in this formulation it doesn't seem like there should be any non-trivial candidates for $\mu$. I suspect it's an issue of quantifiers. $\endgroup$ – Aryeh Aug 22 '18 at 4:06

Let $n=1$. Let $\mu$ be the usual Lebesgue length measure on $[1/2,1]$, and let $\mu$ be the negative of the usual Lebesgue length measure on $[0,1/2]$.

In particular, Lebesgue measure is $|\mu|$.

Let $\mathcal U\subseteq [0,1]$ be a set of Lebesgue measure 0. (For instance, $\mathcal U$ could be the set of all numbers whose binary expansion does not satisfy the law of large numbers...)


  • $\sigma(x)=1$ for $x\in\mathcal U$, and
  • $\sigma(x)=0$ for $x\not\in\mathcal U$.

Then given $y$ and $\theta$, there are two cases.

Case 1: $y\ne 0$. Then $$|\mu|\{X:y^\top X+\theta\in\mathcal U\}=0,$$ hence $$|\mu|\{X:\sigma(y^\top X+\theta)\ne 0\}=0,$$ so $\int \sigma(y^\top X+\theta)d|\mu|(x)=0$, so $\int \sigma(y^\top X+\theta)d\mu(x)=0$.

Case 2: $y=0$. Then $\sigma(y^\top X+\theta)$ is the constant $\sigma(\theta)$, which satisfies $$\int \sigma(\theta)\,d\mu(x)=\sigma(\theta)\int d\mu(x)=\sigma(\theta)\mu([0,1])=0.$$

So, $\sigma$ is non-discriminatory.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.