Examples of nontrivial non-discriminatory functions

I am reading Cybenko's "Approximation by Superpositions of a Sigmoidal Function". The paper defines a discriminatory function as:

$\sigma$ is discriminatory if for a measure $\mu$, \begin{align} \int \sigma (y^\top x + \theta) d \mu(x) = 0 \end{align} for all $y \in R^n$ and $\theta \in R$, then $\mu=0$.

What is an example (with proof) of a non-discriminatory function? (besides 0)

• What about $\sigma$=the sign function? Then for $\mu$ which is a product of normal gaussians, the integral is zero. – Aryeh Aug 22 '18 at 2:29
• well in that case if we let $\theta$ go to infinity, then wouldnt the output of $\sigma$ be 1 for most of the mass, so the integral would be positive? – LYH Aug 22 '18 at 3:08
• Can you spell out the exact quantifiers over $y$, $\theta$, and $\mu$? – Aryeh Aug 22 '18 at 3:11
• $\mu$ is some finite signed regular Borel measure on $[0, 1]^n$. (you only have to find one $\mu$ that works) The condition has to be satisfied for any $y \in R^n$ and any $\theta \in R$. I'm confused how a nontrivial function can make that integral $0$ for all $y$ and all $\theta$. – LYH Aug 22 '18 at 3:25
• I agree, in this formulation it doesn't seem like there should be any non-trivial candidates for $\mu$. I suspect it's an issue of quantifiers. – Aryeh Aug 22 '18 at 4:06

Let $n=1$. Let $\mu$ be the usual Lebesgue length measure on $[1/2,1]$, and let $\mu$ be the negative of the usual Lebesgue length measure on $[0,1/2]$.

In particular, Lebesgue measure is $|\mu|$.

Let $\mathcal U\subseteq [0,1]$ be a set of Lebesgue measure 0. (For instance, $\mathcal U$ could be the set of all numbers whose binary expansion does not satisfy the law of large numbers...)

Let

• $\sigma(x)=1$ for $x\in\mathcal U$, and
• $\sigma(x)=0$ for $x\not\in\mathcal U$.

Then given $y$ and $\theta$, there are two cases.

Case 1: $y\ne 0$. Then $$|\mu|\{X:y^\top X+\theta\in\mathcal U\}=0,$$ hence $$|\mu|\{X:\sigma(y^\top X+\theta)\ne 0\}=0,$$ so $\int \sigma(y^\top X+\theta)d|\mu|(x)=0$, so $\int \sigma(y^\top X+\theta)d\mu(x)=0$.

Case 2: $y=0$. Then $\sigma(y^\top X+\theta)$ is the constant $\sigma(\theta)$, which satisfies $$\int \sigma(\theta)\,d\mu(x)=\sigma(\theta)\int d\mu(x)=\sigma(\theta)\mu([0,1])=0.$$

So, $\sigma$ is non-discriminatory.