3
$\begingroup$

I am reading Cybenko's "Approximation by Superpositions of a Sigmoidal Function". The paper defines a discriminatory function as:

$\sigma$ is discriminatory if for a measure $\mu$, \begin{align} \int \sigma (y^\top x + \theta) d \mu(x) = 0 \end{align} for all $y \in R^n$ and $\theta \in R$, then $\mu=0$.

What is an example (with proof) of a non-discriminatory function? (besides 0)

$\endgroup$
5
  • $\begingroup$ What about $\sigma$=the sign function? Then for $\mu$ which is a product of normal gaussians, the integral is zero. $\endgroup$
    – Aryeh
    Aug 22, 2018 at 2:29
  • $\begingroup$ well in that case if we let $\theta$ go to infinity, then wouldnt the output of $\sigma$ be 1 for most of the mass, so the integral would be positive? $\endgroup$
    – LYH
    Aug 22, 2018 at 3:08
  • $\begingroup$ Can you spell out the exact quantifiers over $y$, $\theta$, and $\mu$? $\endgroup$
    – Aryeh
    Aug 22, 2018 at 3:11
  • 1
    $\begingroup$ $\mu$ is some finite signed regular Borel measure on $[0, 1]^n$. (you only have to find one $\mu$ that works) The condition has to be satisfied for any $y \in R^n$ and any $\theta \in R$. I'm confused how a nontrivial function can make that integral $0$ for all $y$ and all $\theta$. $\endgroup$
    – LYH
    Aug 22, 2018 at 3:25
  • $\begingroup$ I agree, in this formulation it doesn't seem like there should be any non-trivial candidates for $\mu$. I suspect it's an issue of quantifiers. $\endgroup$
    – Aryeh
    Aug 22, 2018 at 4:06

1 Answer 1

1
$\begingroup$

Let $n=1$. Let $\mu$ be the usual Lebesgue length measure on $[1/2,1]$, and let $\mu$ be the negative of the usual Lebesgue length measure on $[0,1/2]$.

In particular, Lebesgue measure is $|\mu|$.

Let $\mathcal U\subseteq [0,1]$ be a set of Lebesgue measure 0. (For instance, $\mathcal U$ could be the set of all numbers whose binary expansion does not satisfy the law of large numbers...)

Let

  • $\sigma(x)=1$ for $x\in\mathcal U$, and
  • $\sigma(x)=0$ for $x\not\in\mathcal U$.

Then given $y$ and $\theta$, there are two cases.

Case 1: $y\ne 0$. Then $$|\mu|\{X:y^\top X+\theta\in\mathcal U\}=0,$$ hence $$|\mu|\{X:\sigma(y^\top X+\theta)\ne 0\}=0,$$ so $\int \sigma(y^\top X+\theta)d|\mu|(x)=0$, so $\int \sigma(y^\top X+\theta)d\mu(x)=0$.

Case 2: $y=0$. Then $\sigma(y^\top X+\theta)$ is the constant $\sigma(\theta)$, which satisfies $$\int \sigma(\theta)\,d\mu(x)=\sigma(\theta)\int d\mu(x)=\sigma(\theta)\mu([0,1])=0.$$

So, $\sigma$ is non-discriminatory.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.