Applications of Takens' theorem to TCS?

Question

My apologies if the question is a tad vague—I did try to search the literature for more, but didn't find anything (the similarity between the keywords "Takens" and "taken" on Google may be partly to blame).

If I understood correctly the Wikipedia page on Takens' theorem, and some discussions I had with people in applied (physics-related) fields that mentioned it to me, Taken's theorem essentially states that

if one has a system parameterized by a state (time-dependent) $x(t)\in\mathbb{R}^d$, then under some natural assumptions on $x$ (in particular, smooth enough) there exists $k$—related to the underlying dimension of the manifold on which $x$ "lives— and some "lag" $\tau$ such that $$ y(t) \stackrel{\rm def}{=} (x_1(t), x_1(t+\tau),\dots,x_1(t+(k-1) \tau)\in\mathbb{R}^k $$ is information-theoretically equivalent to the original parameterization $x$; and moreover, this new parameterization, now living in a "shadow manifold," preserves the topology of the original manifold.

(All imprecisions above definitely mine, I cannot claim I fully understand the theorem.)

This does strike me as a very useful tool, similar in spirit to some type of dimension reduction:

• bringing in domain-specific knowledge (to figure out $\tau$ and $k$, although apparently there have been recently methods designed to find these in a more automated way)
• allowing easy measuring (no need to perform a computationally heavy mapping for the change of parameters: instead, focusing in a single lagged coordinate of the data is enough)

My question is: has this theorem, or some version of it, been applied or explored in the context of computational learning? If so, where can I find references of it? If not, is there a reason why—e.g., "it does not apply in the usual settings we consider because [...]"?

Answer 1

Takens himself did some CS work although not TCS work. He did some attractor reconstruction stuff with neural networks, for example (https://clgiles.ist.psu.edu/papers/NC-2000-learning-chaos-nn.pdf) and it does use his embedding theorem to suggest nonlinear autoregressive (time delay) neural nets, and look towards actual recurrent networks.

Overall, dynamical systems is closest to CS in neural network land because of the easy dynamical interpretation of the models. Try some of the Ganguli gang's publications (https://ganguli-gang.stanford.edu/pubs.html), esp. Exponential expressivity in deep neural networks through transient chaos

apparently there have been recently methods designed to find these in a more automated way

Again, not theoretical, but I've used them for non-example time series data and they aren't incredible in practice, by the way. The denoising aspects of them is actually quite difficult.

Answer 2

Theoretically, Takens's theorem implies that a deterministic system attractor of finite dimension $D$ can be reconstructed by a finite number of time delays $k$ if $$ k \geq 2D + 1. $$ This implies that the dynamics of eligible non-linear functions can be replicated by a non-linear autoregressive model of $k$ delays. Such a model can, and has been, realized with neural networks (See Eric A. Wan's Modeling Nonlinear Dynamics with Neural Networks: Examples in Time Series Prediction).

The same paper by Wan trains a neural network to predict the sum of Henon iterations. Of course, the predicted and actual iterates diverge, but the attractor is nonetheless replicated by the neural network, realizing Takens's theorem. So, Howon's observation rings true: while the dynamics of certain non-linear functions can be captured with finite delay, their values cannot be so captured perfectly.

Further sources: Chapter 5 of Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability by Mandic and Chambers