Firstly, I admit that my question is just something very blurred but I hope someone please give me some documents to read.
The execution of a program $P$ can be considered as a sequence of state transitions $State \to State $ (this sequence should be finite if the program halts). By that way we get execution traces of program. Now, we may note that these traces (denoted as a set $T$) are generated by a static structure, that is the syntax of $P$. So we receive the first constraint: the set $T$ can not be arbitrary, instead $T$ must be result of some static rewriting rules applied on the static syntax of program.
Continue this way, if $P$ is a self-modifying program, roughly speaking $P$ can firstly generate other programs $Q_{1}, Q_{2}, \dotsc$, and these programs, at their turns, will generate other traces $T_{1},T_{2}, \dotsc$, so $T = T_{1} \cup T_{2} \cup \dotsc$. And we receive the second constraint: the set $T$ must be result of some static rewriting rules applying on some static rewriting rules applying on the static syntax of programs.
We can continue this construction to some level (may be there exists a limit ?). But a similar view I get is in the high order category theory where there exist morphisms between morphisms, and morphisms between morphisms between morphisms, etc.
So my question is: Do exist some reseaches on that or my thinking is just a stupid nonsense thing ?
