# Vector representation of probabilistic automaton

Suppose we have $N$ states

$q_i$, $0\leq i<N$

Suppose we have $N\times N$ probability matrix

$p_{ij}$, $0\leq i<N$, $0\leq j<N$,

And suppose a machine, which is being in state $q_i$ then goes to state $q_j$ with probability $p_{ij}$.

Question #1: is this what is called "probabilistic automaton"?

Next.

We can introduce another interpretation: represent each state of a machine as a column of values

$w_i$, $0\leq i<N$

each representing a "weight" of previously defined state $q_i$.

If machine is completely in state $q_0$ then we have

$w_i=1, i=0$

$w_i=0, i\neq0$

and so on.

In this interpretation each state change is just changing of the weights of a state column

$w_i \rightarrow p_{ij} w_i$

assumming summation by.

Question #2: it this a precise interpretation of previous PA? If each first interpretation corresponds with second and vice versa?

• Regarding #1: What you are describing is a (discrete time) Markov chain. In a nutshell, the difference to PA is that the latter also feature nondeterminism, i.e. you actually have several probability matrices (usually labelled by some actions) between which you can choose in each step. – Klaus Draeger Jul 5 '13 at 11:04
• Are these matrices chosen by input symbols? Or by previous states history? Or randomly? – Suzan Cioc Jul 5 '13 at 11:32
• It depends on what you want to model - often, the question is about the minimum/maximum probability of some class of behaviors (like those reaching a given set of goal states). Then the choice is assumed to be made by a minimizing/maximizing strategy (and the question boils down to finding this strategy). This strategy can take the history into account (though this is not necessary for mere reachability). The symbols in general don't uniquely determine a successor distribution; there is a related model (Markov decision processes), where they do. – Klaus Draeger Jul 5 '13 at 11:42
• @KlausDraeger: with the latter comment you mean that there may be an internal non-determinism as well? – Ilya Jul 5 '13 at 14:37