Before we try to get into ergodic or whatever else, let's try to understand what phenomenon a mathematician or scientist is trying to (or could be trying to) model with AEP. Well
Asymptotic for very large $n$, a lot of coin flips, after a long time, etc ...
Equipartition Equally distributed amongst some boxes or bins, Uniformly random, Equilibrium state, "maximum entropy" (if you like statistical physics ... max. ent. is a misnomer if you don't ... see sec. 4.4 of Cover and Thomas), and here's a cool one heat death of the universe
Essentially (strong/weak) AEP is the Information Theory version of the (strong/weak) Law of Large Numbers. In general, if you satisfy some "Law of Large numbers", i.e. your deviations from the mean (or deviations from "typical behavior") decay "quickly" (say perhaps exponentially), then you satisfy an AEP of some kind.
There is a notion of Strong topicality that is used in Csiszár & Körner (Csiszár invented the method of types, a very powerful method in information theory) that might help you understand better i.e. strong typicality means that:
Strong Typicality: The percentage of $a$'s in $(x_1,...,x_n)$ is approximately equal to $p(a)$.
But what is the probability $p$ in this case!!! That is exactly the point, you have some "equilibrium state" $p$ if you are "ergodic." The point is that you start to behave like a bunch of coin flips at the limit, "when the Markov Process stabilizes."
"Ergodic" is just a vast generalization of i.i.d. with respect to "typical behavior."
The inclusions are (roughly) as follows
I.I.D. $\subset$ Stationary Markov $\subset$ "Ergodic" $\subset$ $Q$ is "isomorphic/asymptotically equivalent" to a free/independent product space $P^n$
If we keep trying to generalize further and further we eventually reach "abstract non-sense" land (which is a beautiful subject by the way, I personally recommend Aluffi and Awody).
Cover and Thomas are excellent expositors, they were trying to introduce you to the notion of "typical behavior" one baby step at a time. The point is not to jump to the most general possible statement of AEP but to understand the general principles behind the theory of Information; such as how one can "typically" communicate messages perfectly over a "noisy/imperfect" channel. For example, Csiszár & Körner introduce strong typicality early on so that they can get the best error bounds in their proofs right off the bat. Go ahead and try to read Csiszár & Körner casually; while Csiszár is one of the great giants of Information theory he is not necessarily the best expositor. As my fellow Peter (Peter$\cong$Pedro) was trying to point out it would just be poor exposition for him to have done that. The vast majority of applications in Network theory don't even use that, the noise in telephone networks is modeled as white noise in practice etc ...
By the way the proof you are looking for is Thm.16.8 in Cover and Thomas but I really don't recommend you try to attack that monster just yet. At least try out Sec.4.4 in Cover and Thomas first to get a rigorous explanation of some of the stuff I stated here.
Good luck on your mathematical adventures! Bon Voyage, Fred!