Below is a long-winded answer, but tl;dr in the general case there is no hope for such a formulation, but for many of the particular classes of sparse graphs that have regularity lemmas this formulation exists.
For background, there are two popular versions of the SRL. They are: for any fixed $\varepsilon > 0$ and any $n$-node graph $G = (V, E)$, one can partition $V = V_0 \cup V_1 \cup \dots \cup V_p$ into $p = O_{\varepsilon}(1)$ parts so that ...
(Combinatorial Phrasing) (1) $|V_0| \le \varepsilon n$ and the sizes of any $V_1, \dots, V_p$ differ by at most $1$ ($V_0$ is called the "exceptional set"), and (2) all but $\varepsilon p^2$ pairs of the remaining parts $(V_i, V_j)$ satisfy
$$\left|d(S, T) - d(V_i, V_j)\right| < \varepsilon \text{ for all } S \subseteq V_i, T \subseteq V_j$$
(here $d(\cdot, \cdot)$ gives the density between the parts, i.e. the fraction of edges that are present).
(Analytic Phrasing) Letting
$$\mathop{disc}(V_i, V_j) := \max \limits_{S \subseteq V_i, T \subseteq V_j} |V_i||V_j|\left|d(V_i, V_j) - d(S, T)\right|,$$
we have
$$\sum \limits_{i, j =0}^p \mathop{disc}(V_i, V_j) < \varepsilon n^2.$$
The "combinatorial phrasing" (I just made these names up, they are not standard) is the original and probably more famous one, whereas the "analytic phrasing" is more modern and related to graph limits, etc (I think it was popularized here). To my eye the analytic one is the right formalization of "graph approximated by union of bipartite expanders," since it gives a control on the total "error" of such an approximation, and there is not an exceptional set in which to hide mass. But at this point this is just cosmetic, because it's an easy but important lemma that these two phrasings are equivalent. To get from Combinatorial to Analytic, just union bound the contribution to discrepancy of the irregular parts and exceptional set. To get from Analytic to Combinatorial, just move any part that contributes too much discrepancy to the exceptional set and apply Markov's Inequality to control its mass.
Now to sparse regularity. The goal of sparse regularity is to replace the $\varepsilon$ in the respective inequalities with $\varepsilon d(G)$, where $d(G)$ is the fraction of all possible edges present in $G$. Critically, with this change, the two phrasings are no longer equivalent. Rather, the Analytic phrasing is stronger: it still implies Combinatorial exactly as before, but Combinatorial does not generally imply Analytic, because (as anticipated in the OP) one can potentially hide a lot of density in the exceptional set or between the non-regular pairs of parts. Indeed, this separation is formal: the lower bound graphs for the dense SRL (say, this one) imply that the Analytic Phrasing doesn't extend in general to sparse graphs, but the paper by Scott linked in the OP shows that the Combinatorial Phrasing actually does extend to all sparse graphs with no conditions.
The survey linked in the OP mostly talks about an SRL for "upper-regular" sparse graphs, which roughly means that the graph has no cuts that are denser than average by more than a constant factor. For these particular graphs, the Combinatorial and Analytic phrasings are equivalent, because there can't be too much extra mass hidden in the exceptional parts so their contribution to discrepancy can be union bounded like in the dense case. So these graphs have an "approximated by union of bipartite expanders" interpretation.
Finally, I should mention that there are many other hypotheses in the literature that also imply equivalence between these phrasings. For example, $L_p$ Upper Regularity (defined here) is more general than Upper Regularity and is still enough to imply equivalence. However, for this graph class and others, I am only aware of associated weak regularity lemmas.
