Algorithm for ranking members of a regular language?

A little while back, I was reading a paper that mentioned a method for computing an integer 'rank' for a particular string $s \in L$ where $L$ is some regular language. This rank uniquely determines any string in the language and has the property that, for some lexicographic ordering, $s_1 < s_2 \implies rank(s_1) < rank(s_2)$. Another property is that given $rank(s_0)$ we can efficiently determine $s_0$ using an 'unrank' algorithm.

However, the paper wasn't very specific and I don't know where to find a more detailed description of the algorithm. Can anyone point me in the right direction?

I remember that it involves compiling a regular expression to a DFA and giving the description of the DFA to the ranking algorithm.

• it seems impossible, because there are infinitely many words $w$ such that $a<_{lex}w<_{lex}b$, so it is impossible to choose $\mathit{rank}(a)$ and $\mathit{rank}(b)$ in a coherent way. – Denis Aug 9 '13 at 1:46
• doesnt weighted FSMs fit this? weighted FSMs have been used extensively in speech recognition. see also AT&T fsm library which handles them. – vzn Aug 9 '13 at 14:59
• Are you sure it's an integer rank? There are many choices for this if you use ordinals to rank sentences. – Charles Stewart Aug 13 '13 at 7:19
• Ordinals, you mean like the natural numbers? Yeah that's what I meant, obviously it's nonsensical for strings to have a negative rank. – pg1989 Aug 13 '13 at 16:30

Let $A = \{1, ..., k\}$ be an ordered alphabet. Then each word on $A^*$ can be viewed as a number in base $k + 1$ (note that $0$ is never used on purpose). Now define $$rank(u) = \begin{cases} u &\text{if u \in L} \\ 0 &\text{otherwise} \end{cases}$$ Then $rank$ preserves the shortlex (or radix) order, which is the order $\leqslant$ on $A^*$ defined by $u \leqslant v$ if and only if $|u| < |v|$ or $|u| = |v|$ and $u \leqslant_{lex} v$. That is, words are first ordered by length and words of equal length are ordered according to the lexicographic order. Now, if $L$ is regular, $rank$ is easy to compute and is of course one-to-one on $L$.

This is another classical example of inductive enumeration. I assume that we are given some DFA for the language $L$ over an ordered alphabet $\Sigma$. Let's start with the easier case of words of length $n$. Using linear algebra, given a prefix $w$, we can compute efficiently $N(w,m) = |\{ x \in \Sigma^m : wx \in L\}|$. Define $N(m) = N(\epsilon,m)$ for short (this is the number of words of length $m$). We define the code $C'(w)$ of a word $w \in L$ as follows: $$C'(a_1\cdots a_n) = \sum_{i=1}^n \sum_{b < a_i} N(a_1\cdots a_{i-1} b,n-i).$$ In order to code words of arbitrary length, we simply use the formula $$C(w) = \sum_{n<|w|} N(n) + C'(w).$$ It is straightforward to compute the inverse of this encoding. This encoding deviates from your specification by ordering first according to length and only then lexicographically, but as dkuper mentions this is unavoidable if you want your range to be $\omega$ rather than a more exotic linear order.

There could be a "trick" which allows using simpler formulas in this particular case, but this is the basic idea.