# Efficient and simple randomized algorithms where determinism is difficult

I often hear that for many problems we know very elegant randomized algorithms, but no, or only more complicated, deterministic solutions. However, I only know a few examples for this. Most prominently

• Randomized Quicksort (and related geometric algorithms, e.g. for convex hulls)
• Randomized Mincut
• Polynomial Identity Testing
• Klee's Measure problem

Among these, only polynomial identity testing seems to be really hard without the use of randomness.

Do you know more examples of problems where a randomized solution is very elegant or very efficient, but deterministic solutions are not? Ideally, the problems should be easy to motivate for laymen (unlike e.g. polynomial identity testing).

• Another example is primality testing. The Miller–Rabin and Solovay–Strassen probabilistic primality tests are very simple and efficient. It was a long standing open problem to find an efficient deterministic primality test, which was solved by Agrawal, Kayal and Saxena. The AKS test is a deterministic polynomial-test primality test. However, it is not as simple and not so efficient as probabilistic tests. – Yury Nov 20 '12 at 13:39
• Randomized selection (median finding) is somewhat easier than deterministic. Randomized algorithms for approximately solving packing and covering LPs are faster (in worst-case) than their deterministic counterparts (KY07, GK95). Many online problems have randomized alg's that are more competitive than any deterministic algorithm FK91. – Neal Young Nov 20 '12 at 16:00
• Computing the volume of a convex body in high dimensions admits a $(1+\epsilon)$-approximation via randomization. It is known that no deterministic algorithm can give a good approximation. Hence randomization is essential here. – Chandra Chekuri Nov 20 '12 at 17:00
• @ChandraChekuri that's a good comment and would be an even better answer :) – Suresh Venkat Nov 20 '12 at 17:28
• @ChandraChekuri in the oracle model, otherwise $\mathsf{BPP} \neq \mathsf{P}$ – Sasho Nikolov Nov 20 '12 at 18:30

Sorting nuts and bolts

The following problem was suggested by Rawlins in 1992: Suppose you are given a collection of n nuts and n bolts. Each bolt fits exactly one nut, and otherwise, the nuts and bolts have distinct sizes. The sizes are too close to allow direct comparison between pairs of bolts or pairs of nuts. However, you can compare any nut to any bolt by trying to screw them together; in constant time, you will discover whether the bolt is too large, too small, or just right for the nut. Your task is to discover which bolt fits each nut, or equivalently, to sort the nuts and bolts by size.

A straightforward variant of randomized quicksort solves the problem in $O(n \log n)$ time with high probability. Pick a random bolt; use it to partition the nuts; use the matching nut to partition the bolts; and recurse. However, finding a deterministic algorithm that even runs in $o(n^2)$ is nontrivial. Deterministic $O(n\log n)$-time algorithms were finally found in 1995 by Bradford and independently by Komlós, Ma, and Szemerédi. Under the hood, both algorithms use variants of the AKS parallel sorting network, so the hidden constant in the $O(n\log n)$ time bound is quite large; the hidden constant for the randomized algorithm is 4.

• Noga Alon, Manuel Blum, Amos Fiat, Sampath Kannan, Moni Noar, and Rafail Ostrovsky. Matching nuts and bolts. Proc. 5th Ann. ACM-SIAM Symp. Discrete Algorithms, 690–696, 1994.
• Noga Alon, Phillip G. Bradford, and Rudolf Fleischer. Matching nuts and bolts faster. Inform. Proc. Lett. 59(3):123–127, 1996.
• Phillip G. Bradford. Matching nuts and bolts optimally. Tech. Rep. MPI-I-95-1-025, Max-Planck-Institut für Informatik, 1995. http://domino.mpi-inf.mpg.de/internet/reports.nsf/NumberView/1995-1-025
• Phillip G. Bradford and Rudolf Fleischer. Matching nuts and bolts faster. Proc. 6th. Int. Symp. Algorithms Comput., 402–408, 1995. Lecture Notes Comput. Sci. 1004.
• János Komlós, Yuan Ma, and Endre Szemerédi. Matching nuts and bolts in $O(n\log n)$ time. SIAM J. Discrete Math. 11(3):347–372, 1998.
• Gregory J. E. Rawlins. Compared To What? : An Introduction to The Analysis of Algorithms. Computer Science Press/W. H. Freeman, 1992.
• This is a beautiful example, but it's an oracle problem. Is there some way to remove the oracle from it? – Peter Shor Nov 24 '12 at 21:40
• Got a link to the 98 Szemeredi paper? How is this hard? In parallel compare each bolt to a unique nut and put each pair in sorted order; removing matched elements. In log(n) steps merge the sorted nbnbnbnbnb sequences, kicking out matches as they arise. EDIT: Yeah the noncomparability of nnn and bbbb strings is annoying on the merge step. – Chad Brewbaker Nov 29 '12 at 23:06
• @ChadBrewbaker Suppose in every pair but one, the bolt is smaller than the nut. (Yes, this is possible.) Now what does your algorithm do? In other words, "annoying" = "the whole problem". – Jeffε Nov 30 '12 at 5:29
• I was looking for the Szemeredi paper and thinking out loud how it is hard. Yes, I concur that a merge based approach is nontrivial; but Vishkin's papers on parallel graph connectivity leave a gut feeling it is not impossible. – Chad Brewbaker Nov 30 '12 at 20:08
• With each comparison from a nut and bolt you get a directed edge added to the graph or a match which removes both vertices. The goal would be to merge connected components in such a way that they collapse all matches between them with a linear amount of work and keep the storage size of edges in a connected component bounded to linear space. – Chad Brewbaker Nov 30 '12 at 20:14

Once you are not just talking about poly-time but rather look at the many models of computation we study, there are examples everywhere:

In Logspace: Un-directed ST connectivity (in RL since 1979, and in L only since 2005)

In NC: Finding a perfect matching in a bipartite graph in parallel (in RNC and still not known to be in NC)

In interactive proofs: deterministic ones give NP, while randomized ones can do PSPACE. Related: checking a proof deterministically requires looking at all the proof, while PCP proofs allow you to check only a constant number of bits.

In Algorithmic Mechanism Design: many randomized truthful approximation mechanisms with no deterministic counterpart.

In Communication complexity: the equality function requires linear communication deterministically but logarithmic (or, depending on exact model, constant) communication randomly.

In decision trees: evaluating an and-or tree requires linear queries deterministically but much less with randomization. This is essentially equivalent to alpha-beta pruning which gives a randomized sub-linear algorithm for game-tree evaluation.

In streaming models, distributed computing models: see previous answers.

Most streaming algorithms

In the streaming model of computation (AMS, book), an algorithm processes an online sequence of updates and is restricted to keep only sublinear space. At any point in time, the algorithm should be able to answer a query.

For many problems there exist sublinear space randomized streaming algorithms while provably no deterministic algorithm can solve the problem in sublinear space. This is related to gaps between randomized and deterministic communication complexity. A simple example is the distinct count problem: at each time step $t$ the algorithm is given an integer $i_t \in [n]$, and it should be able to approximate $D_m = |\{i_t: t = 1\ldots m\}|$, i.e. the number of distinct integers seen up to time step $m$. It's relatively easy to show that any deterministic algorithm achieving constant approximation must use $\Omega(n)$ space (see e.g. lecture notes by Piotr Indyk). On the other hand the clever sampling algorithm of Flajolet and Martin (simple analysis with limited randomness in the AMS paper linked above) achieves constant approximation in $O(\log n)$ bits. The latest work on the problem gives an optimal $O(\frac{1}{\epsilon^2} + \log n)$ algorithm that computes an $1\pm \epsilon$ approximation.

Finding a maximal independent set in a distributed network of $n$ nodes with maximum degree $\Delta$. There's a known lower bound [3] of $\min(\Omega(\log\Delta),\Omega(\sqrt{\log n}))$ that holds for randomized and deterministic algorithms.

The following is a simple randomized distributed algorithm [1] that proceeds in synchronous rounds. (In a round, every node $u$ can perform some local computation and send messages to its neighbors. These messages are guaranteed to be received before the start of the next round.)

1. In every round, each active node $u$ marks itself with probability $1/d_u$ where $d_u>0$ is the degree of $u$; if $d_u=0$, $u$ simply enters the independent set. (Initially, every node is active.)
2. If $u$ is the only marked node in its neighborhood, $u$ enters the independent set, deactivates itself and notifies all of its neighbors to deactivate themselves. The degrees of the remaining active nodes are decreased accordingly, i.e., all edges to deactivated nodes are removed.
3. Otherwise, if there is some neighboring node $v$ that is also marked, the lower degree vertex unmarks itself and remains active.

It can be shown that this algorithm terminates in $O(\log n)$ rounds with high probability, by arguing that half of the remaining edges are deleted in every round. In contrast, the fastest known deterministic distributed algorithm [2] takes $O(n^{1/\sqrt{\log n}})$ rounds and is considerably more complicated.

[1] Michael Luby: A Simple Parallel Algorithm for the Maximal Independent Set Problem. SIAM J. Comput. 15(4): 1036-1053 (1986) http://dx.doi.org/10.1137/0215074

[2] Alessandro Panconesi, Aravind Srinivasan: On the Complexity of Distributed Network Decomposition. J. Algorithms 20(2): 356-374 (1996) http://dx.doi.org/10.1006/jagm.1996.0017

[3] Fabian Kuhn, Thomas Moscibroda, Roger Wattenhofer: Local Computation: Lower and Upper Bounds. CoRR abs/1011.5470 (2010) http://arxiv.org/abs/1011.5470

• A recent algorithm (at PODC 2013), inspired by biological systems, achieves performance as good as Luby's by using a simple local feedback mechanism. arxiv.org/abs/1211.0235 – András Salamon May 8 '13 at 15:15

## Leader Election in an Anonymous Ring of Processes

Suppose that you have a ring network of processes that do not have ids and that communicate by message passing. Initially, every process is in the same state. You want to design a distributed algorithm such that eventually exactly $1$ process enters the elected state and all other processes enter the non-elected state. This is the so called leader election problem which is one of the fundamental symmetry breaking tasks in a distributed system and has many applications.

There's a simple argument (e.g. [1]) that there is no deterministic leader election algorithm for an anonymous ring.

Model: We assume that the computation advances in synchronous rounds where, in each round, every process performs some local computation, sends messages to its neighbors in the ring, and receives messages from its neighbors.

For the sake of a contradiction, let's assume that there is such a deterministic leader election algorithm $A$. It is sufficient to show that, at the start of any round $r\ge 0$, all processes are in the same state, since this implies that there cannot be exactly $1$ process in the elected state. Since processes do not have ids and the network is symmetric, every process is in the same initial state, which provides the induction base.

For the induction step, consider some round $r\ge 0$ and assume that every process is in the same state at the start of round $r$. Therefore, since algorithm $A$ is deterministic, every process performs exactly the same computation and sends exactly the same messages during round $r$. This in turn implies that every process receives exactly the same messages during $r$ and, by the start of round $r+1$, is in the same state. Thus, no such algorithm $A$ can exist.

If $A$ is a randomized algorithm on the other hand and processes know the size of the ring $n$, there's an easy way to break symmetry, by generating a random id from the range $[1,n^4]$, which will result in unique ids for all processes with high probability. A simple and naive algorithm proceeds by letting every process send its id along the ring and instruct processes to forward only messages containing the largest id seen so far. This guarantees that only the process who generated the largest id will receive its own message once it has traversed the entire ring and elect itself as the leader.

[1] Dana Angluin: Local and Global Properties in Networks of Processors (Extended Abstract). STOC 1980: 82-93. http://doi.acm.org/10.1145/800141.804655

Majority problem in a query model.

Problem. We are given a set of $n$ balls colored with two or more colors. The goal is to ﬁnd a ball of the majority color (i.e., a color that occurs more than n/2 times) assuming such a color exists, using queries of the form "Does ball $i$ and ball $j$ have the same color?". The strategy must be oblivious, i.e., queries cannot depend on the results of previous queries.

Randomized Algorithm. Pick a random ball and check if more than $n/2$ balls have the same color. This algorithm runs in $O(n)$ time in expectation.

Deterministic $O(n)$ algorithm is quite non-trivial and is a nice application of expanders.

F. R. K. Chung, R. L. Graham, J. Mao, and A. C. Yao, Oblivious and adaptive strategies for the Majority and Plurality problems, Proc. COCOON 2005, pp. 329–338.