Efficient and simple randomized algorithms where determinism is difficult

Question

I often hear that for many problems we know very elegant randomized algorithms, but no, or only more complicated, deterministic solutions. However, I only know a few examples for this. Most prominently

• Randomized Quicksort (and related geometric algorithms, e.g. for convex hulls)
• Randomized Mincut
• Polynomial Identity Testing
• Klee's Measure problem

Among these, only polynomial identity testing seems to be really hard without the use of randomness.

Do you know more examples of problems where a randomized solution is very elegant or very efficient, but deterministic solutions are not? Ideally, the problems should be easy to motivate for laymen (unlike e.g. polynomial identity testing).

Answer 1

Sorting nuts and bolts

The following problem was suggested by Rawlins in 1992: Suppose you are given a collection of n nuts and n bolts. Each bolt fits exactly one nut, and otherwise, the nuts and bolts have distinct sizes. The sizes are too close to allow direct comparison between pairs of bolts or pairs of nuts. However, you can compare any nut to any bolt by trying to screw them together; in constant time, you will discover whether the bolt is too large, too small, or just right for the nut. Your task is to discover which bolt fits each nut, or equivalently, to sort the nuts and bolts by size.

A straightforward variant of randomized quicksort solves the problem in $O(n \log n)$ time with high probability. Pick a random bolt; use it to partition the nuts; use the matching nut to partition the bolts; and recurse. However, finding a deterministic algorithm that even runs in $o(n^2)$ is nontrivial. Deterministic $O(n\log n)$-time algorithms were finally found in 1995 by Bradford and independently by Komlós, Ma, and Szemerédi. Under the hood, both algorithms use variants of the AKS parallel sorting network, so the hidden constant in the $O(n\log n)$ time bound is quite large; the hidden constant for the randomized algorithm is 4.

• Noga Alon, Manuel Blum, Amos Fiat, Sampath Kannan, Moni Noar, and Rafail Ostrovsky. Matching nuts and bolts. Proc. 5th Ann. ACM-SIAM Symp. Discrete Algorithms, 690–696, 1994.
• Noga Alon, Phillip G. Bradford, and Rudolf Fleischer. Matching nuts and bolts faster. Inform. Proc. Lett. 59(3):123–127, 1996.
• Phillip G. Bradford. Matching nuts and bolts optimally. Tech. Rep. MPI-I-95-1-025, Max-Planck-Institut für Informatik, 1995. http://domino.mpi-inf.mpg.de/internet/reports.nsf/NumberView/1995-1-025
• Phillip G. Bradford and Rudolf Fleischer. Matching nuts and bolts faster. Proc. 6th. Int. Symp. Algorithms Comput., 402–408, 1995. Lecture Notes Comput. Sci. 1004.
• János Komlós, Yuan Ma, and Endre Szemerédi. Matching nuts and bolts in $O(n\log n)$ time. SIAM J. Discrete Math. 11(3):347–372, 1998.
• Gregory J. E. Rawlins. Compared To What? : An Introduction to The Analysis of Algorithms. Computer Science Press/W. H. Freeman, 1992.

Answer 2

Once you are not just talking about poly-time but rather look at the many models of computation we study, there are examples everywhere:

In Logspace: Un-directed ST connectivity (in RL since 1979, and in L only since 2005)

In NC: Finding a perfect matching in a bipartite graph in parallel (in RNC and still not known to be in NC)

In interactive proofs: deterministic ones give NP, while randomized ones can do PSPACE. Related: checking a proof deterministically requires looking at all the proof, while PCP proofs allow you to check only a constant number of bits.

In Algorithmic Mechanism Design: many randomized truthful approximation mechanisms with no deterministic counterpart.

In Communication complexity: the equality function requires linear communication deterministically but logarithmic (or, depending on exact model, constant) communication randomly.

In decision trees: evaluating an and-or tree requires linear queries deterministically but much less with randomization. This is essentially equivalent to alpha-beta pruning which gives a randomized sub-linear algorithm for game-tree evaluation.

In streaming models, distributed computing models: see previous answers.

Answer 3

Most streaming algorithms

In the streaming model of computation (AMS, book), an algorithm processes an online sequence of updates and is restricted to keep only sublinear space. At any point in time, the algorithm should be able to answer a query.

For many problems there exist sublinear space randomized streaming algorithms while provably no deterministic algorithm can solve the problem in sublinear space. This is related to gaps between randomized and deterministic communication complexity. A simple example is the distinct count problem: at each time step $t$ the algorithm is given an integer $i_t \in [n]$, and it should be able to approximate $D_m = |\{i_t: t = 1\ldots m\}|$, i.e. the number of distinct integers seen up to time step $m$. It's relatively easy to show that any deterministic algorithm achieving constant approximation must use $\Omega(n)$ space (see e.g. lecture notes by Piotr Indyk). On the other hand the clever sampling algorithm of Flajolet and Martin (simple analysis with limited randomness in the AMS paper linked above) achieves constant approximation in $O(\log n)$ bits. The latest work on the problem gives an optimal $O(\frac{1}{\epsilon^2} + \log n)$ algorithm that computes an $1\pm \epsilon$ approximation.

Answer 4

Finding a maximal independent set in a distributed network of $n$ nodes with maximum degree $\Delta$. There's a known lower bound [3] of $\min(\Omega(\log\Delta),\Omega(\sqrt{\log n}))$ that holds for randomized and deterministic algorithms.

The following is a simple randomized distributed algorithm [1] that proceeds in synchronous rounds. (In a round, every node $u$ can perform some local computation and send messages to its neighbors. These messages are guaranteed to be received before the start of the next round.)

1. In every round, each active node $u$ marks itself with probability $1/d_u$ where $d_u>0$ is the degree of $u$; if $d_u=0$, $u$ simply enters the independent set. (Initially, every node is active.)
2. If $u$ is the only marked node in its neighborhood, $u$ enters the independent set, deactivates itself and notifies all of its neighbors to deactivate themselves. The degrees of the remaining active nodes are decreased accordingly, i.e., all edges to deactivated nodes are removed.
3. Otherwise, if there is some neighboring node $v$ that is also marked, the lower degree vertex unmarks itself and remains active.

It can be shown that this algorithm terminates in $O(\log n)$ rounds with high probability, by arguing that half of the remaining edges are deleted in every round. In contrast, the fastest known deterministic distributed algorithm [2] takes $O(n^{1/\sqrt{\log n}})$ rounds and is considerably more complicated.

---

[1] Michael Luby: A Simple Parallel Algorithm for the Maximal Independent Set Problem. SIAM J. Comput. 15(4): 1036-1053 (1986) http://dx.doi.org/10.1137/0215074

[2] Alessandro Panconesi, Aravind Srinivasan: On the Complexity of Distributed Network Decomposition. J. Algorithms 20(2): 356-374 (1996) http://dx.doi.org/10.1006/jagm.1996.0017

[3] Fabian Kuhn, Thomas Moscibroda, Roger Wattenhofer: Local Computation: Lower and Upper Bounds. CoRR abs/1011.5470 (2010) http://arxiv.org/abs/1011.5470

Answer 5

Majority problem in a query model.

Problem. We are given a set of $n$ balls colored with two or more colors. The goal is to find a ball of the majority color (i.e., a color that occurs more than n/2 times) assuming such a color exists, using queries of the form "Does ball $i$ and ball $j$ have the same color?". The strategy must be oblivious, i.e., queries cannot depend on the results of previous queries.

Randomized Algorithm. Pick a random ball and check if more than $n/2$ balls have the same color. This algorithm runs in $O(n)$ time in expectation.

Deterministic $O(n)$ algorithm is quite non-trivial and is a nice application of expanders.

F. R. K. Chung, R. L. Graham, J. Mao, and A. C. Yao, Oblivious and adaptive strategies for the Majority and Plurality problems, Proc. COCOON 2005, pp. 329–338.

Answer 6

Leader Election in an Anonymous Ring of Processes

Suppose that you have a ring network of processes that do not have ids and that communicate by message passing. Initially, every process is in the same state. You want to design a distributed algorithm such that eventually exactly $1$ process enters the elected state and all other processes enter the non-elected state. This is the so called leader election problem which is one of the fundamental symmetry breaking tasks in a distributed system and has many applications.

There's a simple argument (e.g. [1]) that there is no deterministic leader election algorithm for an anonymous ring.

Model: We assume that the computation advances in synchronous rounds where, in each round, every process performs some local computation, sends messages to its neighbors in the ring, and receives messages from its neighbors.

For the sake of a contradiction, let's assume that there is such a deterministic leader election algorithm $A$. It is sufficient to show that, at the start of any round $r\ge 0$, all processes are in the same state, since this implies that there cannot be exactly $1$ process in the elected state. Since processes do not have ids and the network is symmetric, every process is in the same initial state, which provides the induction base.

For the induction step, consider some round $r\ge 0$ and assume that every process is in the same state at the start of round $r$. Therefore, since algorithm $A$ is deterministic, every process performs exactly the same computation and sends exactly the same messages during round $r$. This in turn implies that every process receives exactly the same messages during $r$ and, by the start of round $r+1$, is in the same state. Thus, no such algorithm $A$ can exist.

If $A$ is a randomized algorithm on the other hand and processes know the size of the ring $n$, there's an easy way to break symmetry, by generating a random id from the range $[1,n^4]$, which will result in unique ids for all processes with high probability. A simple and naive algorithm proceeds by letting every process send its id along the ring and instruct processes to forward only messages containing the largest id seen so far. This guarantees that only the process who generated the largest id will receive its own message once it has traversed the entire ring and elect itself as the leader.

---

[1] Dana Angluin: Local and Global Properties in Networks of Processors (Extended Abstract). STOC 1980: 82-93. http://doi.acm.org/10.1145/800141.804655

Answer 7

Finding square roots modulo prime number:

https://en.wikipedia.org/wiki/Tonelli%E2%80%93Shanks_algorithm

Answer 8

Finding a 2-approximate weighted vertex cover is known to be in RNC [1]. I don't think it is known to be in NC. That is, the problem has a randomized poly-log-time poly-processor algorithm, but I don't think it is known to have a deterministic poly-log-time poly-processor algorithm.

[1] https://dx.doi.org/10.1007/s00446-011-0127-7

Answer 9

This problem defined in 1982 by Papadimitriou and Yannakakis:

Exact Perfect Matching problem

Input: A graph G, with each edge colored red or blue, and integer k.

Problem: Decide whether there exists a perfect matching M in G with exactly k red edges

Exact Perfect Matching lies in the complexity class $RP$ since it has a randomized polynomial time algorithm (given by Mulmuley, Vazirani and Vazirani ). Therefor it is unlikely to be $NP$-hard.