What papers on handling errors in distributed systems do you recommend?
You might want to have a look at the works that won Tushar D. Chandra, Vassos Hadzilacos, and Sam Toueg the Edsger W. Dijkstra Prize in 2010:
- Tushar D. Chandra and Sam Toueg. Unreliable Failure Detectors for Reliable Distributed Systems, Journal of the ACM, 43(2):225-267, 1996
- Tushar D. Chandra, Vassos Hadzilacos and Sam Toueg. The Weakest Failure Detector for Solving Consensus, Journal of the ACM, 43(4):685-722, 1996.
These papers introduce the notion of failure detectors in a distributed system in a general and precise framework. Intuitively, they tried to study the minimal amount of failure information needed to solve consensus. It turns out that you don't need a perfect failure detector to solve consensus. Even unreliable failure detectors satisfying certain minimal conditions will suffice for the task. These papers were very influential on how to deal with failures in distributed systems.
What kind of faults in the system?Are you looking for solutions to handling Byzantine faults or just the classic fail-stop model? Solutions in the presence of Byzantine nodes in a distributed system is the more intriguing problem. The problem was formalized by Leslie Lamport(the Byzantine Generals problem' and the 1999 paper by Barbara Liskov and Miguel Castro presents the closest working practical solution ' Practical Byzantine fault-tolerance'. Original formal models to deal with fault-tolerance include the state-machine approach of Fred Schneider and view-stamped replication I do agree the question is very general, the field is immense and the theory forms the basis of most systems running today online. Maybe a more specific fault-model and the problem domain would help obtain better answers
Here is a collection of patterns for dealing handling errors in distributed systems:
- Patterns for Generation, Handling and Management of Errors by Andy Longshaw and Eoin Woods, EuroPlop 2004.
Alternatively, for more generic work, there is the book Introduction to Reliable Distributed Programming by Rachid Guerraoui and Luis Rodrigues, which has a vast range of practical algorithms including failure recovery variants of many. The more classical text Distributed Algorithms by Nancy Lynch covers similar ground from a more theoretical perspective.