Distributed Systems

Section I: The Basics

In this section, we'll learn the basic programming constructs and libraries we need to write distributed systems in OCaml. After this section, we'll know how to

do multiple things at the same time with Async
read and write data to and from the outside world with the Reader and Writer module
send simple messages between processes using TCP
convert OCaml values into bytes and back using various forms of serialization
send more complex messages between processes using TCP and RPC
Persist information to stable storage

Chapter 1: Async

Concurrency and parallelism are at the heart of distributed systems. When programs scale to multiple threads, processes, or machines, we have to reason about multiple computations that are all happening at the same time. In this chapter, we explore Jane Street's monadic cooperative multithreading library: Async. Async is a library that makes writing concurrent code fun and easy! Async will also serve as a building block throughout the rest of the book; all the code we write will involve async. Before we dive into some code, I recommend you read Chapter 18 of Real World Ocaml!

Resources

Chapter 2: Reader/Writer

Whether it be files or TCP connections, programs often interact with the world by reading and writing data to and from an external source. In this chapter, we'll examine Async's Reader and Writer interfaces. These interfaces provide a uniform way to read and write from all sorts of external sources. After this chapter, you'll be able to read strings from stdin, and you'll be a master a file IO. Readers and Writers will also crop up again when we cover TCP, so you'll be prepared for that too.

Resources

Chapter 3: TCP

Distributed systems involve multiple processes running on different computers, different data centers, or even different continents. Despite the geographic separation, the processes can cooperate and function by communicating with one another with networking protocols like UDP and TCP. In this chapter, we'll explore TCP network programming in OCaml.

Resources

TCP API

Chapter 4: Serialization

Resources

Chapter 5: Messaging

Resources

Chapter 6: Persistence

Appendix

The appendix contains miscellaneous programs and snippets on topics that may be useful, but not necessary, to writing distributed systems.

HTTP