Distributed Systems
Section I: The Basics
In this section, we'll learn the basic programming constructs and libraries we
need to write distributed systems in OCaml. After this section, we'll know how
to
- do multiple things at the same time with Async
- read and write data to and from the outside world with the Reader and
Writer module
- send simple messages between processes using TCP
- convert OCaml values into bytes and back using various forms of
serialization
- send more complex messages between processes using TCP and RPC
- Persist information to stable storage
Chapter 1: Async
Concurrency and parallelism are at the heart of distributed systems. When
programs scale to multiple threads, processes, or machines, we have to reason
about multiple computations that are all happening at the same time. In this
chapter, we explore Jane Street's monadic cooperative multithreading library:
Async. Async is a library that makes writing concurrent code fun and easy!
Async will also serve as a building block throughout the rest of the book; all
the code we write will involve async. Before we dive into some code, I
recommend you read
Chapter 18 of Real World Ocaml!
-
HelloWrong
-
HelloRight
-
HelloCommand
-
Basics
-
Choices
-
Pipes
-
Ivars
-
MoreChoices
-
MorePipes
Resources
Chapter 2: Reader/Writer
Whether it be files or TCP connections, programs often interact with the world
by reading and writing data to and from an external source. In this chapter,
we'll examine Async's Reader and Writer interfaces. These interfaces provide a
uniform way to read and write from all sorts of external sources. After this
chapter, you'll be able to read strings from stdin, and you'll be a master a
file IO. Readers and Writers will also crop up again when we cover TCP, so
you'll be prepared for that too.
-
StdinLine
-
StdoutLine
-
StdinAll
-
WriteString
-
WriteLine
-
WriteWith
-
Diary
-
Cat
Resources
Chapter 3: TCP
Distributed systems involve multiple processes running on different computers,
different data centers, or even different continents. Despite the geographic
separation, the processes can cooperate and function by communicating with one
another with networking protocols like UDP and TCP. In this chapter, we'll
explore TCP network programming in OCaml.
-
PrintServer
-
EchoOnceServer
-
EchoServer
-
ExitEchoServer
-
PrintClient
-
EchoClient
-
TcpShell
-
KeyValueStore
-
RedisOp
-
Redis
Resources
Chapter 4: Serialization
-
HandCard
-
SexpCard
-
MarshalCard
-
BinioCard
Resources
Chapter 5: Messaging
-
ArithMarshal
-
MarshalClient
-
MarshalServer
-
ArithSexp
-
SexpClient
-
SexpServer
-
ToStringProtocol
-
ToStringClient
-
ToStringServer
-
ArithRpc
-
RpcClient
-
RpcServer
Resources
Chapter 6: Persistence
-
Ephemeral
-
Fsync
-
LevelDb
-
MoreLevelDb
Appendix
The appendix contains miscellaneous programs and snippets on topics that may be
useful, but not necessary, to writing distributed systems.
HTTP
-
HelloServer
-
HelloGetServer
-
HelloHtmlServer