Glossary of distributed systems
This is an attempted index of terms used in distributed system protocols.
Machines
- Node: Any machine in the system.
- Process: Originated from distributed protocols in the shared memory model. A process refers to a OS process. NOTE: Do not use unless in the shared memory model.
- Server: A machine that is a part of the protocol in discussion.
- Client: A machine that uses the outputs of the protocol in discussion.
- User: A logical user (may consist of multiple clients) that uses the outputs of the protocol in discussion.
- Replica: This has origins in the literature for crash fault tolerant systems and the primary-backup paradigm of state machine replication. This is a server that can be replicated and take a place of a server just by assigning a new identifier.
- Validator: Originates from the blockchain literature. A validator is any node that validates the blockchain. E.g., a node that downloads the Bitcoin chain and verifies the proof of work. The (incorrect imho) usage in Proof of Stake systems is as the nodes that are a part of the SMR protocol.
State
- Operation: The unit of action on the state of a system. E.g., read, write, pay, readmany, etc.
- Transaction: Originated from the database world. A transaction is a set of operations that a system executes atomically. These operations modify the state. Typically, they are deterministic.
- Commit: Origin - databases. An action on the state where a transaction is accepted as successful and its effects are made visible to users.
Faulty Systems
- Correct: A node is correct if it follows the protocol specification for a period under discussion. This node may be corrupted by the adversary but since it follows the protocol spec., it is deemed to be correct.
- Honest: A node that is under the influence of the adversary until the period under discussion.
- Faulty: Any node that deviates from the protocol specification.
- Non-faulty: All nodes that do not deviate from the protocol specification. Equivalent to Correct nodes.
- BFT: A qualifier for the system. Often abused to mean BFT-SMR.
- CFT: A qualifier for the system. Often abused to mean CFT-SMR or CFT-databases.
- Fail-stop: It is an abstraction of a fault that will stop executing and fail. Note, others can detect the failure. (Fail-Stop Processors: An Approach to Designing Computing Systems. Richard D Schlichting NO_ITEM_DATA:schneiderFailStopProcessorApproach)
- Omission: A fault where some messages can be omitted (by the network or intentionally by the sender)
- Crash: A fault where the node fails irreversibly, i.e., once a node crashes, it cannot come back alive in a protocol.
- Commission: A fault where the node sends messages not specified by the protocol such as equivocating messages, invalid messages, and incorrect round messages.
Protocol Messaging
- Send: A primitive where a message is sent to a neighboring node with a specific name. Implicitly assumes that the sending node knows its neighbors. In a partially connected system, the name must be one of the neighbors, otherwise it is undefined.
- SendAll: A primitive where a message is sent to all neighbors. Implicitly assumes that the sending node knows its neighbors.
- Broadcast: This is a primitive that terminates, and ensures a consistent value is seen by all nodes in the protocol.
Protocol Specification
- Round: Originated from synchronous protocols. A round consists of one send and one wait action by a node. The wait action must terminate when all nodes finish their send action and messages are delivered. In synchronous protocols, a timeout is also a valid wait action.
- Phase: A phase in a protocol changes the way messages are treated. If a message is handled in one way in a phase, in a different phase it should be handled differently. Example: View-change and steady-state phases. The proposals are handled differently in these phases.
- Epoch: Technically, an epoch is a reference point from which some other time is measured. For instance, if you reset some counter, say round counter in a protocol, then this point marks a new epoch. Abused everywhere to mean a super-set of rounds, a time from which a new set of servers are used in a protocol, etc.
Protocols
- Blockchain: Overloaded in the literature. Original meaning is a hash chain of data blocks. Widely used as a reference to cryptocurrency or its underlying agreement protocol. Use SMR, Atomic Broadcast, or the Ledger definitions.
Notes
- I have gathered this after reading countless (does not imply large, just that I have lost count) papers with overloaded meanings.
- As of 2024 May, I have lost all hope for standardization of the terminologies. This document serves to be aware.
- I welcome all feedback at haxolotl dot research at gmail dot com
- Acknowledgements:
- Ittai Abraham