-
Notifications
You must be signed in to change notification settings - Fork 17
Protocol Implementations
This document provides an overview of the fundamental aspects shared by all protocol implementations in Holo. For protocol-specific details, please refer to the corresponding documentation available in the source code.
In Holo, code simplicity is a major goal. All network protocols operate by exchanging packets and reacting to them. To maintain simplicity, it's advantageous to separate packet encoding and decoding logic from the main protocol logic. This separation is achieved by creating Rust structs and enums to represent network packets, with encode() and decode() methods implemented on them.
Apart from simplicity, encapsulating packet encoding and decoding functionality in a separate module greatly facilitates the implementation of unit tests and benchmarks. Testing these code paths using unit tests is much simpler than testing with a running protocol instance. Unit tests offer the ability to precisely define which packets should be deemed malformed and pinpoint the exact reasons why. Ideally, unit tests should strive to achieve 100% code coverage for the packet module, serving as a robust defense against regressions.
When defining structs to represent network packets and their inner contents, such as TLVs, prioritize high-level types. For instance, opt for Ipv4Addr over u32 to store an IPv4 address. Similarly, use Option<Ipv4Addr> instead of
Ipv4Addr::UNSPECIFIED to denote the absence of an address. The encode() and decode() methods should handle the translation between the internal representation of packets and the network format.
Validation checks within decode() methods should primarily focus on syntactic errors, like missing or malformed data. Complex semantic validations, such as validating OSPF Hello fields, should be part of the protocol logic.
For organization, it's recommended to have a file named consts.rs within the packet module, containing all constants defined in the protocol specifications, with links referring to corresponding IANA registries when applicable. Here's an example:
// BGP Message Types.
//
// IANA registry:
// https://www.iana.org/assignments/bgp-parameters/bgp-parameters.xhtml#bgp-parameters-1
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
#[derive(FromPrimitive, ToPrimitive)]
#[derive(Deserialize, Serialize)]
pub enum MessageType {
Open = 1,
Update = 2,
Notification = 3,
Keepalive = 4,
// RFC 2918
RouteRefresh = 5,
}Packet fields containing protocol flags should be represented using the bitflags macro. Here's an example:
// The PrefixOptions Field.
//
// IANA registry:
// https://www.iana.org/assignments/ospfv3-parameters/ospfv3-parameters.xhtml#ospfv3-parameters-4
bitflags! {
#[derive(Clone, Copy, Debug, Default, Eq, PartialEq)]
#[derive(Deserialize, Serialize)]
#[serde(transparent)]
pub struct PrefixOptions: u8 {
const NU = 0x01;
const LA = 0x02;
const P = 0x08;
const DN = 0x10;
const N = 0x20;
}
}For representing fields that can contain multiple entries, it's usually advisable to use a BTreeSet instead of a Vec to store the elements, as BTreeSet naturally eliminates duplicate entries and orders the elements. However, in cases where preserving the order of elements is necessary, such as in the BGP AS_PATH attribute, using Vec or VecDeque is the appropriate choice.
In some cases, the encoding and decoding of packets depend on external context. For instance, decoding of BGP attributes depends on whether the peer is an external or internal session to determine which attributes are valid. In such cases, it's recommended to add EncodeCxt and DecodeCxt structs containing the necessary context information and pass those structs by reference to the encode() and decode() methods. Here's an example:
// BGP message decoding context.
pub struct EncodeCxt {
pub capabilities: BTreeSet<NegotiatedCapability>,
}
// BGP message decoding context.
pub struct DecodeCxt {
pub peer_type: PeerType,
pub peer_as: u32,
pub capabilities: BTreeSet<NegotiatedCapability>,
}
impl Message {
// Encodes BGP message into a bytes buffer.
pub fn encode(&self, cxt: &EncodeCxt) -> Bytes { ... }
// Decode buffer into a BGP message.
pub fn decode(data: &[u8], cxt: &DecodeCxt) -> DecodeResult<Self> { ... }
}Finally, the packet module should also contain a DecodeError type containing all potential errors that may occur during packet decoding. Each error type should encapsulate sufficient data for accurate logging and for the protocol crate to react appropriately to the error.
For protocols with extremely simple packet formats, the packet module can be condensed to a single file, as seen in the holo-bfd crate.
The network module contains primitives used for network communication. Primarily, it comprises functions for socket creation, initialization, and operations for reading from and writing to the network.
Given Holo's adherence to the Sans-IO paradigm in protocol implementations, the network module serves as the bridge between protocol logic and the underlying operating system. By encapsulating all I/O-related functionalities within this module, the process of adapting protocols to run on different operating systems is greatly facilitated.
Typically, the network module can be implemented within a single file, bearing in mind that in Rust, files are treated as modules themselves. However, for protocols with more complex network demands, like LDP, which uses both UDP and TCP, it's advisable to organize the network-related code across multiple files to enhance clarity and maintainability.
The northbound module contains code associated with the management aspects of the protocol. This includes configuration handling, state retrieval, RPCs, and notifications. For detailed information, please refer to the Northbound Framework page.
The southbound module contains functions used for communication with various internal components via the internal bus (ibus). It comprises two main files: tx.rs, which handles the transmission of ibus messages, and rx.rs, dedicated to processing incoming ibus messages.
For instance, routing protocols communicate with holo-interface to retrieve interface statuses from the system and communicate with holo-routing to install routes and manage route redistribution. Additionally, some protocols exchange messages concerning BFD peer registration and updates, recursive nexthop tracking, MPLS route updates, and more.
Below is a list of files that should be present in every protocol implementation.
The lib.rs file serves as the entry point for the protocol crate. It is responsible for importing all modules and declaring any unstable compiler features used within the protocol. It's important to note that crate-level lint settings should not be defined in this file; instead, they should be specified in the root Cargo.toml file.
This file contains the the Instance structure, which contains all information related to the protocol instance. For all protocols, the Instance structure contains at least the following fields:
#[derive(Debug)]
pub struct Instance {
// Instance name.
pub name: String,
// Instance system data.
pub system: InstanceSys,
// Instance configuration data.
pub config: InstanceCfg,
// Instance state data.
pub state: Option<InstanceState>,
// Instance Tx channels.
pub tx: InstanceChannelsTx<Instance>,
// Shared data.
pub shared: InstanceShared,
}Field Explanations:
-
system: contains system information retrieved via ibus message exchanges. Examples include the system auto-generated Router ID and interface addresses. -
config: contains the instance configuration. -
state: contains the instance state information, which is only available when the instance is active. An instance can be inactive, for instance, if it's explicitly disable via configuration, or if it's missing a Router ID. -
tx: contains the transmit end of various channels used for message exchanges. This include northbound, ibus and internal protocol channels. -
shared: contains data that is shared among all protocol instances. This includes access to non-volatile storage, global MPLS Label Manager and system-wide configuration.
In addition to these fields, the Instance structure often maintains lists of objects like interfaces or neighbors, each potentially with their own system, config and state fields.
On the Instance structure, helper methods like start() and stop() are often required to implement instance activation and deactivation. Additionally, the Instance structure must implement the ProtocolInstance trait.
This trait, from the holo-protocol crate, defines common functions that need to be implemented by all protocols. By leveraging this trait, functions shared among protocol implementations are centralized within holo-protocol, preventing unnecessary code duplication. This includes protocol initialization and the main event loop.
As with all routing protocol stacks, Holo adheres to the event loop design pattern. Following initialization, each protocol simply awaits and processes events. These events primarily comprise received messages and timer expirations.
For a better understanding of a routing protocol's implementation, it's best to begin by examining all potential events and their corresponding processing. In most routing protocol implementations, event handling involves registering callbacks for each event, with these callbacks scattered across multiple files. When dealing with a sizable codebase, it's easy to get lost, as event handling is mixed with supportive code spread across numerous files. However, in Holo, the events.rs file of each protocol contains the handling of all conceivable protocol events. This approach aids in understanding a protocol's implementation since its entry points are clearly defined within a single file.
There are no clear and fast rules regarding which code should reside in events.rs and which should be relegated to other files as supportive code. However, as a general principle, event processing code should strive to be as high-level as possible. Larger algorithms, such as SPF or the BGP best-path computation, are best implemented in separate files and then invoked by events.rs.
As previously discussed, protocols are implemented using the actor model. Within this file, all input and output messages exchanged between the main instance task and its child tasks are defined. Input tasks denote tasks that send messages to the main instance task, while output tasks denote tasks that receive messages from the main instance task.
In addition to message definitions, this file also has functions responsible for spawning all child tasks. These tasks are categorized as follows:
- I/O tasks (
Task::spawn()): Primarily dedicated to networking I/O operations, such as packet transmission and reception. - Timer tasks (
IntervalTask::new()): Perform a specified action once a timeout expires. - Interval tasks (
TimeoutTask::new()): Perform a specified action whenever the specified interval timer ticks. Commonly used for sending protocol keepalive messages at regular intervals. - Blocking tasks (
Task::spawn_blocking()): Used for potentially blocking operations, such as CPU-intensive algorithms.
Though initiating an asynchronous task for a single timeout may seem excessive, these tasks are exceptionally lightweight. Their overhead is minimal, akin to setting a timer using event libraries like libev in C.
In the error.rs file, the Error enum should be defined to contain all possible protocol errors. Manual implementation of the std::fmt::Display and std::error::Error traits is necessary, as Holo doesn't use any external dependency for error handling. For improved code organization and maintainability, consider breaking down the Error enum into smaller error types whenever that makes sense. For instance, most protocol implementations contain an IoError type containing all possible I/O errors.
Similarly, within the debug.rs file, include a Debug enum containing all possible debug messages, along with a corresponding log() method. Centralizing debug messages in one location, rather than scattering them throughout the codebase using the debug! macro, promotes clearer code and better separation of user-facing messages from core logic.
Some protocols, such as OSPF and RIP, have different versions that are widely deployed, typically one for IPv4 and another for IPv6. Holo leverages Rust's generics to have version-agnostic protocol implementations, where most of the code is shared by the different protocol versions. This approach reduces the maintenance cost of these protocols and facilitates shipping new features that benefit all protocol versions.
For protocol implementations supporting multiple versions, a file named version.rs should be present. Within this file, a trait named Version is defined, encompassing all version-specific protocol code, including constants, types, and functions. Typically, the Version trait has multiple supertraits related to specialized functionalities, such as NetworkVersion and PacketVersion. Unit structs are then defined for each protocol version, accompanied by corresponding implementations of the Version trait. Below is an example excerpt from holo-rip:
#[derive(Debug, Default)]
pub struct Ripv2();
#[derive(Debug, Default)]
pub struct Ripng();
// ===== impl Ripv2 =====
impl Version for Ripv2 {
const PROTOCOL: Protocol = Protocol::RIPV2;
const ADDRESS_FAMILY: AddressFamily = AddressFamily::Ipv4;
type IpAddr = Ipv4Addr;
type IpNetwork = Ipv4Network;
[snip]
}
// ===== impl Ripng =====
impl Version for Ripng {
const PROTOCOL: Protocol = Protocol::RIPNG;
const ADDRESS_FAMILY: AddressFamily = AddressFamily::Ipv6;
type IpAddr = Ipv6Addr;
type IpNetwork = Ipv6Network;
[snip]
}Additionally, separate directories should be created for each protocol version, such as holo-rip/src/ripv2 and holo-rip/src/ripng, where the Version supertraits are implemented for each respective version.
By abstracting version-specific protocol logic within the Version trait, it's possible to write version-agnostic protocol code that works across all protocol versions. For example, the OSPFv2 and OSPFv3 implementations share roughly 80% of the same code, highlighting the significant reduction in code duplication by avoiding separate implementations.
Testing is essential to ensure adherence to standards and guard against regressions. In Holo, testing occurs across various levels. This includes unit tests for the packet module, as described in the packet section, as well as topology and conformance tests described in the Data Driven Testing page.
Benchmarks are located within the benches directory and should be declared in the crate's Cargo.toml file.
The criterion crate serves as a reliable tool for profiling the code, enabling the detection of performance regressions and generating graphs for in-depth analysis. Presently, most protocols feature benchmarks for both packet encoding and packet decoding. Looking ahead, owing to the modular architecture of protocol implementations, there's potential to expand benchmarking to encompass CPU-intensive algorithms like SPF computations and BGP policy application.
TODO
TODO
- Architecture
- Management Interfaces
- Developer's Documentation
- Example Topology
- Paul's Practical Guide