Networking Stack

The OSv networking stack originates from FreeBSD as of circa 2013 but has since been heavily modified to implement Van Jacobson's "network channels" design, to reduce the number of locks and lock operations. For more theory and high-level design details please read the "Network Channels" chapter of the OSv paper.

This Wiki instead, focuses on the code and where these design ideas are implemented. It still touches just a tip of "the iceberg" which is the code of the networking stack located mostly under the bsd/ subtree.

Slow Path vs Fast Path

One can see many references in the code to both "fast path" and "slow path". To understand both, one can start looking at this code in virtio-net driver (there is a similar code in the vmxnet3 driver):

void net::receiver()
{
...
  bool fast_path = _ifn->if_classifier.post_packet(m_head);
  if (!fast_path) {
      (*_ifn->if_input)(_ifn, m_head);
  }
...
}

In essence, this code is called to process incoming data (RX) from the network card and it tries to "push" the resulting mbuf via the network channel (fast-path). If that fails it falls back to the if_input from the FreeBSD way of doing things.

The if_classifier, a member of the struct ifnet describing network interface and defined in if_var.h, is an instance of the class classifier. The method post_packet() used in the code above, is part of the 'producer' interface and its role is to identify if mbuf in question has some corresponding net channel and if so push the mbuf on that net channel and wake consumers of the net channel. So the network card driver, virtio-net in this context, is a "producer" of the net channels and threads blocked when calling send, recv or poll are "consumers". An instance of a net channel corresponds to a single TCP connection.

Fast path because directly calls net channel rather than traversing all traditional stack calls - slow path.

Top-down Direction

Bottom-up Direction