@node Node and Internet Stack
@chapter Node and Internet Stack
@anchor{chap:Node}

This chapter describes how ns-3 nodes are put together, and provides
a walk-through of how packets traverse an internet-based Node.

@float Figure,fig:node
@caption{High-level node architecture.}
@image{figures/node,5in}
@end float

In ns-3, nodes are instances of @code{class Node}.  This class
may be subclassed, but instead, the conceptual model is that 
we @emph{aggregate} or insert objects to it rather than define 
subclasses.

One might think of a bare ns-3 node as a shell of a computer,
to which one may add NetDevices (cards) and other innards including
the protocols and applications.  @ref{fig:node} illustrates
that Node objects contain a list of Applications (initially,
the list is empty), a list of NetDevices (initially, the list
is empty), a unique integer ID, and a system ID (for
distributed simulation).

The design tries to avoid putting too many dependencies on the
base class Node, Application, or NetDevice for the following:
@itemize @bullet
@item IP version, or whether IP is at all even used in the Node.
@item implementation details of the IP stack 
@end itemize 

From a software perspective, the lower interface of applications
corresponds to the C-based sockets API.  The upper interface
of NetDevice objects corresponds to the device independent
sublayer of the Linux stack.  Everything in between can be
aggregated and plumbed together as needed.

Let's look more closely at the protocol demultiplexer.  We want
incoming frames at layer-2 to be delivered to the right layer-3
protocol such as Ipv4.  The
function of this demultiplexer is to register callbacks for
receiving packets.  The callbacks are indexed based on the 
@uref{http://en.wikipedia.org/wiki/EtherType,,EtherType}
in the layer-2 frame.   

Many different types of higher-layer protocols may be 
connected to the NetDevice, such as IPv4, IPv6, ARP,
MPLS, IEEE 802.1x, and packet sockets.  Therefore, the
use of a callback-based demultiplexer avoids the need to
use a common base class for all of these protocols, which
is problematic because of the different types of objects
(including packet sockets) expected to be registered there.

Each NetDevice delivers packets to a callback with the following
signature:
@verbatim
  /**
   * \param device a pointer to the net device which is calling this callback
   * \param packet the packet received
   * \param protocol the 16 bit protocol number associated with this packet.
   *        This protocol number is expected to be the same protocol number
   *        given to the Send method by the user on the sender side.
   * \param address the address of the sender
   * \returns true if the callback could handle the packet successfully, 
   *        false otherwise.
   */
  typedef Callback<bool, Ptr<NetDevice>, Ptr<Packet>, uint16_t,
    const Address &> ReceiveCallback;
@end verbatim

There is a function in class Node that matches that signature:
@verbatim
private:
  bool ReceiveFromDevice (Ptr<NetDevice> device, Ptr<Packet>,
                          uint16_t protocol, const Address &from);
@end verbatim

However, users do not need to access this function directly.  Instead,
when users call @code{uint32_t AddDevice (Ptr<NetDevice> device)}, 
the implementation of this function sets the callback (and the
function returns the ifIndex of the NetDevice on that Node).

But what does the ReceiveFromDevice function do?  Here, it looks
up another callback, in its list of callbacks, corresponding to the
matching EtherType.  This callback is called a ProtocolHandler, and
is specified as follows:
@verbatim
  typedef Callback<void, Ptr<NetDevice>, Ptr<Packet>, uint16_t,
    const Address &> ProtocolHandler;
@end verbatim

Upper-layer protocols or objects are expected to provide such a function.
and register it with the list of ProtocolHandlers by calling
@code{Node::RegisterProtocolHandler ();}
For instance, if Ipv4 is aggregated to a Node, then the Ipv4 receive
function can be registered with the protocol handler by calling:
@verbatim
  RegisterProtocolHandler (
    MakeCallback (&Ipv4L3Protocol::Receive, ipv4), 
    Ipv4L3Protocol::PROT_NUMBER, 0);
@end verbatim
and likewise for Ipv6, Arp, etc.

@section NodeList

Every Node created is automatically added to the ns-3 @code{NodeList}.
The NodeList class provides an @code{Add()} method and C++ iterators
to allow one to walk the node list or fetch a Node pointer by
its integer identifier.

@section Internet stack aggregation

The above @code{class Node} is not very useful as-is; other objects
must be aggregated to it to provide useful node functionality.

The ns-3 source code directory @code{src/internet-stack} provides
implmentation of TCP/IPv4-related components.  These include IPv4,
ARP, UDP, TCP, and other related protocols.

Internet Nodes are not subclasses of class Node; they are simply Nodes 
that have had a bunch of IPv4-related
objects aggregated to them.  They can be put together by hand, or
via a helper function @code{AddInternetStack ()} which does the 
following:
@verbatim
void AddInternetStack (Ptr<Node> node)
{
  // Create layer-3 protocols 
  Ptr<Ipv4L3Protocol> ipv4 = CreateObject<Ipv4L3Protocol> ();
  Ptr<ArpL3Protocol> arp = CreateObject<ArpL3Protocol> ();
  ipv4->SetNode (node);
  arp->SetNode (node);

  // Create an L4 demux 
  Ptr<Ipv4L4Demux> ipv4L4Demux = CreateObject<Ipv4L4Demux> ();

  // Create transport protocols and insert them into the demux
  Ptr<UdpL4Protocol> udp = CreateObject<UdpL4Protocol> ();
  Ptr<TcpL4Protocol> tcp = CreateObject<TcpL4Protocol> ();
  
  ipv4L4Demux->SetNode (node);
  udp->SetNode (node);
  tcp->SetNode (node);
  
  ipv4L4Demux->Insert (udp);
  ipv4L4Demux->Insert (tcp);
  
  // Add factories for instantiating transport protocol sockets
  Ptr<UdpSocketFactoryImpl> udpFactory = CreateObject<UdpSocketFactoryImpl> ();
  Ptr<TcpSocketFactoryImpl> tcpFactory = CreateObject<TcpSocketFactoryImpl> ();
  Ptr<Ipv4Impl> ipv4Impl = CreateObject<Ipv4Impl> ();
  
  udpFactory->SetUdp (udp);
  tcpFactory->SetTcp (tcp);
  ipv4Impl->SetIpv4 (ipv4);
  
  // Aggregate all of these new objects to the node
  node->AggregateObject (ipv4);
  node->AggregateObject (arp);
  node->AggregateObject (ipv4Impl);
  node->AggregateObject (udpFactory);
  node->AggregateObject (tcpFactory);
  node->AggregateObject (ipv4L4Demux);
}
@end verbatim

@subsection Internet Node structure

The Internet Node (an ns-3 Node augmented by aggregation to have one or more
IP stacks) has the following internal structure.

@subsubsection Layer-3 protocols
At the lowest layer, sitting above the NetDevices, are the "layer 3" 
protocols, including IPv4, IPv6, and ARP.  These protocols provide 
the following key methods and data members:

@verbatim
class Ipv4L3Protocol : public Object
{
public: 
  // Add an Ipv4 interface corresponding to the provided NetDevice
  uint32_t AddInterface (Ptr<NetDevice> device);

  // Receive function that can be bound to a callback, for receiving
  // packets up the stack
  void Receive( Ptr<NetDevice> device, Ptr<Packet> p, uint16_t protocol, 
    const Address &from);

  // Higher-level layers call this method to send a packet
  // down the stack to the MAC and PHY layers
  // 
  void Send (Ptr<Packet> packet, Ipv4Address source,
             Ipv4Address destination, uint8_t protocol);

private:
  Ipv4InterfaceList m_interfaces;

  // Protocol handlers
}
@end verbatim
There are many more functions (such as @code{Forward ()}) but we will
focus on the above four items from an architectural perspective.

First, note that the @code{Receive ()} function has a matching signature
to the ReceiveCallback in the @code{class Node}.  This function pointer
is inserted into the Node's protocol handler when 
@code{AddInterface ()} is called.  The actual registration is done
with a statement such as:
follows:
@verbatim
 RegisterProtocolHandler ( MakeCallback (&Ipv4Protocol::Receive, ipv4),
    Ipv4L3Protocol::PROT_NUMBER, 0);
@end verbatim

The Ipv4L3Protocol object is aggregated to the Node; there is only one
such Ipv4L3Protocol object.  Higher-layer protocols that have a packet
to send down to the Ipv4L3Protocol object can call 
@code{GetObject<Ipv4L3Protocol> ()} to obtain a pointer, as follows:
@verbatim
  Ptr<Ipv4L3Protocol> ipv4 = m_node->GetObject<Ipv4L3Protocol> ();
  if (ipv4 != 0)
    {
      ipv4->Send (packet, saddr, daddr, PROT_NUMBER);
    }
@end verbatim

This class nicely demonstrates two techniques we exploit in
ns-3 to bind objects together:  callbacks, and object aggregation.

Once IPv4 has determined that a packet is for the local node, it 
forwards it up the stack.  This is done with the following function:

@verbatim
void
Ipv4L3Protocol::ForwardUp (Ptr<Packet> p, Ipv4Header const&ip,
                           Ptr<Ipv4Interface> incomingInterface)
{
  NS_LOG_FUNCTION (this << p << &ip);

  Ptr<Ipv4L4Demux> demux = m_node->GetObject<Ipv4L4Demux> ();
  Ptr<Ipv4L4Protocol> protocol = demux->GetProtocol (ip.GetProtocol ());
  protocol->Receive (p, ip.GetSource (), ip.GetDestination (), incomingInterface);
}
@end verbatim

The first step is to find the aggregated Ipv4L4Demux object.  Then, this
object is consulted to look up the right Ipv4L4Protocol, based on IP protocol
number.  For instance, TCP is registered in the demux as protocol number 6.
Finally, the @code{Receive()} function on the Ipv4L4Protocol (such as
@code{TcpL4Protocol::Receive} is called.

We have not yet introduced the class Ipv4Interface.  Basically,
each NetDevice is paired with an IPv4 representation of such device.
In Linux, this @code{class Ipv4Interface} roughly corresponds to
the @code{struct in_device}; the main purpose is to provide 
address-family specific information (addresses) about an interface.

@subsubsection Layer-4 protocols and sockets

We next describe how the transport protocols, sockets, and applications
tie together.  In summary, each transport protocol implementation is
a socket factory.  An application that needs a new socket 

For instance, to create a UDP socket, an application would use a code
snippet such as the following:
@verbatim
      Ptr<Udp> udpSocketFactory = GetNode ()->GetObject<Udp> ();
      Ptr<Socket> m_socket = socketFactory->CreateSocket ();
      m_socket->Bind (m_local_address);
      ...
@end verbatim 
The above will query the node to get a pointer to its UDP socket
factory, will create one such socket, and will use the socket with
an API similar to the C-based sockets API, such as @code{Connect ()}
and @code{Send ()}.  See the chapter on ns-3 sockets for more information.  

We have described so far a socket factory (e.g. @code{class Udp}) and
a socket, which may be specialized (e.g., @code{class UdpSocket}).
There are a few more key objects that relate to the specialized
task of demultiplexing a packet to one or more receiving sockets.
The key object in this task is @code{class Ipv4EndPointDemux}.
This demultiplexer stores objects of @code{class Ipv4EndPoint}.
This class holds the addressing/port tuple (local port, local address, 
destination port, destination address) associated with the socket, 
and a receive callback.  This receive callback has a receive
function registered by the socket.  The @code{Lookup ()} function to
Ipv4EndPointDemux returns a list of Ipv4EndPoint objects (there may
be a list since more than one socket may match the packet).  The
layer-4 protocol copies the packet to each Ipv4EndPoint and calls
its @code{ForwardUp ()} method, which then calls the @code{Receive ()}
function registered by the socket.

An issue that arises when working with the sockets API on real
systems is the need to manage the reading from a socket, using 
some type of I/O (e.g., blocking, non-blocking, asynchronous, ...).
ns-3 implements an asynchronous model for socket I/O; the application
sets a callback to be notified of received data ready to be read, and the 
callback is invoked by the transport protocol when data is available.
This callback is specified as follows:
@verbatim
  void Socket::SetRecvCallback (Callback<void, Ptr<Socket>, 
    Ptr<Packet>, const Address&> receivedData);
@end verbatim
The data being received is conveyed in the Packet data buffer.  An example
usage is in @code{class PacketSink}:
@verbatim
  m_socket->SetRecvCallback (MakeCallback(&PacketSink::HandleRead, this));
@end verbatim

To summarize, internally, the UDP implementation is organized as follows:
@itemize @bullet
@item a @code{UdpImpl} class that implements the Udp socket factory
functionality
@item a @code{UdpL4Protocol} class that implements the protocol logic
that is socket-independent 
@item a @code{UdpSocketImpl} class that implements socket-specific aspects
of UDP
@item a class called @code{Ipv4EndPoint} that stores the
addressing tuple (local port, local address, destination port, destination
address) associated with the socket, and a receive callback for the socket.
@end itemize

@subsection Internet Node interfaces

Many of the implementation details, or internal objects themselves, 
of Internet Node objects are not exposed at the simulator public
API.  This allows for different implementations; for instance, 
replacing the native ns-3 models with ported TCP/IP stack code. 
 
The C++ public APIs of all of these objects is found in the
@code{src/node} directory, including principally:
@itemize @bullet
@item @code{socket.h}
@item @code{tcp.h}
@item @code{udp.h}
@item @code{ipv4.h}
@end itemize 
These are typically base class objects that implement the default
values used in the implementation, implement access methods to get/set
state variables, host attributes, and implement publicly-available methods 
exposed to clients such as @code{CreateSocket}.

@subsection Example path of a packet

These two figures show an example stack trace of how packets flow
through the Internet Node objects.

@float Figure,fig:internet-node-send
@caption{Send path of a packet.}
@image{figures/internet-node-send,5in}
@end float

@float Figure,fig:internet-node-recv
@caption{Receive path of a packet.}
@image{figures/internet-node-recv,5in}
@end float