F5/unison
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fd-net-device: Add DpdkNetDeviceHelper

Browse Source 
This commit:
	- Add DpdkNetDeviceHelper
	- Encapsulate EAL arguments setup in DpdkNetDeviceHelper
	- Update Documentation

Co-authored-by: Harsh Patel <thadodaharsh10@gmail.com>
Co-authored-by: Mohit P. Tahiliani <tahiliani@nitk.edu.in>
This commit is contained in:
Hrishikesh Hiraskar
2020-09-24 13:07:19 +00:00
committed by Tom Henderson
parent fc9f640628
commit dd8faec88f
10 changed files with 223 additions and 152 deletions
Download Patch File Download Diff File Expand all files Collapse all files

50
src/fd-net-device/doc/dpdk-net-device.rst
View File

@@ -30,7 +30,7 @@ Design
Initialization
##############
``EmuFdNetDeviceHelper`` model is responsible for the initialization of ``DpdkNetDevice``. It contains a boolean flag named ``isDpdkMode`` which upon setting, initializes the ``DpdkNetDevice``. After this, the EAL is initialized, a memory pool is allocated, access to the Ethernet port is obtained and it is initialized, reception (Rx) and transmission (Tx) queues are set up on the port, Rx and Tx buffers are set up and LaunchCore method is called which will launch the ``HandleRx`` method to handle reading of packets in burst.
``DpdkNetDeviceHelper`` model is responsible for the initialization of ``DpdkNetDevice``. After this, the EAL is initialized, a memory pool is allocated, access to the Ethernet port is obtained and it is initialized, reception (Rx) and transmission (Tx) queues are set up on the port, Rx and Tx buffers are set up and LaunchCore method is called which will launch the ``HandleRx`` method to handle reading of packets in burst.
Packet Transfer
###############
@@ -230,10 +230,7 @@ it is found, a user should see:
DPDK NetDevice : enabled
``DpdkNetDevice`` does not use a dedicated helper class, but reuses the
``EmuFdNetDeviceHelper`` class provided in the ``fd-net-device/helper``
directory. The ``EmuFdNetDeviceHelper`` allows us to set ``dpdkMode``
which will launch the ``DpdkNetDevice`` instead of the ``FdNetDevice``.
``DpdkNetDeviceHelper`` class supports the configuration of ``DpdkNetDevice``.
.. sourcecode:: text
@@ -262,42 +259,19 @@ which will launch the ``DpdkNetDevice`` instead of the ``FdNetDevice``.
+-------------- ( Internet ) ----
Initialization of DPDK requires passing of EAL arguments. These are passed to ``EmuFdNetDevice::SetDpdkMode``, which initializes the ``DPDKNetDevice``. The ``deviceName`` variable is the PCI address of the ethernet controller (for eg. ``0000:00:1f.6``), which can be obtained by ``lspci``. The following code snippet shows the whole procedure:
Initialization of DPDK driver requires initialization of EAL. EAL requires PMD (Poll Mode Driver) Library for using NIC. DPDK supports multiple Poll Mode Drivers and you can use one that works for your NIC. PMD Library can be set via ``DpdkNetDeviceHelper::SetPmdLibrary``, as follows:
.. sourcecode:: text
char **ealArgv = new char*[20];
// arg[0] is program name (optional)
ealArgv[0] = new char[20];
strcpy (ealArgv[0], "");
// logical core usage
ealArgv[1] = new char[20];
strcpy (ealArgv[1], "-l");
// Use core 0 and 1
ealArgv[2] = new char[20];
strcpy (ealArgv[2], "0,1");
// Load library
ealArgv[3] = new char[20];
strcpy (ealArgv[3], "-d");
// Use e1000 driver library (this is for IGb PMD supproting Intel 1GbE NIC)
// NOTE: DPDK supports multiple Poll Mode Drivers (PMDs) and you can use it
// based on your NIC. You just need to add it as a library using -d option as
// used below.
ealArgv[4] = new char[20];
strcpy (ealArgv[4], "librte_pmd_e1000.so");
// Load library
ealArgv[5] = new char[20];
strcpy (ealArgv[5], "-d");
// Use mempool ring library
ealArgv[6] = new char[50];
strcpy (ealArgv[6], "librte_mempool_ring.so");
EmuFdNetDeviceHelper emu;
emu.SetDpdkMode(7, ealArgv);
emu.SetDeviceName(deviceName);
NetDeviceContainer devices = emu.Install(node);
Ptr<NetDevice> device = devices.Get(0);
device->SetAttribute("Address", Mac48AddressValue(Mac48Address::Allocate ()));
DpdkNetDeviceHelper* dpdk = new DpdkNetDeviceHelper ();
dpdk->SetPmdLibrary("librte_pmd_e1000.so");
Also, NIC should be bound to DPDK Driver in order to be used with EAL. The default driver used is ``uio_pci_generic`` which supports most of the NICs. You can change it using ``DpdkNetDeviceHelper::SetDpdkDriver``, as follows:
.. sourcecode:: text
DpdkNetDeviceHelper* dpdk = new DpdkNetDeviceHelper ();
dpdk->SetDpdkDriver("igb_uio");
Attributes
==========
@@ -323,5 +297,5 @@ Examples
The following examples are provided:
* ``fd-emu-ping.cc``: This example can be configured to use the EmuFdNetDeviceHelper in DPDK mode to send ICMP traffic bypassing the kernel over a real channel.
* ``fd-emu-ping.cc``: This example can be configured to use the ``DpdkNetDevice`` to send ICMP traffic bypassing the kernel over a real channel.
* ``fd-emu-onoff.cc``: This example can be configured to measure the throughput of the ``DpdkNetDevice`` by sending traffic from the simulated node to a real device using the ``ns3::OnOffApplication`` while leveraging DPDK’s fast packet processing abilities. This is achieved by saturating the channel with TCP/UDP traffic.

34
src/fd-net-device/examples/fd-emu-onoff.cc
View File

@@ -124,7 +124,7 @@ main (int argc, char *argv[])
{
uint16_t sinkPort = 8000;
uint32_t packetSize = 1400; // bytes
std::string dataRate("1000Mb/s");
std::string dataRate ("1000Mb/s");
bool serverMode = false;
std::string deviceName ("eth0");
@@ -215,34 +215,14 @@ main (int argc, char *argv[])
#ifdef HAVE_DPDK_USER_H
if (emuMode == "dpdk")
{
EmuFdNetDeviceHelper* dpdk = new EmuFdNetDeviceHelper;
// set the dpdk emulation mode
char **ealArgv = new char*[20];
// arg[0] is program name (optional)
ealArgv[0] = new char[20];
strcpy (ealArgv[0], "");
// logical core usage
ealArgv[1] = new char[20];
strcpy (ealArgv[1], "-l");
// Use core 0 and 1
ealArgv[2] = new char[20];
strcpy (ealArgv[2], "0,1");
// Load library
ealArgv[3] = new char[20];
strcpy (ealArgv[3], "-d");
DpdkNetDeviceHelper* dpdk = new DpdkNetDeviceHelper ();
// Use e1000 driver library (this is for IGb PMD supproting Intel 1GbE NIC)
// NOTE: DPDK supports multiple Poll Mode Drivers (PMDs) and you can use it
// based on your NIC. You just need to add it as a library using -d option as
// used below.
ealArgv[4] = new char[20];
strcpy (ealArgv[4], "librte_pmd_e1000.so");
// Load library
ealArgv[5] = new char[20];
strcpy (ealArgv[5], "-d");
// Use mempool ring library
ealArgv[6] = new char[50];
strcpy (ealArgv[6], "librte_mempool_ring.so");
dpdk->SetDpdkMode (7, ealArgv);
// based on your NIC. You just need to set pmd library as follows:
dpdk->SetPmdLibrary ("librte_pmd_e1000.so");
// Set dpdk driver to use for the NIC. `uio_pci_generic` supports most NICs.
dpdk->SetDpdkDriver ("uio_pci_generic");
// Set device name
dpdk->SetDeviceName (deviceName);
helper = dpdk;
}

36
src/fd-net-device/examples/fd-emu-ping.cc
View File

@@ -48,7 +48,7 @@
// 1) You need to decide on a physical device on your real system, and either
// overwrite the hard-configured device name below (eth0) or pass this
// device name in as a command-line argument
// 1') If you run emulation in dpdk mode, use device address (eg. 0000:00.1f.6)
// 1') If you run emulation in dpdk mode, use device address (eg. 0000:00.1f.6)
// as device name. This address can be obtained by running `lspci`
// 2) The host device must be set to promiscuous mode
// (e.g. "sudo ifconfig eth0 promisc")
@@ -74,7 +74,7 @@
//
// $ sudo chown root.root build/src/fd-net-device/ns3-dev-raw-sock-creator
// $ sudo chmod 4755 build/src/fd-net-device/ns3-dev-raw-sock-creator
//
//
// or (if you run emulation in netmap mode):
// $ sudo chown root.root build/src/fd-net-device/ns3-dev-netmap-device-creator
// $ sudo chmod 4755 build/src/fd-net-device/ns3-dev-netmap-device-creator
@@ -197,34 +197,14 @@ main (int argc, char *argv[])
#ifdef HAVE_DPDK_USER_H
if (emuMode == "dpdk")
{
EmuFdNetDeviceHelper* dpdk = new EmuFdNetDeviceHelper;
// set the dpdk emulation mode
char **ealArgv = new char*[20];
// arg[0] is program name (optional)
ealArgv[0] = new char[20];
strcpy (ealArgv[0], "");
// logical core usage
ealArgv[1] = new char[20];
strcpy (ealArgv[1], "-l");
// Use core 0 and 1
ealArgv[2] = new char[20];
strcpy (ealArgv[2], "0,1");
// Load library
ealArgv[3] = new char[20];
strcpy (ealArgv[3], "-d");
DpdkNetDeviceHelper* dpdk = new DpdkNetDeviceHelper ();
// Use e1000 driver library (this is for IGb PMD supproting Intel 1GbE NIC)
// NOTE: DPDK supports multiple Poll Mode Drivers (PMDs) and you can use it
// based on your NIC. You just need to add it as a library using -d option as
// used below.
ealArgv[4] = new char[20];
strcpy (ealArgv[4], "librte_pmd_e1000.so");
// Load library
ealArgv[5] = new char[20];
strcpy (ealArgv[5], "-d");
// Use mempool ring library
ealArgv[6] = new char[50];
strcpy (ealArgv[6], "librte_mempool_ring.so");
dpdk->SetDpdkMode (7, ealArgv);
// based on your NIC. You just need to set pmd library as follows:
dpdk->SetPmdLibrary ("librte_pmd_e1000.so");
// Set dpdk driver to use for the NIC. `uio_pci_generic` supports most NICs.
dpdk->SetDpdkDriver ("uio_pci_generic");
// Set device name
dpdk->SetDeviceName (deviceName);
helper = dpdk;
}

91
src/fd-net-device/helper/dpdk-net-device-helper.cc Normal file
View File

@@ -0,0 +1,91 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2020 NITK Surathkal
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation;
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include "dpdk-net-device-helper.h"
#include "ns3/dpdk-net-device.h"
namespace ns3 {
NS_LOG_COMPONENT_DEFINE ("DpdkNetDeviceHelper");
DpdkNetDeviceHelper::DpdkNetDeviceHelper ()
: m_lCoreList ("0,1"),
m_pmdLibrary ("librte_pmd_e1000.so"),
m_dpdkDriver ("uio_pci_generic")
{
NS_LOG_FUNCTION (this);
SetTypeId ("ns3::DpdkNetDevice");
}
void
DpdkNetDeviceHelper::SetLCoreList (std::string lCoreList)
{
m_lCoreList = lCoreList;
}
void
DpdkNetDeviceHelper::SetPmdLibrary (std::string pmdLibrary)
{
m_pmdLibrary = pmdLibrary;
}
void
DpdkNetDeviceHelper::SetDpdkDriver (std::string dpdkDriver)
{
m_dpdkDriver = dpdkDriver;
}
Ptr<NetDevice>
DpdkNetDeviceHelper::InstallPriv (Ptr<Node> node) const
{
NS_LOG_FUNCTION (this);
Ptr<NetDevice> device = FdNetDeviceHelper::InstallPriv (node);
Ptr<DpdkNetDevice> dpdkDevice = DynamicCast<DpdkNetDevice> (device);
dpdkDevice->SetDeviceName (m_deviceName);
// set eal args
char **ealArgv = new char*[20];
// arg[0] is program name (optional)
ealArgv[0] = new char[20];
strcpy (ealArgv[0], "");
// logical core usage
ealArgv[1] = new char[20];
strcpy (ealArgv[1], "-l");
ealArgv[2] = new char[20];
strcpy (ealArgv[2], m_lCoreList.c_str ());
// Load library
ealArgv[3] = new char[20];
strcpy (ealArgv[3], "-d");
ealArgv[4] = new char[50];
strcpy (ealArgv[4], m_pmdLibrary.c_str ());
// Use mempool ring library
ealArgv[5] = new char[20];
strcpy (ealArgv[5], "-d");
ealArgv[6] = new char[50];
strcpy (ealArgv[6], "librte_mempool_ring.so");
dpdkDevice->InitDpdk (7, ealArgv, m_dpdkDriver);
return dpdkDevice;
}
} // namespace ns3

96
src/fd-net-device/helper/dpdk-net-device-helper.h Normal file
View File

@@ -0,0 +1,96 @@
/* -*- Mode:C++; c-file-style:"gnu"; indent-tabs-mode:nil; -*- */
/*
* Copyright (c) 2020 NITK Surathkal
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation;
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#ifndef DPDK_NET_DEVICE_HELPER_H
#define DPDK_NET_DEVICE_HELPER_H
#include "ns3/emu-fd-net-device-helper.h"
namespace ns3 {
/**
* \ingroup dpdk-net-device
* \brief build a DpdkNetDevice object attached to a physical network
* interface
*
*/
class DpdkNetDeviceHelper : public EmuFdNetDeviceHelper
{
public:
/**
* Construct a DpdkNetDeviceHelper and initialize DPDK EAL
*
* \param argc Dpdk EAL args count.
* \param argv Dpdk EAL args list.
*/
DpdkNetDeviceHelper ();
virtual ~DpdkNetDeviceHelper ()
{}
/**
* Sets list of logical cores to use
*
* \param lCoreList Comma seperated logical core list. eg: "0,1"
*/
void SetLCoreList (std::string lCoreList);
/**
* Sets PMD Library to be used for the NIC
*
* \param pmdLibrary The PMD Library
*/
void SetPmdLibrary (std::string pmdLibrary);
/**
* Sets DPDK Driver to bind NIC to
*
* \param dpdkDriver The DPDK Driver
*/
void SetDpdkDriver (std::string dpdkDriver);
protected:
/**
* This method creates an ns3::FdNetDevice attached to a physical network
* interface
*
* \param node The node to install the device in
* \returns A container holding the added net device.
*/
Ptr<NetDevice> InstallPriv (Ptr<Node> node) const;
/**
* Logical cores to use
*/
std::string m_lCoreList;
/**
* PMD Library to use for NIC
*/
std::string m_pmdLibrary;
/**
* DPDK Driver to bind NIC to
*/
std::string m_dpdkDriver;
};
} // namespace ns3
#endif /* DPDK_NET_DEVICE_HELPER_H */

31
src/fd-net-device/helper/emu-fd-net-device-helper.cc
View File

@@ -29,10 +29,6 @@
#include "ns3/simulator.h"
#include "ns3/trace-helper.h"
#ifdef HAVE_DPDK_USER_H
#include "ns3/dpdk-net-device.h"
#endif
#include <arpa/inet.h>
#include <errno.h>
#include <iostream>
@@ -67,7 +63,6 @@ EmuFdNetDeviceHelper::EmuFdNetDeviceHelper ()
{
m_deviceName = "undefined";
m_hostQdiscBypass = false;
m_dpdkMode = false;
}
void
@@ -81,22 +76,6 @@ EmuFdNetDeviceHelper::HostQdiscBypass (bool hostQdiscBypass)
{
m_hostQdiscBypass = hostQdiscBypass;
}
void
EmuFdNetDeviceHelper::SetDpdkMode (int argc, char **argv)
{
NS_LOG_FUNCTION (this);
#ifdef HAVE_DPDK_USER_H
FdNetDeviceHelper::SetTypeId ("ns3::DpdkNetDevice");
m_dpdkMode = true;
m_ealArgc = argc;
m_ealArgv = argv;
NS_LOG_LOGIC ("Set DPDK Mode");
#else
NS_FATAL_ERROR ("EmuFdNetDeviceHelper::SetDpdkMode (): Attempted to set DPDK Mode without DPDK support enabled");
#endif
}
std::string
EmuFdNetDeviceHelper::GetDeviceName (void)
@@ -123,16 +102,6 @@ EmuFdNetDeviceHelper::SetFileDescriptor (Ptr<FdNetDevice> device) const
NS_FATAL_ERROR ("EmuFdNetDeviceHelper::SetFileDescriptor (): m_deviceName is not set");
}
#ifdef HAVE_DPDK_USER_H
if (m_dpdkMode)
{
Ptr<DpdkNetDevice> dpdkDevice = StaticCast<DpdkNetDevice> (device);
dpdkDevice->SetDeviceName (m_deviceName);
dpdkDevice->InitDpdk (m_ealArgc, m_ealArgv);
return;
}
#endif
//
// Call out to a separate process running as suid root in order to get a raw
// socket. We do this to avoid having the entire simulation running as root.

26
src/fd-net-device/helper/emu-fd-net-device-helper.h
View File

@@ -45,8 +45,7 @@ public:
*/
EmuFdNetDeviceHelper ();
virtual ~EmuFdNetDeviceHelper ()
{
}
{}
/**
* Get the device name of this device.
@@ -68,14 +67,6 @@ public:
*/
void HostQdiscBypass (bool hostQdiscBypass);
/**
* Set the device in DPDK mode.
*
* \param argc Dpdk EAL args count.
* \param argv Dpdk EAL args list.
*/
void SetDpdkMode (int argc, char **argv);
protected:
/**
@@ -104,21 +95,6 @@ protected:
*/
std::string m_deviceName;
bool m_hostQdiscBypass;
/**
* The DPDK mode of the device.
*/
bool m_dpdkMode;
/**
* DPDK EAL arguments count.
*/
int m_ealArgc;
/**
* DPDK EAL arguments list.
*/
char **m_ealArgv;
};
} // namespace ns3

6
src/fd-net-device/model/dpdk-net-device.cc
View File

@@ -257,14 +257,16 @@ DpdkNetDevice::IsLinkUp (void) const
}
void
DpdkNetDevice::InitDpdk (int argc, char** argv)
DpdkNetDevice::InitDpdk (int argc, char** argv, std::string dpdkDriver)
{
NS_LOG_FUNCTION (this << argc << argv);
NS_LOG_INFO ("Binding device to DPDK");
std::string command;
command.append ("dpdk-devbind.py --force ");
command.append ("--bind=uio_pci_generic ");
command.append ("--bind=");
command.append (dpdkDriver.c_str ());
command.append (" ");
command.append (m_deviceName.c_str ());
printf ("Executing: %s\n", command.c_str ());
if (system (command.c_str ()))

3
src/fd-net-device/model/dpdk-net-device.h
View File

@@ -76,8 +76,9 @@ public:
*
* \param argc Dpdk EAL args count.
* \param argv Dpdk EAL args list.
* \param dpdkDriver Dpdk Driver to bind NIC to.
*/
void InitDpdk (int argc, char **argv);
void InitDpdk (int argc, char **argv, std::string dpdkDriver);
/**
* Set device name.

2
src/fd-net-device/wscript
View File

@@ -209,9 +209,11 @@ def build(bld):
if bld.env['ENABLE_DPDKNETDEV']:
module.use.append('DPDK')
module.source.extend([
'helper/dpdk-net-device-helper.cc',
'model/dpdk-net-device.cc'
])
headers.source.extend([
'helper/dpdk-net-device-helper.h',
'model/dpdk-net-device.h'
])
if 'src' in bld.env['DPDK_LIB_MODE']: