F5/unison
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

documentation: Update instructions for Python bindings

Browse Source
This commit is contained in:
Tom Henderson
2017-11-13 14:39:51 -08:00
parent b3638ce818
commit 6372023689
1 changed files with 167 additions and 70 deletions
Download Patch File Download Diff File Expand all files Collapse all files

237
doc/manual/source/python.rst
View File

@@ -1,6 +1,13 @@
.. include:: replace.txt
.. highlight:: python
.. heading hierarchy:
------------- Chapter
************* Section (#.#)
============= Subsection (#.#.#)
############# Paragraph (no number)
Using Python to Run |ns3|
-------------------------
@@ -122,7 +129,7 @@ Caveats
Python bindings for |ns3| are a work in progress, and some limitations are known by developers. Some of these limitations (not all) are listed here.
Incomplete Coverage
+++++++++++++++++++
===================
First of all, keep in mind that not 100% of the API is supported in Python. Some of the reasons are:
@@ -133,7 +140,7 @@ First of all, keep in mind that not 100% of the API is supported in Python. Som
Most of the missing APIs can be wrapped, given enough time, patience, and expertise, and will likely be wrapped if bug reports are submitted. However, don't file a bug report saying "bindings are incomplete", because we do not have manpower to complete 100% of the bindings.
Conversion Constructors
+++++++++++++++++++++++
=======================
`Conversion constructors <http://publib.boulder.ibm.com/infocenter/compbgpl/v9v111/topic/com.ibm.xlcpp9.bg.doc/language_ref/cplr384.htm>`_ are not fully supported yet by PyBindGen, and they always act as explicit constructors when translating an API into Python. For example, in C++ you can do this:
@@ -152,7 +159,7 @@ In Python, for the time being you have to do:
ipAddrs.Assign(backboneDevices)
CommandLine
+++++++++++
===========
:cpp:func:`CommandLine::AddValue` works differently in Python than it does in |ns3|. In Python, the first parameter is a string that represents the command-line option name. When the option is set, an attribute with the same name as the option name is set on the :cpp:func:`CommandLine` object. Example:
@@ -175,7 +182,7 @@ CommandLine
num_nodes_side = int(cmd.NumNodesSide)
Tracing
+++++++
=======
Callback based tracing is not yet properly supported for Python, as new |ns3| API needs to be provided for this to be supported.
@@ -198,28 +205,8 @@ Working with Python Bindings
Python bindings are built on a module-by-module basis, and can be found in each module's ``bindings`` directory.
Python Bindings Workflow
++++++++++++++++++++++++
The process by which Python bindings are handled is the following:
#. Periodically a developer uses a CastXML (https://github.com/CastXML/CastXML) based API scanning script, which saves the scanned API definition as ``bindings/python/ns3_module_*.py`` files or as Python files in each modules' ``bindings`` directory. These files are kept under version control in the main |ns3| repository;
#. Other developers clone the repository and use the already scanned API definitions;
#. When configuring |ns3|, pybindgen will be automatically downloaded if not already installed. Released |ns3| tarballs will ship a copy of pybindgen.
If something goes wrong with compiling Python bindings and you just want to ignore them and move on with C++, you can disable Python with:
.. sourcecode:: bash
$ ./waf --disable-python
Instructions for Handling New Files or Changed API's
****************************************************
So you have been changing existing |ns3| APIs and Python bindings no longer compile? Do not despair, you can rescan the bindings to create new bindings that reflect the changes to the |ns3| API.
Overview
++++++++
========
The python bindings are generated into an 'ns' namespace. Examples:
@@ -241,18 +228,142 @@ example. There is no structured documentation for the Python bindings
like there is Doxygen for the C++ API, but the Doxygen can be consulted
to understand how the C++ API works.
Scanning the Modular Python Bindings
++++++++++++++++++++++++++++++++++++
Python Bindings Workflow
========================
Scanning of the C++ API is only necessary if a user is using Python and
is changing the C++ API to introduce new methods (that he or she wishes
to be accessible from Python) or is changing the C++ API in a way that
breaks the compilation of the existing Python bindings.
The process by which Python bindings are handled is the following:
There are two steps. First, the bindings toolchain must be enabled in
the |ns3| build. This requires that the castxml and pygccxml tools be
installed on the system or using the bake build tool. Second, Waf
can be used to update the bindings.
#. Periodically a developer uses a CastXML (https://github.com/CastXML/CastXML) based API scanning script, which saves the scanned API definition as ``bindings/python/ns3_module_*.py`` files or as Python files in each modules' ``bindings`` directory. These files are kept under version control in the main |ns3| repository;
#. Other developers clone the repository and use the already scanned API definitions;
#. When configuring |ns3|, pybindgen will be automatically downloaded if not already installed. Released |ns3| tarballs will ship a copy of pybindgen.
If something goes wrong with compiling Python bindings and you just want to ignore them and move on with C++, you can disable Python with:
.. sourcecode:: bash
$ ./waf configure --disable-python ...
To add support for modular bindings to an existing or new |ns3| module, simply add the following line to its wscript build() function:
::
bld.ns3_python_bindings()
One must also provide the bindings files (usually by running the scanning
framework).
Regenerating the Python bindings
================================
|ns3| will fail to successfully compile the Python bindings if the C++
headers are changed and no longer align with the stored Python bindings.
In this case, the developer has two main choices: 1) disable Python
as described above, or 2) update the bindings to align with the new C++
API.
Process Overview
################
|ns3| has an automated process to regenerate Python bindings from the C++
header files. The automated process is only semi-automated at the moment
(ns-3.27) because we are in the midst of transition to new tools. The
current process is outlined below. In short, the process currently
requires the following steps.
1. Prepare the system for scanning by installing the prerequisites,
including a development version of ``clang``, the ``CastXML`` package,
and ``pygccxml``.
2. Perform a 64-bit scan of the module or modules of interest
3. Create the 32-bit bindings file from the 64-bit scan output
Installing a clang development environment
##########################################
Make sure you have a development version of the clang compiler installed
on your system. This can take a long time to build from source. Linux
distributions provide binary library packages such as ``clang-dev`` or
``clang-devel``. The version should not be too important; version 3.8 is
known to work. Note that there is a problem with the Ubuntu
package installation of ``clang-dev``; see the Installation wiki page for
details on how to fix using some symlinks.
Installing other prerequisites
##############################
``cxxfilt`` is a new requirement, typically installed using ``pip``; e.g.
::
sudo pip install cxxfilt
See also the wiki for installation notes for your system.
Set up a ``bake`` build environment
###################################
Try the following commands::
$ cd bake
$ export BAKE_HOME=`pwd`
$ export PATH=$PATH:$BAKE_HOME/build/bin
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$BAKE_HOME/build/lib
$ export PYTHONPATH=$PYTHONPATH:$BAKE_HOME/build/lib
$ mkdir -p build/lib
Configure
#########
Perform a configuration at the bake level::
$ ./bake.py configure -e ns-3-dev -e pygccxml-1.9.1
The output of ``bake show`` should show something like this:
::
$ ./bake.py show
Should say::
-- System Dependencies --
> clang-dev - OK
> g++ - OK
> libxml2-dev - OK
> pygoocanvas - OK
> pygraphviz - OK
> python-dev - OK
> qt - OK
> setuptools - OK
Download
########
Try the following command::
$ ./bake.py download
>> Searching for system dependency python-dev - OK
>> Searching for system dependency clang-dev - OK
>> Searching for system dependency g++ - OK
>> Searching for system dependency setuptools - OK
>> Searching for system dependency pygoocanvas - OK
>> Searching for system dependency pygraphviz - OK
>> Searching for system dependency qt - OK
>> Downloading castxml - OK
>> Downloading netanim - OK
>> Downloading pygccxml-1.9.1 - OK
>> Downloading pygccxml - OK
>> Downloading pybindgen - OK
>> Downloading ns-3-dev - OK
Build
#####
Try the following commands::
$ mkdir -p build/lib
$ ./bake.py build
It should fail on the |ns3| bindings complilation.
The output of './waf configure' can be inspected to see if Python API scanning
support is enabled:
@@ -271,45 +382,24 @@ In this case, the user must take steps to install castxml and pygccxml;
castxml binary must be in the shell's path, and pygccxml must be in the
Python path.
An automated setup for this is provided by the `bake` build system, if the
user selects the 'ns-allinone-3.nn' configuration target (where 'nn' is the
release number. For example:
Rescan 64-bit
#############
It is important that you scan the failing module with '--no32bit-scan'; e.g.:
::
./bake.py configure -e ns-allinone-3.27
./bake.py download
./bake.py build
$ cd source/ns-3-dev
$ ./waf --apiscan=wifi --no32bit-scan
Once API scanning support is enabled, to scan the modular Python bindings
for the core module, for example, do the following:
Generate 32-bit
###############
.. sourcecode:: bash
Once the 64-bit bindings are fixed, if you are a maintainer and need to
generate the 32-bit equivalent:
$ ./waf --apiscan=core --no32bit-scan
To scan the modular Python bindings for all of the modules, do the following:
.. sourcecode:: bash
$ ./waf --apiscan=all --no32bit-scan
The ``--no32bit-scan`` flag is described below.
Adding Modular Bindings To A Existing or New Module
+++++++++++++++++++++++++++++++++++++++++++++++++++
To add support for modular bindings to an existing or new |ns3| module, simply add the following line to its wscript build() function:
::
bld.ns3_python_bindings()
One must also provide the bindings files (usually by running the scanning
framework).
Differences between MacOS and Linux bindings
++++++++++++++++++++++++++++++++++++++++++++
1. copy the newly scanned LP64.py file(s) to the ILP32.py filename
2. on the ILP32.py file(s), perform global substitition of ``unsigned long`` with ``unsigned long long``
Linux (64-bit, as most modern installations use) and MacOS use different
data models, as explained here: https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.3.0/com.ibm.zos.v2r3.cbcpx01/datatypesize64.htm
@@ -333,7 +423,7 @@ bindings file and converting them to 'unsigned long long', such as:
+ cls.add_instance_attribute('nMarkedBytes', 'std::map< std::string, unsigned long long >', is_const=False)
In summary, to generate LP64 bindings for Linux 64-bit systems, it is
sufficient to call:
sufficient to call (e.g. for the ``core`` module):
::
@@ -345,8 +435,15 @@ hand-edit that file, replacing all instances of 'unsigned long' with
'unsigned long long'. |ns3| maintainers are working to better automate
this process for future releases.
Generating bindings on MacOS
############################
In principle, this should work (and should generate the 32-bit bindings).
However, it is untested and we are not sure what instructions to offer.
We would welcome suggestions on how to enable scanning for MacOS.
Organization of the Modular Python Bindings
+++++++++++++++++++++++++++++++++++++++++++
===========================================
The ``src/<module>/bindings`` directory may contain the following files, some of them optional: