OpenDataPlane (ODP) Implementers-Guide

Within a given test executable CUnit is used to run the different tests. The usage of CUnit implies the following structure:

Moreover, two extra functions can be used to initialize/terminate the test executable (these are not part of CUnit). A test executable return success (0) if every test of each suite succeed.

More details about CUnit can be found in the CUnit User’s Guide

All the above symbols are part of the generated libtest<Module>.la libraries. The generated main executable(s) (named <module>_main[_*], where the optional suffix is used to distinguish the executables belonging to the same module, if many) simply call(s) the related <Module>_main[_*] from the library.

If the considered platform needs no platform specific tests, this directory simply needs to contain a single Makefile.am listing each of the executables (named <module>_main) built from the platform agnostic area. The executables are listed in the automake TEST variable and will therefore be run on "make check".

For the linux-generic platform, most tested modules fall into this category: currently, the test/linux-generic/test/Makefile.am looks roughly as follows:

With the exception for module pktio, all other modules testing just involves calling the platform agnostic <module>_main executables (in /test/common_plat/validation/api).

The pktio module, above, is actually tested using a bash script. This script is needed to set up the interfaces used by the tests. The pktio_run script eventually calls the platform agnostic test/common_plat/validation/api/pktio/pktio_main after setting up the interfaces needed by the tests. Notice that the path to the script, validation/api/pktio/pktio_run.sh, is pointing to a file within the <PLATFORM_SPECIFIC> tree so is private to this platform. Any languages supported by the tested platform can be used there, as it will not impact other platforms. The platform "private" executables (such as this script), of course, must also return one of the return code expected by the automake test harness (0 for success, 77 for skipped, other values for errors).

The pktio case above is actually using a script as wrapper around the "standard" (platform independent) test executable. Wrappers can also be defined by using the LOG_COMPILER variable of automake. This is applicable in cases where the same wrapper should be used for more then one test, as the test name is passed has parameter to the wrapper. A wrapper is just a program expecting one argument: the test name.

Automake also supports the usage different wrappers based of the executable filename suffix. See Parallel-Test-Harness for more information.

To add a wrapper around the executed test, just add the following LOG_COMPILER definition line in the <PLATFORM_SPECIFIC>/Makefile.am:

Here follows a dummy example of what wrapper-script could be:

Note how the above script stores the return code of the test executable to return it properly to the automake test harness.

Sometimes, it may be necessary to call platform specific system calls to check some functionality: For instance, testing odp_cpumask_* could involve checking the underlying system CPU mask. On linux, such a test would require using the CPU_ISSET macro, which is linux specific. Such a test would be written in <PLATFORM_SPECIFIC>/<test-group>/<interface>/cpumask/…​ The contents of this directory would be very similar to the contents of the platform agnostic side cpu_mask tests (including a Makefile.am…​), but platform specific test would be written there. <PLATFORM_SPECIFIC>/Makefile.am would then trigger the building of the platform specific tests (by listing their module name in SUBDIRS and therefore calling the appropriate Makefile.am) and then it would call both the platform agnostic executable(s) and the platform specific test executable.

The shm module of the linux-generic ODP API does have a validation test written this way. You can see it at: test/linux-generic/validation/api/shmem

The general policy is that a full run of the validation suite (a make check) must pass at all times. However a particular platform may have one or more test cases that are known to be unimplemented either during development or permanently, so to avoid these test cases being reported as failures it’s useful to be able to skip them. This can be achieved by creating a new test executable (still on the platform side), giving the platform specific initialization code the opportunity to modify the registered tests in order to mark unwanted tests as inactive while leaving the remaining tests active. It’s important that the unwanted tests are still registered with the test framework to allow the fact that they’re not being tested to be recorded.

The odp_cunit_update() function is intended for this purpose, it is used to modify the properties of previously registered tests, for example to mark them as inactive. Inactive tests are registered with the test framework but aren’t executed and will be recorded as inactive in test reports.

In test/common_plat/validation/api/foo/foo.c, define all validation tests for the 'foo' module:

In <platform>/validation/api/foo/foo_main.c, register all the tests defined in the foo module, then mark a single specific test case as inactive:

So foo_test_a will be executed and foo_test_b is inactive.

It’s expected that early in the development cycle of a new implementation the inactive list will be quite long, but it should shrink over time as more parts of the API are implemented.

Some tests may require specific conditions to make sense: for instance, on pktio, checking that sending a packet larger than the MTU is rejected only makes sense if packets can indeed, on that ODP implementation, exceed the MTU. A test can be marked conditional as follows:

Foo_test_x is the usual test function. Foo_check_x is the test precondition, i.e. a function returning a Boolean (int). It is called before the test suite is started. If it returns true, the test (foo_test_x) is run. If the precondition function (foo_check_x above) returns false, the test is not relevant (or impossible to perform) and it will be skipped.

The tests (both platform agnostic and platform dependent tests) make use of a set of functions defined in a helper library. The helper library tries to abstract and regroup common actions that applications may perform but which are not part of the ODP API (i.e. mostly OS system calls). Using these functions is recommended, as running the tests on a different OS could (hopefully) be as simple as changing the OS related helper lib.

In the linux helper, two functions are given to create and join ODP threads:

odph_thread_create()

odph_thread_join()

These two functions abstract what an ODP thread really is and their usage is recommended as they would be implemented in other OS`s helper lib.

Five older functions exist to tie and ODP thread to a specific implementation:

odph_linux_pthread_create()

odph_linux_pthread_join()

odph_linux_process_fork_n()

odph_linux_process_fork()

odph_linux_process_wait_n()

The usage of these functions should not occur within ODP examples nor tests. The usage of these functions in other application is not recommended.

ABIs can be defined at two levels. The simplest ABI is within a specific Instruction Set Architecture (ISA). So, for example, an ABI might be defined among ODP implementations for the AArch64 or x86 architectures. This traditional approach is shown here:

In the traditional approach, multiple target platforms agree on a common set of typedefs, etc. so that the resulting output from compilation is directly executable on any platform that subscribes to that ABI. Adding a new platform in this approach simply requires that platform to accept the existing ABI specification. Note that since the output of compilation in a traditional ABI is a ISA-specific binary that applications cannot offer binary compatibility across platforms that use different ISAs.

An ABI an also be defined at a higher level by moving to a more sophisticated tool chain (such as is possible using LLVM) that implements a split compilation model. In this model, the output from a compiler is not directly executable. Rather it is a standardized intermediate representation called bitcode that must be further processed to result in an executable image as shown here:

The key to this model is that the platform linking and optimization that is needed to create a platform executable is a system rather than a developer responsibility. The developer’s output is a universal bitcode binary. From here, the library system creates a series of managed binaries that result from performing link-time optimization against a set of platform definitions. When a universal application is to run on a specific target platform, the library system selects the appropriate managed binary for that target platform and loads and runs it.

Adding a new platform in this approach involves adding the definition for that platform to the library system so that a managed binary for it can be created and deployed as needed. This occurs without developer involvement since the bitcode format that is input to this backend process is independent of the specific target platform. Note also that since bitcode is not tied to any ISA, applications using bitcode ABIs are binary portable between platforms that use different ISAs. This occurs without loss of efficiency because the process of creating a managed binary is itself a secondary compilation and optimization step. The difference is that performing this step is a system rather than a developer responsibility.

Certain implementation limits can best be represented by #define statements that are set at compile time. Examples of this can be seen in the odp-linux reference implementation in the file platform/linux-generic/include/odp_config_internal.h.

Here two fundamental limits, the number of CPUs supported and the maximum number of pools that can be created via the odp_pool_create() API are defined. By using #define, the implementation can configure supporting structures (bit strings and arrays) statically, and can also allow static compile-time validation/consistency checks to be done using facilities like ODP_STATIC_ASSERT(). This results in more efficient code since these limits need not be computed at runtime.

Users are able to change these limits (potentially within documented absolute bounds) by changing the relevant source file and recompiling that ODP implementation.

The ODP reference implementation, like many open source projects, makes use of autotools to simplify project configuration and support for various build targets. These same tools permit compile-time configuration options to be specified without requiring changes to source files.

In addition to the "standard" configure options for specifying prefixes, target install paths, etc., the odp-linux reference implementation supports a large number of static configuration options that control how ODP is built. Use the ./configure --help command for a complete list. Here we discuss simply a few for illustrative purposes:

Applications that accept a command line passed to their main() function can use this to tailor how they use ODP. This may involve self-imposed limits driven by the application or these can specify arguments that are to be passed to ODP initialization via the odp_init_global() API. The syntax of that API is:

and the odp_init_t struct is used to pass platform-independent parameters that control ODP behavior while the odp_platform_init_t struct is used to pass platform-specific parameters. The odp-linux reference platform does not make use of these platform-specific parameters, however the odp-dpdk reference implementation uses these to allow applications to pass DPDK initialization parameters to it via these params.

ODP itself uses the odp_init_t parameters to allow applications to specify override logging and abort functions. These routines are called to perform these functions on behalf of the ODP implementation, thus better allowing ODP to interoperate with application-defined logging and error handling facilities.

Linux environment variables set via the shell provide a convenient means of passing dynamic configuration values. Each ODP implementation defines which environment variables it looks for and how they are used. For example, the odp-dpdk implementation uses the variable ODP_PLATFORM_PARAMS as an alternate means of passing DPDK initialization parameters.

Another important environment variable that ODP uses is ODP_CONFIG_FILE that is used to specify the file path of a configuration override file, as described in the next section.

The libconfig library provides a standard set of APIs and tools for parsing configuration files. ODP uses this to provide a range of dynamic configuration options that users may wish to specify.

ODP uses a base configuration file that contains system-wide defaults, and is located in the config/odp-linux-generic.conf file within the ODP distribution. This specifies a range of overridable configuration options that control things like shared memory usage, queue and scheduler limits and tuning parameters, timer processing options, as well as I/O parameters for various pktio classes.

While users of ODP may modify this base file before building it, users can also supply an override configuration file that sets specific values of interest while leaving other parameters set to their defaults as defined by the base configuration file. As noted earlier, the ODP_CONFIG_FILE environment variable is used to point to the override file to be used.