Some API calls missed const qualifiers on read-only data pointers.

When available, parameter init functions must be used to initialize parameters into their default values.

CPU frequencies or CPU cycle counter may not be available on all HW/SW platforms. Zero is returned if those cannot be read.

Modify documentations of functions, which decrease packet length, to clearly state what are the allowed values for length argument. This is done to avoid creating zero length packets which are not allowed by the packet API.

An application may use this for testing or other purposes, when perception of the packet input interface need to be changed.

Added thread local cache size parameter and capability. This allows application to control thread local caching and prepare large enough pool when implementation caches events per thread. The default value is implementation specific for backwards compatibility.

The default value is implementation specific and may not be equal to the maximum capability.

Added packet pool parameter to request minimum data alignment for user allocated packets. When user allocates a new packet and fills in protocol headers, it’s convenient that data alignment does not need to be checked (and tuned) on each allocated packet.

Specify queue maximum size capability the same way for all non-blocking levels (ODP_BLOCKING, ODP_NONBLOCKING_LF and ODP_NONBLOCKING_WF). Max_size value of zero means that there is no size limit.

Clarify that queue enqueue operations include memory barrier of release semantics and queue dequeue operations include acquire semantics.

It may not be possible for random generator functions to return requested number of bytes. Clarify that implementation is not required to loop until len bytes are available. Instead application should contain such logic.

Stores performed while holding an atomic scheduling context are seen correctly by other thread when they hold the same context later on. This is guaranteed also when queue enqueue is not used in between.

Clarify that event schedule operations include memory barrier of acquire semantics.

This can be used to memory allocations where both CPUs and HW accelerators access the same memory area. These HW accelerators may be programmed outside of ODP APIs, but the memory is reserved and shared normally inside/between ODP applications.

Application needs often store object handles for later usage. From current APIs, e.g. buffers and queues could be used to store these handles, but buffers consume more memory than is necessary and event queues are not needed for this simple use case. This new API maybe implemented e.g. as a ring of object handles in memory, or with a HW buffer manager.

Added functions to get the current local/global time stamp directly in nanoseconds. For example, odp_time_local_ns() is equivalent of calling odp_time_to_ns(odp_time_local()). This simplifies use cases where time will be always converted to nanoseconds. However, when time API performance is important conversions to nanoseconds should be avoided or minimized.

Specification was open if returned expiration time is the original or actual expiration time. HW based implementations will not likely modify timeout event with actual expiration time. Also original expiration time is more valuable to an application as it can be used to calculate e.g. the next period.

Added option to specify timer pool resolution in hertz. High resolution values in nanoseconds result small numbers and thus poor granularity. Hertz offers better granularity with high resolution values. User gives resolution either in nanoseconds or hertz, and sets the other parameter to zero.

Traffic Manager API defines all types as platform specific, yet unit tests expect to be able to print them. Therefore introduce u64 debug print conversion functions for all TM types: odp_tm_to_u64(), odp_tm_queue_to_u64(), odp_tm_node_to_u64(), odp_tm_shaper_to_u64(), odp_tm_sched_to_u64(), odp_tm_threshold_to_u64(), odp_tm_wred_to_u64()

Clarify that info structures (odp_tm_node_info_t, odp_tm_node_fanin_info_t, odp_tm_queue_info_t, odp_tm_query_info_t) are written only on success.

Added a macro and version number defines for easier comparison of API version numbers.

Fixed issues reported by GCC 10. The code base builds now with GCC 10 also when LTO is enabled.

Added --with-config-file=FILE configuration option for setting path to the default configuration file. The default configuration file has to include all configuration options.

The PMR term documentation was not explicit about endianness of value and mask fields. Most terms assumed CPU endian, but terms with larger data sizes assumed big endian (MAC, IPv6 address and custom).

Term specification was harmonized so that all terms expect value/mask data to be in big-endian format, have fixed size, and allow free memory alignment. Packet length term is an exception to this as it does not represent a field in a packet.

Added new ODP_PMR_CUSTOM_L3 term to match custom layer 3 protocol fields. PMR offset refers to the start of layer 3 in the packet. Other PMR rules (e.g. L2 classification rules) may precede custom L3 rules.

Improved test result output when some tests are inactive (skipped due to missing capability). Now inactive tests are listed by default in the test suite results.

Cleaned up validation test code. Classifier implementation missed reporting some capabilities, tests didn’t check those capabilities and they were tested. Source IPv4 term test was missing.

Added serial PMR test, which tests series of PMR and CoS:

Added parallel PMR test, which test serial and parallel PMR and CoS:

Added new validation tests to verify that packet order is maintained when events are enqueued and dequeued to/from a plain queue while maintaining atomic/ordered scheduling context.

Added private timer pool test, which creates a timer pool with the priv flag set. The same thread calls schedule / queue_deq that created the pool.

Added new test application to measure schedule call CPU cycle consumption with various timer pool parameter combinations. This measurement is mostly interesting with software-based timer implementations, where timers may be polled from the schedule call.

This application replies to IPv4 ping requests. It can be used to test connectivity with standard ping utility. ARP table needs to be setup manually on the sender side as the application does not reply to ARP requests.

Added source CoS name into the parameter list of -p option. This enables linking classification rules together. When a source CoS is not defined, the default CoS is used as the source CoS.

Option -p parameter list format is now:

Where <src_cos> is optional and number of "xxx", "yyy", etc. parameters is term depend.

Added -v option which prints received packet with CoS queue name. This can be used for debugging classification rules.

Added support for ODP_PMR_VLAN_ID_0, ODP_PMR_ETHTYPE_0, ODP_PMR_CUSTOM_L3, ODP_PMR_ETHTYPE_X, ODP_PMR_VLAN_ID_X, ODP_PMR_UDP_DPORT, ODP_PMR_UDP_SPORT, ODP_PMR_TCP_DPORT, and ODP_PMR_TCP_SPORT rules.

Fix bug in timer start offset calculation. Test results were offset by current local time value.

Added delay and extra schedule calls to ensure that (software) timer implementation has passed its initial state before setting up timers.

Added -f option to control offset to the first timer. This can be used e.g. to avoid best/worst case synchronization of timers to timer tick boundaries / resolution.

Added ICMP support. Print ICMP offset and ICMPv4 type/code. This helps to verify that ping requests are received.

Added missing tests for the following packet functions:

Cleaned up implementation. Fixed endianness.

Cleaned up code. Fixed accuracy issue with high resolution (<100us) timer pools. Added configuration option inline_poll_interval_nsec to select default poll interval in nsec.

Added configure file option for inline timer (inline_thread_type) to select if control threads poll shared timer pools or not. Control threads still poll their private pools. With this user can configure control and worker threads to use separate timer pools.

Enable explicit pktio type definition. The pktio type is selected by adding pktio_type prefix in front of device name separated by a colon (<pktio_type>:<if_name>).

The socket mmsg pktio implementation has been refactored and performance has been improved.

The IPC pktio implementation has been modified to use standard ODP internal ring_ptr_t rings to implement IPC rings. This enables removing the duplicate _ring_t implementation.

Packet segmentation has been reimplemented using a linked list. The new implementation is simpler and has the added benefit of greatly reducing packet and buffer header sizes.

Added a parameter to control if a destination queue does not require event re-ordering. By default, event order is maintained for all destination queues as before. Application may use ODP_QUEUE_ORDER_IGNORE for those queues that are used under ordered queue context, but do not require re-ordering and/or do need to avoid extra re-ordering delays.

When set, this flag guarantees that the memory reserved by odp_shm_reserve() is allocated from huge pages. If enough huge page memory is not available the call will fail.

Improve specification of initialization and termination steps. Explicitly list those functions that may be used before global init (those that are needed for setting up parameters). No changes to functionality.

Highlight that queue context default value is NULL. This was defined already in odp_queue_param_t specification.

Support for new crypto algorithms is added by defining symbols for:

Support for new crypto authentication algorithms is added by defining symbols for:

Added enumeration for digest (unkeyed) algorithms. They are added as auth algorithms with empty key required.

Correct documentation for ODP_AUTH_ALG_SNOW3G_UIA2 to reference 128-EIA1 instead of 128-EEA1.

Clarify IV data format and point to proper documents for 3GPP algorithms.

Timer API intention has always been to allow any event type to be used as timeout events. Application should use ODP_EVENT_TIMEOUT type events (odp_timeout_t) by default, but also other event types may be used. Also, clarified timer set/reset functionality of odp_timer_set_abs() and odp_timer_set_rel() functions.

API functionality not changed, just wording updated.

Typically, timer implementation needs to trade-off between highest resolution and longest timeout. Add new capability information (odp_timer_res_capability_t) and function to check limits between resolution and maximum timeout length.

Defines for a minute and an hour are useful e.g. when setting timers for a bit longer (background, session lifetime, etc.) timeouts.

Added helper library version defines, so that application can track helper version independent of ODP API version:

Also, added a function for generating easy printout of the versions number.

Defined new versions of thread create and join calls. The new calls explicitly support thread create and join in multiple steps. Also, per thread (odph_thread_param_t) and common parameters (odph_thread_common_param_t) have been improved.

Old functions odph_odpthreads_create() and odph_odpthreads_join() have been deprecated.

Added debug defines/macros into helper API and configure options to enable/disable helper debugging. These defines may be used both in helper and application code:

Enables passing tests on high core count devices.

Test min/max timeouts with the highest resolution and longest timeout parameters.

The group test would fail if odp_schedule(ODP_SCHED_NO_WAIT) wouldn’t return any events on the first call.

scheduler_test_pause_resume() would get stuck if all events were not successfully enqueued.

Fixed a number of synchronization problems revealed by a scheduler implementation doing pre-scheduling. Makes sure scheduling context is always properly released.

Some of the test array elements may have not been set if the number of supported scheduling priorities was low.

Added more debug information on magic number misses.

Added missing local init/term calls to all test cases.

Added test case to set num_worker and num_control parameters.

Added test case to set not used features flags in init parameters.

Test suite missed to output the return value of odp_cunit_run().

Check that queue context pointer value is NULL by default.

Check pktio level inline IPsec support before running tests.

Added support for AES-194-GMAC and AES-256-GMAC algorithms.

Added more AES-CBC and AES-CTR test vectors.

The application receives packets from one interface and passes them through 0-N worker stages before outputting them from a second network interface. The RX, worker, and TX stages are connected using plain queues and each stage is run on a separate CPU thread. Optionally, the worker stages calculate CRC-32C over packet data.

Fix single worker deadlock and run the application as part of make check.

Fixed a number of issues in IPsec example applications and run them during make check.

Refactor local buffer caching into separate functions and optimize implementation by caching buffer header pointers instead of indices.

New configuration options have been added for local cache (pool:local_cache_size) and burst sizes (pool:burst_size).

Pool implementation has been modified to store buffer headers instead of indices to reduce type conversion overhead.

Added warm up period into POSIX timer pool startup to avoid the first timeout to slip. Otherwise, timer pthread receives the first signal after about 15ms and first timeout of the pool slips.

The zero-copy DPDK pktio implementation has been cleaned up resulting a small performance improvement. Linking to DPDK library has also been simplified.

The latest LTS release of DPDK (v18.11) is now supported by ODP. Support for the previous LTS release (v17.11) is retained. Prior versions of DPDK are no longer supported.

As part of the introduction of flow-aware scheduling, capabilities associated with SCHED queues have been moved from the odp_queue_capabilities_t struct returned by odp_queue_capabilities() to the new odp_sched_capabilities_t struct returned by odp_sched_capabilities().

Capabilities moved include max_ordered_locks, max_sched_groups, and sched_prios. The max_sched_groups capability is renamed max_groups. In addition, max_queues, max_queue_size, and the support capabilities for lock free and wait free non blocking queues is now part of the scheduler capabilities.

In support of flow aware scheduling mode, the max_flows scheduler capability is renamed max_flow_id. A value of 0 indicates that flow aware mode scheduling is not available in this ODP implementation.

The "BUILD_ONLY` environment variable has been renamed CHECK for improved output readability.

As part of flow aware mode, scheduler validation tests will now test this mode if odp_schedule_capability() indicates that flow-aware mode is supported.

A number of changes to the ODP ABI have also been made in this release to improve application binary portability.

A new configuration file mechanism is introduced that makes use of libconfig to enable various runtime ODP parameters to be specified dynamically.

Default configuration values for the odp-linux reference implementation are contained in the config/odp-linux-generic.conf file. Users may override these default values by supplying their own configuration file. At odp_init_global() time, if the ODP_CONFIG_FILE environment variable is set, this is used to locate the path to the override configuration file.

The odp-linux reference implementation now supports applications that run in process mode (mem_model = ODP_MEM_MODEL_PROCESS) as well as the default thread mode. This support only applies within a single ODP instance, so any fork() calls must be done only after odp_init_global() has been called to initialize ODP on a root process.

The iQuery scheduler is removed from the odp-linux reference implementation, as it offers no performance advantages and has not seen application use.

The odp-linux reference implementation now supports up to 256 CPUs by default (increased from 128).

The odp-linux reference implementation now supports large burst sizes for both I/O and non-I/O scheduled events. Large bursts (when available) are received directly to the application without any stashing for improved throughput. Burst sizes are configurable via the new configuration file mechanism, as described above.

When building odp-linux using --without-openssl a warning will be issued cautioning that strong cryptography will not be available.

The various enq/deq APIs are now subject to inlining when odp-linux is built with --disable-abi-compat.

Inline timers are now controlled via a config file option. Timer polling frequency is similarly controlled via the config file.

The config file is now used to specify the huge page usage limit.

The queue implementation in odp-linux now automatically makes use of optimized single and multi-consumer/producer rings to significantly speed up enq/deq processing.

The output from odp_shm_print_all() is reworked to provide more useful and comprehensive shared memory usage information in odp-linux.

SA lifetime checking is now more scalable to multiple threads. This significantly reduces overhead for multithreaded IPsec applications.

When running in non-ABI compatibility mode, odp-linux now enables native machine-specific optimizations for the CPU architecture of the local machine.

SCTP test packets are now used in parser testing. SCTP headers are added to ODP and ODP helpers and SCTP checksums are now inserted and verified as part of validation testing.

The packet validation test suite now properly tests odp_packet_reset().

The odph_linux_thread_type_t enum has been replaced with the new odp_mem_model_t type.

The new odph_options() getter function is added that returns applicable options in effect via the new odph_helper_options_t struct. This currently includes the memory model (thread or process) that is in use.

The new helper APIs odph_sctp_chksum_set() and odph_sctp_chksum_verify() are added to facilitate working with SCTP packet checksums.

A new odp_pool_perf test has been added that stress-tests ODP pool functions in a multithreaded environment to generate performance statistics.

A new odp_sched_perf test has been added that stress-tests the scheduler in a multithreaded environment.

A new odp_cpu_bench performance test has been added that runs compute-intensive packet operations in a multithreaded environment and prints the observed maximum throughput for each thread.

The odp_classifier example program now uses a reduced number of threads by default to reduce elapsed run time. ODP_THREAD_COUNT_MAX is also now used as the max worker count.

The odp_generator example has numerous cleanups and performance improvements.

The odp_ipsec example now properly stops and closes pktio devices on exit.

A new odp_packet_dump example is added that prints received packets to the terminal. This is useful for debugging packet I/O interfaces.

A new odp_sysinfo example is provided that prints system information. Useful for checking the ODP environment during debugging. This includes providing detailed information about the various crypto facilities supported, as well as the feature flags used at build time (e.g., if the binary was built with ARMv8.0 or ARMv8.1 instructions).

The traffic manager example now properly destroys all TM queues it creates for improved reliability. It also now always prints a proper termination summary message.

IPsec extended sequence number support is missing

Separate IP ID allocation for transport and tunnel mode SAs may cause duplicate IDs

Unexpected IV causes IPsec API validation to fail

Clang build fails on Ubuntu 18.04

There are no API changes in this release.

ODP is now paired with DPDK 17.11.3 for its DPDK-related support. This is the current service level for the DPDK 17.11 LTS package used by ODP Tiger Moth.

ODP supports inlining of functions in embedded environments when ABI compatibility is not needed. ODP itself now makes use of inlined functions for all relevant internal use of its APIs, leading to improved performance.

The odp_hash_crc_gen64() API is now properly implemented and appropriate validation tests added to support it.

In addition, a streamlined table-based implementation of the basic CRC functions eliminates the previous dependency on zlib.

To better integrate with DPDK parsing capabilities, ODP packet parsing has been restructured to operate at the PktIO level. This permits DPDK PktIO types to exploit the native DPDK packet parser and checksum facilities, leading to better integration.

The PktIO functions have been streamlined and refactored in a number of ways to provide better long-term maintainability of these functions. This includes moving per-PktIO data into individual files rather than sharing a common file, as well as better placement for I/O statistics.

Loop PktIO interfaces now add the capability to validate packet L3 and L4 checksums as part of receive processing. The existing odp_pktio_capability() API now reports that checksum validation is available for these interfaces.

When defining lock free queues that have only a single producer and consumer, a streamlined implementation offers significant speedup.

Fast pcap capture is now provided in odp-linux to capture packets on any interface. This is enabled via the --enable-pcapng-support configuration option. Once enabled, packets can be captured using a sequence such as:

Interfaces of interest are identified by a string consisting of the application process ID, the interface name, and the queue number of interest, if the interface supports multiple queues.

A number of autotools/autoconf M4 macros used in configuring odp-linux have been rewritten to avoid potential GPL licensing concerns. These macros are used only during ODP configuration processing and have no role in ODP or ODP application use.

The validation test suite for queue API testing is enhanced to now test operation on queue pairs properly. This enables the various enqueue/dequeue modes defined by the ODP specification to be more fully exercised, leading to improved API conformance.

The scheduling validation tests now better use the various capability APIs to ensure that implementations are only tested for advertised capabilities.

A new odp_queue_perf test has been added to test plain (non-scheduled) queue performance in various modes.

Numerous changes to Travis integration are added to improve the quality and reliability of Continuous Integration (CI) testing.

Timeout accuracy breaks down with odd resolution requests

timer validation test fails when using 1GB huge pages

example/l2fwd_simple fails on some systems when using 1GB huge pages

Shmem validation test runs indefinitely with 1GB huge pages

There are several issues that seem to arise when running ODP applications on platforms with more than 64 cores. While the most critical of these, which affected the scheduler, have been fixed in this release, there are others that are still under investigation. These will be addressed in the next release of ODP.

There are no API changes in this release.

ODP is now paired with DPDK 17.11.2 for it’s DPDK-related support. This is the current service level for the DPDK 17.11 LTS package used by ODP Tiger Moth and incorporates a number of important bug fixes.

The GCC 8 series of compilers provides additional warnings about possible string truncation. A few ODP modules were changed to avoid misleading warnings when compiling with this level of GCC.

The libnuma library is used for DPDK pktios to provide proper memory allocation on NUMA-aware systems. Linking with this library is improved to avoid extraneous error messages at build time.

Packet metadata is reorganized to reduce the cache footprint used by the ODP reference implementation, resulting in performance improvements.

The odp_random_xxx() family of APIs was moved to a separate module (odp_random.c) for modularity and better isolation from planned crypto enhancements.

Unnecessary locking is removed from the odp_fdserver module, streamlining operations on shared memory.

The default number of timer pools supported by the ODP reference implementation has been reduced from 255 to 32. This lower number remains generously adequate for most applications and meaningfully reduces memory footprint, giving better performance.

During initialization ODP normally measures timer resolution to set the reported highest_res_ns. When such measurement is not able to be performed, this is not limited to 500ns to avoid bounds errors with overly precise resolutions.

The validation test now correctly handles corner cases when the implementation under test fails to process any test packets. It also includes the previously missing ODP_CIPHER_ALG_AES_CTR name.

Additionally, since individual implementations indicate which crypto/hash algorithms are supported via the odp_crypto_capability() API, the crypto validation test now properly uses this information and only tests those algorithms that the implementation reports as supported. The list of unsupported algorithms is also reported as part of the test results.

The number of input/output queues used by this test can now be specified, providing additional controls for test flexibility. In addition, pktout queues are now selected based on input queue rather than worker id, thus ensuring packet order flow is maintained.

Finally, an inactivity timer is added that allows the test to report when packets were handled due to timeout rather than I/O activity.

The validation test for the ODP timer APIs has been reorganized to better characterize an implementation’s conformance to the ODP Timer API specification. Since implementations can have widely differing timer accuracy levels, particularly when running in virtualized environments, the test also relaxes its bounds checking and enforcement somewhat to better avoid false negative test results.

Additionally, a timer pool create/destroy test was added as this area was not adequately covered previously.

PktIO does not work with Mellanox Interfaces

RX UDP checksum offload drops valid UDP packets with Niantic

IP header checksum not inserted if L4 offset not set

fdserver process interferes with signal handling

tests should fail if odp_crypto_op/op_enq process 0 packets

IPsec code can occasionally damage packets

Timer segfaults when creating and destroying multiple timer pools

linux-gen: ipc test fails to reserve memory

Shmem validation test runs indefinitely with 1GB huge pages

Timeout accuracy breaks down with odd resolution requests

No functional changes for this release. The Tiger Moth API was frozen in ODP v1.18.0.0.

The C++ test included in the ODP validation suite now uses cout instead of printf() to ensure that C++ is being used to compile it.

For the ODP Reference Implementation, The config/odp-linux-generic.conf file is extended with sections to control the default and maximum sizes for basic queues, and the priority spread used by the scheduler for scheduled queues.

The configuration file is a template named platform/odp-$platform.conf so this can be easily inherited by other ODP implementations.

For the ODP Reference Implementation, the default values of the application-provided config file (if used) override the values provided by the built-in config/odp-linux-generic.conf file.

PktIO netmap support now uses the ODP config file to allow rings used for VALE processing to be specified. The supplied defaults provide optimal performance in typical settings.

These crypto/authentication algorithms are now implemented.

Proper packet checksum validation and insertion, in conformance with the relevant ODP APIs, is now provided.

The Tiger Moth LTS release is synchronized with the most recent DPDK LTS release for DPDK pktio support.

This dependency is removed. The Reference Implementation build tools now use the standard od tool rather than the optional xxd package.

A new test has been added to test the performance of PktIO operations in scheduled mode. Scheduled PktIO is inherently more scalable and simpler from an application standpoint than direct (polled) I/O, but depending on the efficiency of the scheduler implementation can incur additional levels of overhead. This test can give insight into a given platform’s scheduler efficiency. For the odp-linux reference implementation, this test has shown scheduled I/O to be within 10% of rates achievable via direct I/O, meaning that for many applications the simplicity and scalability of the event model is preferable.

A new test has been added that measures outbound (TX) IPsec performance with a variety of cipher and authentication algorithms.

The README file associated with this example has been clarified to explain that this example is a throughput test and as a result does not preserve packet order under all conditions.

ODP linux-generic fails on AArch64 in non-ABI-compat mode.

PktIO does not work with Mellanox Interfaces

RX UDP checksum offload drops valid UDP packets with Niantic

IP header checksum not inserted if L4 offset not set

fdserver process interferes with signal handling

return value not checked for some fdserver interface functions

ODP exposes symbols outside of odp*/_odp* namespace

No changes for this release.

This release corrects a merge issue with crypto functionality that resulted in incorrect crypto operation for certain cipher suites.

Introduces a runtime configuration file that can be used by applications to set selected ODP configuration parameters dynamically at runtime rather than at configure time. At present this is used to configure parameters needed by DPDK PMDs when using PktIO interfaces in the DPDK class. The intention is to expand the use of this file over time to allow more dynamic control of other aspect of ODP runtime configuration.

For the ODP Reference Implementation, a template configuration file is provided in config/odp-linux.conf. Introduction of this support generates an additional dependency on the libconfig package, which must be present to build ODP.

A few missing implementation features associated with TFC packet generation have been added in this fix level. This support is now functionally complete in the ODP Reference Implementation.

The information provided in debug builds of the Reference Implementation is improved to print the interface name on open, start, stop, and close calls. The driver name and number of queues are also printed to ease verification of correct configuration.

The performance of the default scheduler in the Reference Implementation is significantly improved in providing scheduled access to PktIO queues. Scheduled I/O now operates within 10% of the performance achievable using Direct I/O, while providing incomparably better scalability in multicore environments.

In preparation for the Tiger Moth official release, ODP has adopted a simplified .so naming scheme, which is introduced here. ODP .so numbers are now tied to the ODP release number since ODP does not promise backward compatibility across release boundaries.

The following new and changed APIs are included in this release:

In the LNG Reference Implementations of ODP, the odp_pktio_open() API may now specify devices of class null to indicate the PktIO is treated as a dummy device. Null devices behave just like Linux /dev/null in that they never receive packets and simply discard any packets sent to them.

Note that not all ODP implementations may support this device class. The specific device classes supported by each ODP implementation may vary and are listed in their own documentation.

The ODP Reference Implementation offers both a default and a number of alternate scheduler implementations. Previously these were selectable only at configure time. They can now be dynamically selected at runtime by the use of the ODP_SCHEDULER environment variable. If this environment variable is not set, the default (basic) scheduler is used. It can be set to select alternate schedulers:

The ODP Reference Implementation now uses a ring model for implementing ODP queues. This results in greatly improved efficiency for queue operations. The default maximum queue depth used is 4096 elements, and this information is returned via the odp_queue_capability() API.

The tests that are part of the validation test suite are reorganized and simplified by having a single test file for each API rather than separate CUnit driver files and test files.

The crypto validation test suite now offers better information on which crypto algorithms were skipped because they are not available. Testing of full HMAC lengths is now added

The odp_generator example now offers configurable RX burst size, selectable packet handling (Direct I/O or Scheduled I/O), as well as streamlined packet processing.

The l2fwd example offers improved efficiency via better cache usage.

timer test might fail

time_main test fails if run under heavy load

classification: CoS queues in invalid table index

classification: multiqueue CoS will always fail

classification: requested number of queues is ignored in multiqueue CoS

cls: capability to return max hash queues

classification: invalid memory access in RSS hash

classification: incorrect IPv6 RSS hash

IPsec SA may be used before fully initialized

IPsec SA lookup may leave extra SAs locked

new generator test assumes that null:0 pktio is always present

packet_main test can fail

DPDK pktio stops receiving packets if all configured RX queues are not used

Another timer_main failure

Creating a pool with total size over 4.29GB (UINT32_MAX) leads to undefined behavior

ODP exposes symbols outside of odp*/_odp* namespace

ODP linux-generic fails on AArch64 in non-ABI-compat mode

The following new and changed APIs are included in this release:

When compiling with --enable-abi-compat=no the following additional packet functions are inlined:

The ODP test suite now automatically skips C tests if no C compiler is available.

The odp_pktio_ordered tests are only performed if PCAP is available.

The DEPENDENCIES file has been updated to reflect build/test requirements for running under Red Hat Linux distributions.

PktIO DPDK support has been upgraded to DPDK 17.08.

CID 1461688: odp_pool_create: Dereference before null check

CID 1396968: buggy check

l2fwd segfaults on api-next with dpdk checksum insertion override

packet header parsing routines should verify header checksums

timer test might fail

The following new and changed APIs are included in this release:

The default packet headroom in odp-linux has been increased from 66 to 128 bytes for better compatibility with odp-dpdk.

The odp-linux reference implementation now fully supports zero-copy packet references. Previously these APIs were implemented via packet copies, which while functionally correct, were not how these APIs are intended to operate.

The --enable-dpdk-zero-copy configure option is added to allow DPDK PktIO devices to avoid data copies, leading to improved performance.

DPDK PktIO now makes use of RX and TX IP/UDP/TCP checksum offload.

In the odp-linux reference implementation, shared memory is now backed to /dev/shm rather than /tmp for better reliability and robustness. This may be overridden as part of ODP build-time customization if desired.

PktIO IPC support has received improvements in both performance and reliability and is now suitable for development use.

The thread ID is now used to create unique vdev MAC addresses to avoid conflicts with multiple ODP processes running on the same host system.

The include/odp/drv directory and related files have been removed. Driver support is moved to a follow-on ODP release, so removing these files avoids confusion as they are still incomplete.

Since some build environments do not supply autoconf-archive, this dependency is removed.

The ODP DPDK Packet I/O support has been upgraded to work with the DPDK 17.08 release.

ODP use of OpenSSL for crypto processing has been upgraded to allow use of OpenSSL 1.1.x.

The test directory has been reorganized and streamlined. Platform-specific tests are moved from test/linux-generic to platform/linux-generic/test/. As a result, the test/common_plat directory is deleted so that test/validation, test/performance, etc. are now used for all platform-independent tests.

check-odp: valgrind generates "No rule to make target"

test odp_l2fwd_run.sh contains todo items

Helper/test/process fails on a single core system

Remove redundant loop_support parameter in pktio capability

Make file deps failure

IP headers checksum functions are incorrect

odp_traffic_mngr example is broken

pktio_ipc_run test can fail due to segfault

User guide error (packet diagram fix)

api-next out of tree build broken

Cross compile broken for ARMv8

IPC pktio test fails with taskset -c 1-2

Test case for classification enable

Memory allocation checks (in traffic manager)

Adding --enable-helper-linux configure flag breaks build

Doxygen warnings on helper header files

codecov: _odp_packet_cmp_data is not covered

setup_pktio_entry missing unlock

odp_cpu_hz() does not work on all Linux distros

Missing doxygen detected by Travis

'num_queues' isn’t ignored when "classifier_enable" is enabled

Validation tests cannot be disabled after commit b4d17b1

wrong openssl_lock pointer type

Cannot run l2fwd application on Cavium ThunderX platform

To permit smoother evolution of the ODP API specification, a deprecation framework is introduced to permit controlled deprecation.

When an ODP API or defined struct parameter is deprecated, ODP validation tests will be updated to no longer use that API and instead use the replacement API. By default, attempts to compile with the older API/feature will fail and applications wishing to move to the new ODP release should be updated to use the replacement API. To permit evaluation of new ODP releases in advance of such updating, however, ODP supports the configure option --enable-deprecated, which makes the obsolete APIs visible again.

This feature will be used on a case-by-case basis and documented in the release notes for each release that introduces replacement API(s). In general the deprecated forms will not be maintained for more than a single release cycle. After that they will no longer be present in the API specification and the replacement forms must be used to compile with that level of the API specification.

A number of new and refined APIs are introduced in crypto, packet parsing, and queue configuration:

ODP helper functions fall into two categories: system-independent, and those that rely on Linux. For backwards compatibility, these have been reorganized into a new helper/linux directory that is installed only when the --enable-helper-linux configure option is specified.

The default scheduler is streamlined by using precomputed weight tables and separated priority queues for different scheduling groups. This improves both latency and scalability.

Also included in this change, the default number of scheduling groups is lowered from 256 to 32, which experience has shown is sufficient for most applications.

The scheduler fairness checks are also enhanced to avoid low priority event starvation if CONFIG_BURST_SIZE is set to 1.

ARMv8 compilation now works properly when using the clang compiler.

example/generator/odp_generator.c contains todo items

Error handling issues (CHECKED_RETURN)

Unchecked return in pool.c

Unchecked return in mmap_unmap_sock()

ODP_STATIC_ASSERT() fails when used by C++

odp_pktio_ordered_run.sh: line 34: integer expression expected

load sensitive tests fail on CI

The odp_crypto_operation() routine does not work with multi-segment packets.

Packet validation test fails if implementation does not limit packet length

odph_iplookup_table_put_value() uses overlapping pointer addresses

Miss to call unlock if there are some errors happen in loopback_send() function

odp_packet_seg_t fails ABI compatibility between odp-linux and odp-dpdk

Compilation fails using clang 4.0.0

doxygen errors and travis does not catch them

TM validation test does find traffic_mngr_main

odp_rwlock_read_trylock() fails if lock is already held for read access

Packet order lost when enqueuing to pktout queue

AES-GCM returns 'valid' tag when checking invalid tag

Various failures if CONFIG_PACKET_MAX_SEGS is set to 1

Travis: time main test out of boundaries

Compilation failures using GCC 7 series

Socket pktio recv fails on large number of packet

odp_traffic_mngr example is broken

pktio_ips_run test can fail due to segfault

ODP v1.14.0.0 represents another preview of the Tiger Moth release series and introduces new APIs and extensions.

ODP has supported an Application Binary Interface (ABI) that permits applications to be generated that are binary portable between select ODP implementations. This is now formalized with the addition of structures that permit ABI specifications to be defined that are shared between ODP implementations. ODP provides a default ABI specification that is the same as was provided before. This change means that additional ABIs may be defined that are optimized to collections of platforms that share an Instruction Set Architecture (ISA) and subscribe to them.

ABI specifications live in the include/odp/arch/…​ directory.

Adding to the range of scheduler implementations supported by odp-linux, a new scheduler, the instance query scheduler, is available by specifying --enable-schedule-iquery at configure time.

This scheduler uses sparse bitmaps and offers scalability advantages when dealing with large numbers of schedule queues. It otherwise offers performance comparable to the default ODP scheduler implementation.

The odp-linux reference implementation now offers improved inlining support for ODP APIs for applications compiled against ODP configured with the --enable-abi-compat=no option, meaning they wish to forego binary compatibility in exchange for improved performance. These applications remain source portable to other ODP implementations.

A native implementation of the odp_cpu_pause() API has been added for ARMv7 and ARMv8 processors.

The odp_packet_bench microbenchmark application has been added to the test performance directory to allow implementations to measure and calibrate the performance of individual ODP packet APIs.

The odp_pktio_ordered application has been added to the test performance directory to provide stress-testing of packet ordering features of ODP.

The hello.c application can now run properly if Core 0 is not available (any available core will be used).

Provide proper return code checking within _ishm.c

Remove unused variables in iplookuptable helper routine.

Avoid memory leaks on error paths in the cuckoo table helper functions.

Fixes a race condition in shared memory allocation.

Provide proper fallback for shared memory when hugepages are not available.

Avoid null pointer dereference in the iplookuptable helper routine.

Missing doxygen documentation for helper table functions are added.

C++ programs fail to compile if they attempt to include the ODP helper APIs.

The odp_crypto_operation() routine does not work with multi-segment packets.

ODP v1.13.0.0 represents the initial preview of the Tiger Moth release series and as such introduces new APIs and extensions that will be built on as this release matures.

Storage pools used for buffers and packets are now implemented via lockless ring structures that support burst mode read/writes to local caches for significantly improved multi-core scalability

The layout of the structs used to support buffers and packets has been optimized to reduce cache footprint and minimize path lengths in packet manipulation.

The implementation of ordered queues has been streamlined and made more scalable in multi-core systems.

The more advance segmentation capabilities present in the new ODP packet implementation are used to streamline the implementation of APIs like odp_packet_concat() and the head/tail extend/trunc APIs.

A number of "todos" were removed from the packet validation test suite.

The CPU affinity is now correctly read from the cpuset.

The PktIO validation test no longer uses invalid MAC addresses.

The TCP checksum is now properly calculated for generated packets.

The odp-linux reference implementation makes use of the OpenSSL library to support the odp_random_xxx() APIs and some crypto operations. To support OpenSSL versions prior to 1.1.0, which are not thread safe, support is added for OpenSSL locking callbacks that use ODP ticketlocks to provide OpenSSL thread safety.

Make check fails on a single core VM in the process mode helper test.

ODP v1.12.0.0 has no API changes from previous v1.11.0 Monarch LTS. Version is increased in current development release to make room for Monarch updates numbers.

Support is added to enable ODP applications to be binary compatible across different implementations of ODP sharing the same Instruction Set Architecture (ISA). This support introduces a new configure option:

Note that ODP applications retain source code portability between ODP implementations regardless of the ABI mode chosen. To move to a different ODP application running on a different ISA, code need simply be recompiled against that target ODP implementation.

The ODP classifier adds support for recognizing Stream Control Transmission Protocol (SCTP) packets. The APIs for this were previously not implemented.

ODP no longer has a dependency on Linux headers. This will help make the odp-linux reference implementation more easily portable to non-Linux environments.

The odp-linux implementation has been made independent of the helper library to avoid circular dependency issues with packaging. Helper functions may use ODP APIs, however ODP implementations should not use helper functions.

The test directory has been reorganized to better support a unified approach to ODP component testing. API tests now live in test/common_plat/validation/api instead of the former test/validation. With this change performance and validation tests, as well as common and platform-specific tests can all be part of a unified test hierarchy.

The resulting test tree now looks like:

The maximum number of pools that may be created in the odp-linux reference implementation has been raised from 16 to 64.

The DPDK pktio support in odp-linux has been upgraded to work with DPDK 16.07. A number of miscellaneous fixes and performance improvements in this support are also present.

Packet I/O interfaces operating in TAP mode now can feed packets to the ODP classifier the same as other pktio modes can do.

The scheduler and pktio components have been reworked to use burst-mode buffer allocation/deallocation, yielding a measurable performance gain in almost all cases.

ODP queues internally now attempt to use burst-mode enq/deq operations to accelerate performance where applicable.

The ODP scheduler has been enhanced to use ring-based priority queues, resulting in significantly better scalability in many core environments.

ODP now supports the Travis framework needed to trigger CI automation in conjunction with GitHub. This support is considered experimental for now.

A new example application is provided that illustrates use of ODP for simple Layer 3 forwarding across multiple interfaces.

ODP now generates "pure" (implementation independent) doxygen documentation in addition to the specific documentation for the odp-linux implementation. The pure version is applicable to any ODP implementation as it simply describes the ODP API specification. Implementation-specific versions of this documentation describe both the APIs as well as specifics concerning typedefs, enums, etc. This should help clarify what aspects of ODP are platform independent.

The definition of an ODP thread in odp-linux has been clarified to specify that for this implementation ODP threads are Linux pthreads or Linux processes. Currently threads as processes is considered experimental in odp-linux.

ODP_TIMEOUT_INVALID is now defined consistently with other pool elements so the restriction on using Pool 0 as a timer pool is removed.

The odp_packet_copy() API now correctly handles user areas between pools that are configured with different sized per-packet user area definitions.

Corrects the implementation of AES GCM decryption in the ODP crypto API.

The Timer validation test fails sporadically in environments with high core counts.

As part of the final Monarch LTS API set, a minor change to the odp_tm_egress_t struct was made to better reflect the capabilities of SoC platforms targeting Monarch support. This change adds the boolean egress_fcn_supported that indicates whether TM systems support an egress function. In addition, each TM system is now associated with a PktIO rather than a PktOUT queue. This struct is input to the odp_tm_create() API and output from the odp_tm_find() API.

Building on the modular scheduler framework introduced in v1.10.1.0, An alternate Strict Priority (SP) scheduler is now available. The SP scheduler is selected when ODP is configured with the --enable_schedule_sp option.

The SP scheduler is designed to favor low-latency processing of high priority work at the expense of throughput. This is considered experimental code at this point and should be treated as such by those wishing to use it.

The scheduler now properly cleans up PktIOs operating in SCHEDULE mode following odp_pktio_close() calls. This resolves Bug 2301.

The odph_chksum() helper routine is now C++ conformant. This resolves Bug 2302.

The odp_packet_copy() API now copies the packet user area as part of its processing and the packet validation test has been enhanced to verify that this is done properly. This resolves Bug 2310.

The internal definition of ODP_TIMEOUT_INVALID was changed in odp-linux to enable the use of Pool 0 as a timeout pool. This resolves Bug 2316.

To facilitate API tracking, libodphelper.so has been split out from libodp.so.

To avoid circular dependencies that cause issues when packaging ODP as a shared library, the odp-linux implementation no longer makes and use of the ODP helper library.

The Queue validation test now properly exercises the odp_queue_capability() API and attempts to create the maximum reported number of supported queues.

The platform-specific tests have been moved from platform/linux-generic/test to test/platform/linux-generc so that all tests are now found under a single test directory.

The Doxygen API guide has been split so that both a "pure" version that is platform-independent and retains unresolved ODP abstract types as well as an implementation-specific version (for odp-linux) that shows platform-specific typedef and enum values.

The ODP Implementer’s Guide has been enhanced with a section on implementation considerations, including issues surrounding the definition of Application Binary Interfaces (ABIs).

The timer validation tests have seen occasional failures when run on systems containing a larger number of CPUs (typically 24 or more).

ODP cannot be compiled using the -m32 option to generate 32-bit code on a 64-bit platform when using GCC 4.9 (the default GCC found in Ubuntu 15.04). This is due to a known bug in GCC 4.9 and is closed as a permanent restriction.

Doxygen generates a warning for ODP_BIG_ENDIAN_BITFIELD when compiling on little endian systems. A similar warning for ODP_LITTLE_ENDIAN_BITFIELD is issued when compiling on big endian systems.

The traffic manager cannot be used with two PktIOs simultaneously. That is, odp-linux does not properly support applications that create more than one TM system. This is a bug in the odp-linux implementation that will be corrected. It is not a limit of the architecture or other ODP implementations.

Packet queue pool causes out-of-bounds accesses. This is closely related to Bug 2444 and reflects that odp-linux does not properly support multiple TM systems. This is scheduled for correction in the first maintenance update for this ODP implementation.

ODP v1.10.1.0 is a maintenance release and as such introduces no new APIs or other changes that might result in application source incompatibility with v1.10.0.0.

The ODP scheduler included as part of the odp-linux reference implementation has been modularized to support a "pluggable" architecture that will permit the easy introduction of alternate scheduler implementations that permit both easy experimentation as well as alternate scheduling models that are tuned to specific application workloads (i.e., a strict priority scheduler for latency-sensitive workloads). Included in this release is the just the foundation and alternate schedulers are expected to be added in future ODP releases.

While the platform name within the ODP git repository remains linux-generic, the package name of this reference implementation has been changed to odp-linux to better reflect that this is an implementation of ODP that is dependent only on the Linux kernel and is intended to be runnable on any Linux distribution. All ODP documentation has been changed to reflect this name change.

The current helper APIs for linux pthreads (odph_linux_pthread_create(), etc.) and processes (odph_linux_process_fork(), etc.) have been augmented with a new "agnostic" set of helper routines for general ODP thread management.

odph_odpthreads_create() and odph_odpthreads_join() are now the preferred helpers to use for creating and joining threads that may be implemented as either pthreads or Linux processes. Although currently ODP threads in Monarch are assumed to share the same (single) address space, this change is to permit evolution towards multi-address space (i.e., process oriented) threading models without requiring application and/or validation test changes.

The various ODP examples and validation tests that make use of threading have all been changed to new these new helper APIs.

As part of the new "agnostic" threading support, two additional helper APIs (odph_odpthread_getaffinity() and odph_odpthread_setaffinity()) have been added to permit easy management of thread affinity.

The CUnit framework used by the ODP validation suite has been changed to set the affinity of the initial thread to the first available control CPU. This avoids inconsistencies in a number of timing tests.

The configure utility now correctly identifies and sets variables needed for proper compilation on 32-bit x86 systems. In addition the Traffic Manager now executes properly on 32-bit systems.

The odp_timer_t abstract type is now strongly typed for consistency with other ODP types, and the odp_timer_to_u64() and odp_timer_pool_to_u64() APIs are now properly implemented. In addition, an issue with compiling the timer routines on 32-bit systems using clang was resolved.

The default scheduler has been improved to avoid starvation and increase fairness when running with a limited number of threads or CPUs.

The -mcx16 compiler option is now properly identified and used to support 128-bit atomics on systems that support these operations. This leads to increased efficiency in the implementation of both timers as well as other atomic operations.

In systems that support multiple hugepage sizes, the default hugepage size is now reported properly by the odp_sys_huge_page_size() API.

Numerous issues identified by the Coverity scan tool have been cleaned up and resolved, leading to improved robustness in the odp-linux reference implementation. In addition, several memory leaks identified by the valgrind tool have been fixed.

The Traffic Manager validation test has certain sub-tests that require a minimum of 2 CPUs to operate properly. These tests are now skipped with a warning when run on uniprocessor systems.

To get better formatting capabilities and future extensibility the ODP user documentation system has been switched from AsciiDoc to AsciiDoctor. There are a handful of notable changes as a result of this. Those submitting documentation patches should be aware of the following:

yields:

Note that this change only affects user documentation. Code documentation still uses Doxygen.

The images in the user guide have been trimmed to fit the page better. In addition, many new images have been added to better illustrate ODP API usage.

The ODP User’s Guide has been updated to include new sections documenting the use of the full Monarch packet APIs as well as the Timer APIs. In addition, the crypto, classification, and traffic manager API documentation has been improved to cover all of the latest Monarch features, and an introductory section on overall ODP packet flow architecture has been added.

The ODP Helper library now has its own document that describes these additional functions of use to ODP applications and validation tests.

The Process Guide, FAQ, CONTRIBUTING, and CHANGELOG files have been reworked for the new AsciiDoctor structure, leading to a more uniform method of presenting ODP support documentation. In addition, an option for generating both a "pure" ODP API document that defines the APIs in their implementation-independent forms as well as their implementation-dependent forms for the odp-linux reference implementation is now available.

The scheduler does not remove PKTIN queues following odp_pktio_close()

The timer validation tests have seen occasional failures when run on systems containing a larger number of CPUs (typically 24 or more).