diff options
74 files changed, 1715 insertions, 791 deletions
diff --git a/VNFs/DPPD-PROX/README b/VNFs/DPPD-PROX/README index 344214f0..1d7ad51f 100644 --- a/VNFs/DPPD-PROX/README +++ b/VNFs/DPPD-PROX/README @@ -24,74 +24,104 @@ finer grained network functions like QoS, Routing, load-balancing... Compiling and running this application -------------------------------------- -This application supports DPDK 16.04, 16.11, 16.11.1, 17.02, 17.05, 17.08, -17.11, 18.02, 18.05, 18.08, 18.11, 19.02, 19.05, 19.08, 19.11, 20.02 and -20.05 20.11 +This application supports DPDK 16.04, 16.07, 16.11, 17.02, 17.05, 17.08, +17.11, 18.02, 18.05, 18.08, 18.11, 19.02, 19.05, 19.08, 19.11, 20.02, 20.05, +20.08, 20.11, 21.02, 21.05, 21.08, 21.11, 22.03, 22.07, 22.11 -Meson compilation ------------------ -Support for 'make' has been deprecated in DPDK from v20.11 onward. +DPDK meson compilation +---------------------- +Compilation with meson and ninja is supported since DPDK 18.02, while support +for make has been removed in DPDK 20.11. Example: DPDK 20.11 installation with meson ------------------------------------------- +cd /your/path/for/dpdk/meson/compilation/ git clone http://dpdk.org/git/dpdk-stable +cd dpdk-stable/ git checkout 20.11 -meson build -cd build +meson setup build +# For DPDK 21.11 and above, please run: +# meson setup -Denable_driver_sdk=true build # instead, or +# meson configure build/ -Denable_driver_sdk=true # afterwards. +cd build/ ninja -ninja install -ldconfig +sudo ninja install +sudo ldconfig -PROX meson compilation (From the root of this repo) --------------------------------------------------- +PROX meson compilation +---------------------- Depending on the distribution in use the DPDK libraries will be installed in -different locations. The 'PKG_CONFIG_PATH' environment variable is used to +different locations. The PKG_CONFIG_PATH environment variable is used to point to the correct location. -For RHEL/CentOS systems: -export PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig -meson build -cd build -ninja - -Additional options can be passed to the meson build system. See the -'meson_options.txt' file for possible options - -Legacy Make compilation ------------------------ -The following commands assume that the following variables have been set: - -export RTE_SDK=/path/to/dpdk -export RTE_TARGET=x86_64-native-linuxapp-gcc - -IPSec is only supported in PROX starting from DPDK 17.11 -It will only be compiled if CONFIG_RTE_LIBRTE_PMD_AESNI_MB is -set in DPDK .config. This also requires AESNI_MULTI_BUFFER_LIB_PATH to point to -the multi-buffer library which can be downloaded from -<https://github.com/01org/intel-ipsec-mb>. -See doc/guides/cryptodevs/aesni_mb.rst within dpdk for more details - -Example: DPDK 17.05 installation with make +On RHEL/CentOS: export PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig +On Ubuntu: export PKG_CONFIG_PATH=/usr/local/lib/x86_64-linux-gnu/pkgconfig + +cd /the/path/where/you/cloned/this/repo/ +cd VNFs/DPPD-PROX/ +meson setup build +# Additional options can be specified on the 'meson setup' command line, or +# using 'meson configure' afterwards. See the meson_options.txt file for +# possible options. +ninja -C build/ + +Legacy DPDK make compilation +---------------------------- +Compilation with make has been supported until DPDK 20.08, and has been removed +in DPDK 20.11. + +The following commands require that the following environment variables are +properly defined, as shown in the examples below: +- RTE_SDK: absolute path to the DPDK sources directory, +- RTE_TARGET: target (arch-machine-execenv-toolchain format) for + which we are building DPDK, +- RTE_DEVEL_BUILD: set it to 'n' to prevent warnings to be considered + as errors when building DPDK inside a git tree. + +Configuration options can be defined, before building DPDK, by means of +appending lines into the config/defconfig_$RTE_TARGET file. + +For example, IPSec is supported in PROX (handle_esp.c) since DPDK 17.11, +although it has not been recently verified, but it only gets compiled when +CONFIG_RTE_LIBRTE_PMD_AESNI_MB=y has been defined in DPDK configuration. It also +requires AESNI_MULTI_BUFFER_LIB_PATH environment variable to point to the +"Multi-Buffer Crypto for IPsec" library, which can be downloaded from +https://github.com/intel/intel-ipsec-mb. See doc/guides/cryptodevs/aesni_mb.rst +within DPDK sources directory for more details. + +Example: DPDK 20.05 installation with make ------------------------------------------ +cd /your/path/for/dpdk/make/compilation/ git clone http://dpdk.org/git/dpdk -cd dpdk -git checkout v17.05 -make install T=$RTE_TARGET - -PROX make compilation ----------------- -The Makefile with this application expects RTE_SDK to point to the -root directory of DPDK (e.g. export RTE_SDK=/root/dpdk). If RTE_TARGET -has not been set, x86_64-native-linuxapp-gcc will be assumed. +cd dpdk/ +git checkout v20.05 +export RTE_SDK=$PWD +export RTE_TARGET=x86_64-native-linuxapp-gcc +export RTE_DEVEL_BUILD=n +# Edit config/defconfig_$RTE_TARGET file to define options as needed. +make config T=$RTE_TARGET O=$RTE_TARGET +make O=$RTE_TARGET + +Legacy PROX make compilation +---------------------------- +As explained above, PROX Makefile expects RTE_SDK to point to the DPDK sources +directory. If RTE_TARGET is not set, it defaults to x86_64-native-linuxapp-gcc. + +cd /the/path/where/you/cloned/this/repo/ +cd VNFs/DPPD-PROX/ +export RTE_SDK=/your/path/for/dpdk/make/compilation/./dpdk/ +export RTE_TARGET=x86_64-native-linuxapp-gcc +export RTE_DEVEL_BUILD=n +make Running PROX ------------ -After DPDK has been set up, run make from the directory where you have -extracted this application. A build directory will be created -containing the PROX executable. The usage of the application is shown -below. Note that this application assumes that all required ports have -been bound to the DPDK provided igb_uio driver. Refer to the "Getting -Started Guide - DPDK" document for more details. +After DPDK has been installed and PROX has been compiled, the build subdirectory +has been created and contains the PROX executable. The usage of the application +is shown below. Note that this application assumes that all required ports have +been bound to the DPDK provided igb_uio driver. Refer to the "Getting Started +Guide" (http://doc.dpdk.org/guides/linux_gsg/ or doc/guides/linux_gsg/*.rst in +DPDK sources directory) for more details. Usage: ./build/prox [-f CONFIG_FILE] [-l LOG_FILE] [-p] [-o DISPLAY] [-v] [-a|-e] \ [-m|-s|-i] [-n] [-w DEF] [-q] [-k] [-d] [-z] [-r VAL] [-u] [-t] diff --git a/VNFs/DPPD-PROX/commands.c b/VNFs/DPPD-PROX/commands.c index c5ffed84..a8953a68 100644 --- a/VNFs/DPPD-PROX/commands.c +++ b/VNFs/DPPD-PROX/commands.c @@ -1004,7 +1004,7 @@ void cmd_set_vlan_offload(uint8_t port_id, unsigned int val) } plog_info("setting vlan offload to %d\n", val); - if (val & ~(ETH_VLAN_STRIP_OFFLOAD | ETH_VLAN_FILTER_OFFLOAD | ETH_VLAN_EXTEND_OFFLOAD)) { + if (val & ~(RTE_ETH_VLAN_STRIP_OFFLOAD | RTE_ETH_VLAN_FILTER_OFFLOAD | RTE_ETH_VLAN_EXTEND_OFFLOAD)) { plog_info("wrong vlan offload value\n"); } int ret = rte_eth_dev_set_vlan_offload(port_id, val); diff --git a/VNFs/DPPD-PROX/defaults.c b/VNFs/DPPD-PROX/defaults.c index b93b7275..f5624b97 100644 --- a/VNFs/DPPD-PROX/defaults.c +++ b/VNFs/DPPD-PROX/defaults.c @@ -50,7 +50,11 @@ static const struct rte_eth_conf default_port_conf = { .rxmode = { .mq_mode = 0, - .max_rx_pkt_len = PROX_MTU + PROX_RTE_ETHER_HDR_LEN + PROX_RTE_ETHER_CRC_LEN +#if RTE_VERSION < RTE_VERSION_NUM(21,11,0,0) + .max_rx_pkt_len = PROX_MTU + PROX_RTE_ETHER_HDR_LEN + PROX_RTE_ETHER_CRC_LEN, +#else + .mtu = PROX_MTU, +#endif }, .rx_adv_conf = { .rss_conf = { @@ -226,9 +230,9 @@ void set_port_defaults(void) // CRC_STRIP becoming the default behavior in DPDK 18.08, and // DEV_RX_OFFLOAD_CRC_STRIP define has been deleted -#if defined (DEV_RX_OFFLOAD_CRC_STRIP) - prox_port_cfg[i].requested_rx_offload = DEV_RX_OFFLOAD_CRC_STRIP; +#if defined (RTE_ETH_RX_OFFLOAD_CRC_STRIP) + prox_port_cfg[i].requested_rx_offload = RTE_ETH_RX_OFFLOAD_CRC_STRIP; #endif - prox_port_cfg[i].requested_tx_offload = DEV_TX_OFFLOAD_IPV4_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM; + prox_port_cfg[i].requested_tx_offload = RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | RTE_ETH_TX_OFFLOAD_UDP_CKSUM; } } diff --git a/VNFs/DPPD-PROX/display_ports.c b/VNFs/DPPD-PROX/display_ports.c index 891ff6f6..d2140f1e 100644 --- a/VNFs/DPPD-PROX/display_ports.c +++ b/VNFs/DPPD-PROX/display_ports.c @@ -180,8 +180,8 @@ static void display_ports_draw_per_sec_stats(void) struct percent rx_percent; struct percent tx_percent; if (strcmp(prox_port_cfg[port_id].short_name, "i40e_vf") == 0) { -#if defined (DEV_RX_OFFLOAD_CRC_STRIP) - if (prox_port_cfg[port_id].requested_rx_offload & DEV_RX_OFFLOAD_CRC_STRIP) { +#if defined (RTE_ETH_RX_OFFLOAD_CRC_STRIP) + if (prox_port_cfg[port_id].requested_rx_offload & RTE_ETH_RX_OFFLOAD_CRC_STRIP) { rx_percent = calc_percent(last->rx_bytes - prev->rx_bytes + 20 * (last->rx_tot - prev->rx_tot), delta_t); tx_percent = calc_percent(last->tx_bytes - prev->tx_bytes + 24 * (last->tx_tot - prev->tx_tot), delta_t); } else { @@ -189,7 +189,7 @@ static void display_ports_draw_per_sec_stats(void) tx_percent = calc_percent(last->tx_bytes - prev->tx_bytes + 20 * (last->tx_tot - prev->tx_tot), delta_t); } } else { - if (prox_port_cfg[port_id].requested_rx_offload & DEV_RX_OFFLOAD_CRC_STRIP) { + if (prox_port_cfg[port_id].requested_rx_offload & RTE_ETH_RX_OFFLOAD_CRC_STRIP) { rx_percent = calc_percent(last->rx_bytes - prev->rx_bytes + 24 * (last->rx_tot - prev->rx_tot), delta_t); tx_percent = calc_percent(last->tx_bytes - prev->tx_bytes + 24 * (last->tx_tot - prev->tx_tot), delta_t); } else { @@ -198,8 +198,8 @@ static void display_ports_draw_per_sec_stats(void) } } #else -#if defined DEV_RX_OFFLOAD_KEEP_CRC - if (prox_port_cfg[port_id].requested_rx_offload & DEV_RX_OFFLOAD_KEEP_CRC ) { +#if defined RTE_ETH_RX_OFFLOAD_KEEP_CRC + if (prox_port_cfg[port_id].requested_rx_offload & RTE_ETH_RX_OFFLOAD_KEEP_CRC ) { rx_percent = calc_percent(last->rx_bytes - prev->rx_bytes + 20 * (last->rx_tot - prev->rx_tot), delta_t); tx_percent = calc_percent(last->tx_bytes - prev->tx_bytes + 20 * (last->tx_tot - prev->tx_tot), delta_t); } else { @@ -207,7 +207,7 @@ static void display_ports_draw_per_sec_stats(void) tx_percent = calc_percent(last->tx_bytes - prev->tx_bytes + 24 * (last->tx_tot - prev->tx_tot), delta_t); } } else { - if (prox_port_cfg[port_id].requested_rx_offload & DEV_RX_OFFLOAD_KEEP_CRC ) { + if (prox_port_cfg[port_id].requested_rx_offload & RTE_ETH_RX_OFFLOAD_KEEP_CRC ) { rx_percent = calc_percent(last->rx_bytes - prev->rx_bytes + 20 * (last->rx_tot - prev->rx_tot), delta_t); tx_percent = calc_percent(last->tx_bytes - prev->tx_bytes + 20 * (last->tx_tot - prev->tx_tot), delta_t); } else { @@ -216,7 +216,7 @@ static void display_ports_draw_per_sec_stats(void) } } #else -#error neither DEV_RX_OFFLOAD_CRC_STRIP or DEV_RX_OFFLOAD_KEEP_CRC is defined +#error neither RTE_ETH_RX_OFFLOAD_CRC_STRIP or RTE_ETH_RX_OFFLOAD_KEEP_CRC is defined #endif #endif diff --git a/VNFs/DPPD-PROX/handle_cgnat.c b/VNFs/DPPD-PROX/handle_cgnat.c index b3c22c41..9ce63b20 100644 --- a/VNFs/DPPD-PROX/handle_cgnat.c +++ b/VNFs/DPPD-PROX/handle_cgnat.c @@ -797,6 +797,7 @@ static int lua_to_hash_nat(struct task_args *targ, struct lua_State *L, enum lua .key_len = sizeof(struct private_key), .hash_func = rte_hash_crc, .hash_func_init_val = 0, + .socket_id = socket, }; plogx_info("hash table name = %s\n", hash_params.name); struct private_key private_key; @@ -964,7 +965,7 @@ static void init_task_nat(struct task_base *tbase, struct task_args *targ) struct prox_port_cfg *port = find_reachable_port(targ); if (port) { - task->offload_crc = port->requested_tx_offload & (DEV_TX_OFFLOAD_IPV4_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM); + task->offload_crc = port->requested_tx_offload & (RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | RTE_ETH_TX_OFFLOAD_UDP_CKSUM); } } diff --git a/VNFs/DPPD-PROX/handle_esp.c b/VNFs/DPPD-PROX/handle_esp.c index 447fcfa2..a78130bf 100644 --- a/VNFs/DPPD-PROX/handle_esp.c +++ b/VNFs/DPPD-PROX/handle_esp.c @@ -34,7 +34,6 @@ #include "defines.h" #include <rte_ip.h> #include <rte_cryptodev.h> -#include <rte_cryptodev_pmd.h> #include <rte_bus_vdev.h> #include "prox_port_cfg.h" #include "prox_compat.h" @@ -55,9 +54,9 @@ typedef unsigned char u8; #define MAX_SESSIONS 1024 #define POOL_CACHE_SIZE 128 -#define NUM_OPS 256 - -struct task_esp_enc { +//#define NUM_OPS 256 +#define NUM_OPS 128 +struct task_esp { struct task_base base; uint8_t cdev_id; uint16_t qp_id; @@ -69,19 +68,12 @@ struct task_esp_enc { struct rte_mempool *session_pool; struct rte_cryptodev_sym_session *sess; struct rte_crypto_op *ops_burst[NUM_OPS]; -}; - -struct task_esp_dec { - struct task_base base; - uint8_t cdev_id; - uint16_t qp_id; - uint32_t local_ipv4; - prox_rte_ether_addr local_mac; - prox_rte_ether_addr dst_mac; - struct rte_mempool *crypto_op_pool; - struct rte_mempool *session_pool; - struct rte_cryptodev_sym_session *sess; - struct rte_crypto_op *ops_burst[NUM_OPS]; + unsigned len; //number of ops ready to be enqueued + uint32_t pkts_in_flight; // difference between enqueued and dequeued + uint8_t (*handle_esp_finish)(struct task_esp *task, + struct rte_mbuf *mbuf, uint8_t status); + uint8_t (*handle_esp_ah)(struct task_esp *task, struct rte_mbuf *mbuf, + struct rte_crypto_op *cop); }; static uint8_t hmac_sha1_key[] = { @@ -117,253 +109,115 @@ static void printf_cdev_info(uint8_t cdev_id) } } -#if 0 static uint8_t get_cdev_id(void) { - //crypto devices must be configured in the config file - //eal=-b 0000:00:03.0 --vdev crypto_aesni_mb0 --vdev crypto_aesni_mb1 - - static uint8_t cdev_id=0; - PROX_PANIC(cdev_id+1 > rte_cryptodev_count(), "not enough crypto devices\n"); - //eal=-b 0000:00:03.0 --vdev crypto_aesni_mb0 --vdev crypto_aesni_mb1 - return cdev_id++; -} -#else -static uint8_t get_cdev_id(void) -{ - static uint8_t cdev_id=0; + static uint8_t last_unused_cdev_id=0; char name[64]={0}; - - sprintf(name, "crypto_aesni_mb%d", cdev_id); - - int cdev_id1 = rte_cryptodev_get_dev_id(name); - if (cdev_id1 >= 0){ - plog_info("crypto dev %d preconfigured\n", cdev_id1); - ++cdev_id; - return cdev_id1; + uint8_t cdev_count, cdev_id; + + cdev_count = rte_cryptodev_count(); + plog_info("crypto dev count: %d \n", cdev_count); + for (cdev_id = last_unused_cdev_id; cdev_id < cdev_count; cdev_id++) { + if (cdev_id != 1) { + printf_cdev_info(cdev_id); + last_unused_cdev_id = cdev_id + 1; + return cdev_id; + } } + sprintf(name, "crypto_aesni_mb%d", cdev_count); + #if RTE_VERSION < RTE_VERSION_NUM(18,8,0,0) int ret = rte_vdev_init(name, "max_nb_queue_pairs=8,max_nb_sessions=1024,socket_id=0"); #else int ret = rte_vdev_init(name, "max_nb_queue_pairs=8,socket_id=0"); #endif PROX_PANIC(ret != 0, "Failed rte_vdev_init\n"); + cdev_id = rte_cryptodev_get_dev_id(name); - return cdev_id++; + printf_cdev_info(cdev_id); + last_unused_cdev_id = cdev_id + 1; + return cdev_id; } -#endif -static void init_task_esp_enc(struct task_base *tbase, struct task_args *targ) +static inline uint8_t handle_enc_finish(struct task_esp *task, + struct rte_mbuf *mbuf, uint8_t status) { - struct task_esp_enc *task = (struct task_esp_enc *)tbase; - - tbase->flags |= FLAG_NEVER_FLUSH; - - uint8_t lcore_id = targ->lconf->id; - char name[64]; - sprintf(name, "core_%03u_crypto_pool", lcore_id); - task->crypto_op_pool = rte_crypto_op_pool_create(name, RTE_CRYPTO_OP_TYPE_SYMMETRIC, - 8192, 128, MAXIMUM_IV_LENGTH, rte_socket_id()); - PROX_PANIC(task->crypto_op_pool == NULL, "Can't create ENC CRYPTO_OP_POOL\n"); - - task->cdev_id = get_cdev_id(); - - struct rte_cryptodev_config cdev_conf; - cdev_conf.nb_queue_pairs = 2; - //cdev_conf.socket_id = SOCKET_ID_ANY; - cdev_conf.socket_id = rte_socket_id(); - rte_cryptodev_configure(task->cdev_id, &cdev_conf); - - unsigned int session_size = rte_cryptodev_sym_get_private_session_size(task->cdev_id); - plog_info("rte_cryptodev_sym_get_private_session_size=%d\n", session_size); - sprintf(name, "core_%03u_session_pool", lcore_id); - task->session_pool = rte_mempool_create(name, - MAX_SESSIONS, - session_size, - POOL_CACHE_SIZE, - 0, NULL, NULL, NULL, - NULL, rte_socket_id(), - 0); - PROX_PANIC(task->session_pool == NULL, "Failed rte_mempool_create\n"); - - task->qp_id=0; - plog_info("enc: task->qp_id=%u\n", task->qp_id); - struct prox_rte_cryptodev_qp_conf qp_conf; - qp_conf.nb_descriptors = 128; - qp_conf.mp_session = task->session_pool; - prox_rte_cryptodev_queue_pair_setup(task->cdev_id, task->qp_id, &qp_conf, rte_cryptodev_socket_id(task->cdev_id)); - - int ret = rte_cryptodev_start(task->cdev_id); - PROX_PANIC(ret < 0, "Failed to start device\n"); - - struct rte_cryptodev *dev; - dev = rte_cryptodev_pmd_get_dev(task->cdev_id); - PROX_PANIC(dev->attached != RTE_CRYPTODEV_ATTACHED, "No ENC cryptodev attached\n"); - - //Setup Cipher Parameters - struct rte_crypto_sym_xform cipher_xform = {0}; - struct rte_crypto_sym_xform auth_xform = {0}; - - cipher_xform.type = RTE_CRYPTO_SYM_XFORM_CIPHER; - cipher_xform.next = &auth_xform; - - cipher_xform.cipher.algo = RTE_CRYPTO_CIPHER_AES_CBC; - cipher_xform.cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT; - cipher_xform.cipher.key.data = aes_cbc_key; - cipher_xform.cipher.key.length = CIPHER_KEY_LENGTH_AES_CBC; - - cipher_xform.cipher.iv.offset = IV_OFFSET; - cipher_xform.cipher.iv.length = CIPHER_IV_LENGTH_AES_CBC; - - //Setup HMAC Parameters - auth_xform.type = RTE_CRYPTO_SYM_XFORM_AUTH; - auth_xform.next = NULL; - auth_xform.auth.op = RTE_CRYPTO_AUTH_OP_GENERATE; - auth_xform.auth.algo = RTE_CRYPTO_AUTH_SHA1_HMAC; - auth_xform.auth.key.length = DIGEST_BYTE_LENGTH_SHA1; - auth_xform.auth.key.data = hmac_sha1_key; - auth_xform.auth.digest_length = DIGEST_BYTE_LENGTH_SHA1; - - auth_xform.auth.iv.offset = 0; - auth_xform.auth.iv.length = 0; - - task->sess = rte_cryptodev_sym_session_create(task->session_pool); - PROX_PANIC(task->sess == NULL, "Failed to create ENC session\n"); - - ret = rte_cryptodev_sym_session_init(task->cdev_id, task->sess, &cipher_xform, task->session_pool); - PROX_PANIC(ret < 0, "Failed sym_session_init\n"); - - //TODO: doublecheck task->ops_burst lifecycle! - if (rte_crypto_op_bulk_alloc(task->crypto_op_pool, - RTE_CRYPTO_OP_TYPE_SYMMETRIC, - task->ops_burst, NUM_OPS) != NUM_OPS) { - PROX_PANIC(1, "Failed to allocate ENC crypto operations\n"); - } - - task->local_ipv4 = rte_cpu_to_be_32(targ->local_ipv4); - task->remote_ipv4 = rte_cpu_to_be_32(targ->remote_ipv4); - //memcpy(&task->src_mac, &prox_port_cfg[task->base.tx_params_hw.tx_port_queue->port].eth_addr, sizeof(prox_rte_ether_addr)); - struct prox_port_cfg *port = find_reachable_port(targ); - memcpy(&task->local_mac, &port->eth_addr, sizeof(prox_rte_ether_addr)); - - if (targ->flags & TASK_ARG_DST_MAC_SET){ - memcpy(&task->dst_mac, &targ->edaddr, sizeof(task->dst_mac)); - plog_info("TASK_ARG_DST_MAC_SET ("MAC_BYTES_FMT")\n", MAC_BYTES(task->dst_mac.addr_bytes)); - //prox_rte_ether_addr_copy(&ptask->dst_mac, &peth->d_addr); - //rte_memcpy(hdr, task->src_dst_mac, sizeof(task->src_dst_mac)); - } + prox_rte_ether_hdr *peth = rte_pktmbuf_mtod(mbuf, + prox_rte_ether_hdr *); + prox_rte_ipv4_hdr* pip4 = (prox_rte_ipv4_hdr *)(peth + 1); + pip4->dst_addr = task->remote_ipv4; + pip4->src_addr = task->local_ipv4; + prox_ip_cksum(mbuf, pip4, sizeof(prox_rte_ether_hdr), + sizeof(prox_rte_ipv4_hdr), 1); + return 0; } -static void init_task_esp_dec(struct task_base *tbase, struct task_args *targ) +static inline uint8_t handle_dec_finish(struct task_esp *task, + struct rte_mbuf *mbuf, uint8_t status) { - struct task_esp_dec *task = (struct task_esp_dec *)tbase; - - tbase->flags |= FLAG_NEVER_FLUSH; - - uint8_t lcore_id = targ->lconf->id; - char name[64]; - sprintf(name, "core_%03u_crypto_pool", lcore_id); - task->crypto_op_pool = rte_crypto_op_pool_create(name, RTE_CRYPTO_OP_TYPE_SYMMETRIC, - 8192, 128, MAXIMUM_IV_LENGTH, rte_socket_id()); - PROX_PANIC(task->crypto_op_pool == NULL, "Can't create DEC CRYPTO_OP_POOL\n"); - - task->cdev_id = get_cdev_id(); - struct rte_cryptodev_config cdev_conf; - cdev_conf.nb_queue_pairs = 2; - cdev_conf.socket_id = SOCKET_ID_ANY; - cdev_conf.socket_id = rte_socket_id(); - rte_cryptodev_configure(task->cdev_id, &cdev_conf); - - unsigned int session_size = rte_cryptodev_sym_get_private_session_size(task->cdev_id); - plog_info("rte_cryptodev_sym_get_private_session_size=%d\n", session_size); - sprintf(name, "core_%03u_session_pool", lcore_id); - task->session_pool = rte_mempool_create(name, - MAX_SESSIONS, - session_size, - POOL_CACHE_SIZE, - 0, NULL, NULL, NULL, - NULL, rte_socket_id(), - 0); - PROX_PANIC(task->session_pool == NULL, "Failed rte_mempool_create\n"); - - task->qp_id=0; - plog_info("dec: task->qp_id=%u\n", task->qp_id); - struct prox_rte_cryptodev_qp_conf qp_conf; - qp_conf.nb_descriptors = 128; - qp_conf.mp_session = task->session_pool; - prox_rte_cryptodev_queue_pair_setup(task->cdev_id, task->qp_id, &qp_conf, rte_cryptodev_socket_id(task->cdev_id)); - - int ret = rte_cryptodev_start(task->cdev_id); - PROX_PANIC(ret < 0, "Failed to start device\n"); - - struct rte_cryptodev *dev; - dev = rte_cryptodev_pmd_get_dev(task->cdev_id); - PROX_PANIC(dev->attached != RTE_CRYPTODEV_ATTACHED, "No ENC cryptodev attached\n"); - - //Setup Cipher Parameters - struct rte_crypto_sym_xform cipher_xform = {0}; - struct rte_crypto_sym_xform auth_xform = {0}; - - cipher_xform.type = RTE_CRYPTO_SYM_XFORM_CIPHER; - cipher_xform.next = NULL; - cipher_xform.cipher.algo = RTE_CRYPTO_CIPHER_AES_CBC; - cipher_xform.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; - cipher_xform.cipher.key.data = aes_cbc_key; - cipher_xform.cipher.key.length = CIPHER_KEY_LENGTH_AES_CBC; - - cipher_xform.cipher.iv.offset = IV_OFFSET; - cipher_xform.cipher.iv.length = CIPHER_IV_LENGTH_AES_CBC; - - //Setup HMAC Parameters - auth_xform.type = RTE_CRYPTO_SYM_XFORM_AUTH; - auth_xform.next = &cipher_xform; - auth_xform.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY; - auth_xform.auth.algo = RTE_CRYPTO_AUTH_SHA1_HMAC; - auth_xform.auth.key.length = DIGEST_BYTE_LENGTH_SHA1; - auth_xform.auth.key.data = hmac_sha1_key; - auth_xform.auth.digest_length = DIGEST_BYTE_LENGTH_SHA1; - - auth_xform.auth.iv.offset = 0; - auth_xform.auth.iv.length = 0; - - task->sess = rte_cryptodev_sym_session_create(task->session_pool); - PROX_PANIC(task->sess == NULL, "Failed to create ENC session\n"); - - ret = rte_cryptodev_sym_session_init(task->cdev_id, task->sess, &cipher_xform, task->session_pool); - PROX_PANIC(ret < 0, "Failed sym_session_init\n"); - - //TODO: doublecheck task->ops_burst lifecycle! - if (rte_crypto_op_bulk_alloc(task->crypto_op_pool, - RTE_CRYPTO_OP_TYPE_SYMMETRIC, - task->ops_burst, NUM_OPS) != NUM_OPS) { - PROX_PANIC(1, "Failed to allocate DEC crypto operations\n"); - } - - task->local_ipv4 = rte_cpu_to_be_32(targ->local_ipv4); - //memcpy(&task->src_mac, &prox_port_cfg[task->base.tx_params_hw.tx_port_queue->port].eth_addr, sizeof(prox_rte_ether_addr)); - struct prox_port_cfg *port = find_reachable_port(targ); - memcpy(&task->local_mac, &port->eth_addr, sizeof(prox_rte_ether_addr)); + if (likely(status == RTE_CRYPTO_OP_STATUS_SUCCESS)) { + u8* m = rte_pktmbuf_mtod(mbuf, u8*); + rte_memcpy(m + sizeof(prox_rte_ipv4_hdr) + + sizeof(struct prox_esp_hdr) + + CIPHER_IV_LENGTH_AES_CBC, m, + sizeof(prox_rte_ether_hdr)); + m = (u8*)rte_pktmbuf_adj(mbuf, sizeof(prox_rte_ipv4_hdr) + + sizeof(struct prox_esp_hdr) + + CIPHER_IV_LENGTH_AES_CBC); + prox_rte_ipv4_hdr* pip4 = (prox_rte_ipv4_hdr *)(m + + sizeof(prox_rte_ether_hdr)); + + if (unlikely((pip4->version_ihl >> 4) != 4)) { + // plog_info("non IPv4 packet after esp dec %i\n", + // pip4->version_ihl); + // plogdx_info(mbuf, "DEC TX: "); + return OUT_DISCARD; + } + if (pip4->time_to_live) { + pip4->time_to_live--; + } + else { + plog_info("TTL = 0 => Dropping\n"); + return OUT_DISCARD; + } + uint16_t ipv4_length = rte_be_to_cpu_16(pip4->total_length); + int len = rte_pktmbuf_pkt_len(mbuf); + rte_pktmbuf_trim(mbuf, len - sizeof(prox_rte_ether_hdr) - + ipv4_length); - if (targ->flags & TASK_ARG_DST_MAC_SET){ - memcpy(&task->dst_mac, &targ->edaddr, sizeof(task->dst_mac)); - plog_info("TASK_ARG_DST_MAC_SET ("MAC_BYTES_FMT")\n", MAC_BYTES(task->dst_mac.addr_bytes)); - //prox_rte_ether_addr_copy(&ptask->dst_mac, &peth->d_addr); - //rte_memcpy(hdr, task->src_dst_mac, sizeof(task->src_dst_mac)); +#if 0 + do_ipv4_swap(task, mbuf); +#else + prox_rte_ether_hdr *peth = rte_pktmbuf_mtod(mbuf, + prox_rte_ether_hdr *); + prox_rte_ether_addr_copy(&task->local_mac, &peth->s_addr); + prox_rte_ether_addr_copy(&task->dst_mac, &peth->d_addr); + //rte_memcpy(peth, task->dst_mac, sizeof(task->dst_mac)); +#endif + pip4->dst_addr = task->remote_ipv4; + pip4->src_addr = task->local_ipv4; + prox_ip_cksum(mbuf, pip4, sizeof(prox_rte_ether_hdr), + sizeof(prox_rte_ipv4_hdr), 1); + return 0; + } + else { + return OUT_DISCARD; } - } -static inline uint8_t handle_esp_ah_enc(struct task_esp_enc *task, struct rte_mbuf *mbuf, struct rte_crypto_op *cop) +static inline uint8_t handle_esp_ah_enc(struct task_esp *task, + struct rte_mbuf *mbuf, struct rte_crypto_op *cop) { u8 *data; - prox_rte_ether_hdr *peth = rte_pktmbuf_mtod(mbuf, prox_rte_ether_hdr *); + prox_rte_ether_hdr *peth = rte_pktmbuf_mtod(mbuf, + prox_rte_ether_hdr *); prox_rte_ipv4_hdr* pip4 = (prox_rte_ipv4_hdr *)(peth + 1); uint16_t ipv4_length = rte_be_to_cpu_16(pip4->total_length); struct rte_crypto_sym_op *sym_cop = cop->sym; if (unlikely((pip4->version_ihl >> 4) != 4)) { - plog_info("Received non IPv4 packet at esp enc %i\n", pip4->version_ihl); - plogdx_info(mbuf, "ENC RX: "); + plog_info("Received non IPv4 packet at esp enc %i\n", + pip4->version_ihl); return OUT_DISCARD; } if (pip4->time_to_live) { @@ -389,7 +243,8 @@ static inline uint8_t handle_esp_ah_enc(struct task_esp_enc *task, struct rte_mb encrypt_len += padding; } - const int extra_space = sizeof(prox_rte_ipv4_hdr) + sizeof(struct prox_esp_hdr) + CIPHER_IV_LENGTH_AES_CBC; + const int extra_space = sizeof(prox_rte_ipv4_hdr) + + sizeof(struct prox_esp_hdr) + CIPHER_IV_LENGTH_AES_CBC; prox_rte_ether_addr src_mac = peth->s_addr; prox_rte_ether_addr dst_mac = peth->d_addr; @@ -399,7 +254,8 @@ static inline uint8_t handle_esp_ah_enc(struct task_esp_enc *task, struct rte_mb uint8_t version_ihl = pip4->version_ihl; peth = (prox_rte_ether_hdr *)rte_pktmbuf_prepend(mbuf, extra_space); // encap + prefix - peth = (prox_rte_ether_hdr *)rte_pktmbuf_append(mbuf, 0 + 1 + 1 + padding + 4 + DIGEST_BYTE_LENGTH_SHA1); // padding + pad_len + next_head + seqn + ICV pad + ICV + peth = (prox_rte_ether_hdr *)rte_pktmbuf_append(mbuf, 0 + 1 + 1 + + padding + 4 + DIGEST_BYTE_LENGTH_SHA1); // padding + pad_len + next_head + seqn + ICV pad + ICV peth = rte_pktmbuf_mtod(mbuf, prox_rte_ether_hdr *); l1 = rte_pktmbuf_pkt_len(mbuf); peth->ether_type = ETYPE_IPv4; @@ -419,11 +275,15 @@ static inline uint8_t handle_esp_ah_enc(struct task_esp_enc *task, struct rte_mb pip4->time_to_live = ttl; pip4->next_proto_id = IPPROTO_ESP; // 50 for ESP, ip in ip next proto trailer pip4->version_ihl = version_ihl; // 20 bytes, ipv4 - pip4->total_length = rte_cpu_to_be_16(ipv4_length + sizeof(prox_rte_ipv4_hdr) + sizeof(struct prox_esp_hdr) + CIPHER_IV_LENGTH_AES_CBC + padding + 1 + 1 + DIGEST_BYTE_LENGTH_SHA1); // iphdr+SPI+SN+IV+payload+padding+padlen+next header + crc + auth + pip4->total_length = rte_cpu_to_be_16(ipv4_length + + sizeof(prox_rte_ipv4_hdr) + sizeof(struct prox_esp_hdr) + + CIPHER_IV_LENGTH_AES_CBC + padding + 1 + 1 + + DIGEST_BYTE_LENGTH_SHA1); // iphdr+SPI+SN+IV+payload+padding+padlen+next header + crc + auth pip4->packet_id = 0x0101; pip4->type_of_service = 0; pip4->time_to_live = 64; - prox_ip_cksum(mbuf, pip4, sizeof(prox_rte_ether_hdr), sizeof(prox_rte_ipv4_hdr), 1); + prox_ip_cksum(mbuf, pip4, sizeof(prox_rte_ether_hdr), + sizeof(prox_rte_ipv4_hdr), 1); data = (u8*)(pip4 + 1); #if 0 @@ -434,17 +294,20 @@ static inline uint8_t handle_esp_ah_enc(struct task_esp_enc *task, struct rte_mb pesp->spi = src_addr;//for simplicity assume 1 tunnel per source ip static u32 sn = 0; pesp->seq = ++sn; - pesp->spi=0xAAAAAAAA;//debug - pesp->seq =0xBBBBBBBB;//debug +// pesp->spi=0xAAAAAAAA;//debug +// pesp->seq =0xBBBBBBBB;//debug #endif u8 *padl = (u8*)data + (8 + encrypt_len - 2 + CIPHER_IV_LENGTH_AES_CBC); // No ESN yet. (-2 means NH is crypted) //padl += CIPHER_IV_LENGTH_AES_CBC; *padl = padding; *(padl + 1) = 4; // ipv4 in 4 - sym_cop->auth.digest.data = data + 8 + CIPHER_IV_LENGTH_AES_CBC + encrypt_len; + sym_cop->auth.digest.data = data + 8 + CIPHER_IV_LENGTH_AES_CBC + + encrypt_len; //sym_cop->auth.digest.phys_addr = rte_pktmbuf_mtophys_offset(mbuf, (sizeof(prox_rte_ether_hdr) + sizeof(prox_rte_ipv4_hdr) + 8 + CIPHER_IV_LENGTH_AES_CBC + encrypt_len)); - sym_cop->auth.digest.phys_addr = rte_pktmbuf_iova_offset(mbuf, (sizeof(prox_rte_ether_hdr) + sizeof(prox_rte_ipv4_hdr) + 8 + CIPHER_IV_LENGTH_AES_CBC + encrypt_len)); + sym_cop->auth.digest.phys_addr = rte_pktmbuf_iova_offset(mbuf, + (sizeof(prox_rte_ether_hdr) + sizeof(prox_rte_ipv4_hdr) + + 8 + CIPHER_IV_LENGTH_AES_CBC + encrypt_len)); //sym_cop->auth.digest.length = DIGEST_BYTE_LENGTH_SHA1; //sym_cop->cipher.iv.data = data + 8; @@ -465,25 +328,31 @@ static inline uint8_t handle_esp_ah_enc(struct task_esp_enc *task, struct rte_mb #else //uint64_t *iv = (uint64_t *)(pesp + 1); //memset(iv, 0, CIPHER_IV_LENGTH_AES_CBC); - sym_cop->cipher.data.offset = sizeof(prox_rte_ether_hdr) + sizeof(prox_rte_ipv4_hdr) + sizeof(struct prox_esp_hdr); + sym_cop->cipher.data.offset = sizeof(prox_rte_ether_hdr) + + sizeof(prox_rte_ipv4_hdr) + sizeof(struct prox_esp_hdr); sym_cop->cipher.data.length = encrypt_len + CIPHER_IV_LENGTH_AES_CBC; #endif - sym_cop->auth.data.offset = sizeof(prox_rte_ether_hdr) + sizeof(prox_rte_ipv4_hdr); - sym_cop->auth.data.length = sizeof(struct prox_esp_hdr) + CIPHER_IV_LENGTH_AES_CBC + encrypt_len;// + 4;// FIXME + sym_cop->auth.data.offset = sizeof(prox_rte_ether_hdr) + + sizeof(prox_rte_ipv4_hdr); + sym_cop->auth.data.length = sizeof(struct prox_esp_hdr) + + CIPHER_IV_LENGTH_AES_CBC + encrypt_len;// + 4;// FIXME sym_cop->m_src = mbuf; rte_crypto_op_attach_sym_session(cop, task->sess); + //cop->type = RTE_CRYPTO_OP_TYPE_SYMMETRIC; //cop->status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED; return 0; } -static inline uint8_t handle_esp_ah_dec(struct task_esp_dec *task, struct rte_mbuf *mbuf, struct rte_crypto_op *cop) +static inline uint8_t handle_esp_ah_dec(struct task_esp *task, + struct rte_mbuf *mbuf, struct rte_crypto_op *cop) { struct rte_crypto_sym_op *sym_cop = cop->sym; - prox_rte_ether_hdr *peth = rte_pktmbuf_mtod(mbuf, prox_rte_ether_hdr *); + prox_rte_ether_hdr *peth = rte_pktmbuf_mtod(mbuf, + prox_rte_ether_hdr *); prox_rte_ipv4_hdr* pip4 = (prox_rte_ipv4_hdr *)(peth + 1); uint16_t ipv4_length = rte_be_to_cpu_16(pip4->total_length); u8 *data = (u8*)(pip4 + 1); @@ -496,9 +365,12 @@ static inline uint8_t handle_esp_ah_dec(struct task_esp_dec *task, struct rte_mb rte_crypto_op_attach_sym_session(cop, task->sess); - sym_cop->auth.digest.data = (unsigned char *)((unsigned char*)pip4 + ipv4_length - DIGEST_BYTE_LENGTH_SHA1); + sym_cop->auth.digest.data = (unsigned char *)((unsigned char*)pip4 + + ipv4_length - DIGEST_BYTE_LENGTH_SHA1); //sym_cop->auth.digest.phys_addr = rte_pktmbuf_mtophys_offset(mbuf, sizeof(prox_rte_ether_hdr) + sizeof(prox_rte_ipv4_hdr) + sizeof(struct prox_esp_hdr)); // FIXME - sym_cop->auth.digest.phys_addr = rte_pktmbuf_iova_offset(mbuf, sizeof(prox_rte_ether_hdr) + sizeof(prox_rte_ipv4_hdr) + sizeof(struct prox_esp_hdr)); + sym_cop->auth.digest.phys_addr = rte_pktmbuf_iova_offset(mbuf, + sizeof(prox_rte_ether_hdr) + sizeof(prox_rte_ipv4_hdr) + + sizeof(struct prox_esp_hdr)); //sym_cop->auth.digest.length = DIGEST_BYTE_LENGTH_SHA1; //sym_cop->cipher.iv.data = (uint8_t *)data + 8; @@ -516,19 +388,25 @@ static inline uint8_t handle_esp_ah_dec(struct task_esp_dec *task, struct rte_mb CIPHER_IV_LENGTH_AES_CBC); #endif - sym_cop->auth.data.offset = sizeof(prox_rte_ether_hdr) + sizeof(prox_rte_ipv4_hdr); - sym_cop->auth.data.length = ipv4_length - sizeof(prox_rte_ipv4_hdr) - 4 - CIPHER_IV_LENGTH_AES_CBC; + sym_cop->auth.data.offset = sizeof(prox_rte_ether_hdr) + + sizeof(prox_rte_ipv4_hdr); + sym_cop->auth.data.length = ipv4_length - sizeof(prox_rte_ipv4_hdr) - 4 - + CIPHER_IV_LENGTH_AES_CBC; - sym_cop->cipher.data.offset = sizeof(prox_rte_ether_hdr) + sizeof(prox_rte_ipv4_hdr) + sizeof(struct prox_esp_hdr) + CIPHER_IV_LENGTH_AES_CBC; - sym_cop->cipher.data.length = ipv4_length - sizeof(prox_rte_ipv4_hdr) - CIPHER_IV_LENGTH_AES_CBC - 28; // FIXME + sym_cop->cipher.data.offset = sizeof(prox_rte_ether_hdr) + + sizeof(prox_rte_ipv4_hdr) + sizeof(struct prox_esp_hdr) + + CIPHER_IV_LENGTH_AES_CBC; + sym_cop->cipher.data.length = ipv4_length - sizeof(prox_rte_ipv4_hdr) - + CIPHER_IV_LENGTH_AES_CBC - 28; // FIXME sym_cop->m_src = mbuf; return 0; } -static inline void do_ipv4_swap(struct task_esp_dec *task, struct rte_mbuf *mbuf) +static inline void do_ipv4_swap(struct task_esp *task, struct rte_mbuf *mbuf) { - prox_rte_ether_hdr *peth = rte_pktmbuf_mtod(mbuf, prox_rte_ether_hdr *); + prox_rte_ether_hdr *peth = rte_pktmbuf_mtod(mbuf, + prox_rte_ether_hdr *); prox_rte_ether_addr src_mac = peth->s_addr; prox_rte_ether_addr dst_mac = peth->d_addr; uint32_t src_ip, dst_ip; @@ -544,162 +422,312 @@ static inline void do_ipv4_swap(struct task_esp_dec *task, struct rte_mbuf *mbuf prox_rte_ether_addr_copy(&task->local_mac, &peth->s_addr); } -static inline uint8_t handle_esp_ah_dec_finish(struct task_esp_dec *task, struct rte_mbuf *mbuf) + +static void init_task_esp_enc(struct task_base *tbase, struct task_args *targ) { - prox_rte_ether_hdr *peth = rte_pktmbuf_mtod(mbuf, prox_rte_ether_hdr *); - rte_memcpy(((u8*)peth) + sizeof(prox_rte_ether_hdr), ((u8*)peth) + sizeof(prox_rte_ether_hdr) + - + sizeof(prox_rte_ipv4_hdr) + 4 + 4 + CIPHER_IV_LENGTH_AES_CBC, sizeof(prox_rte_ipv4_hdr));// next hdr, padding - prox_rte_ipv4_hdr* pip4 = (prox_rte_ipv4_hdr *)(peth + 1); + struct task_esp *task = (struct task_esp *)tbase; + unsigned int session_size; - if (unlikely((pip4->version_ihl >> 4) != 4)) { - plog_info("non IPv4 packet after esp dec %i\n", pip4->version_ihl); - plogdx_info(mbuf, "DEC TX: "); - return OUT_DISCARD; - } - if (pip4->time_to_live) { - pip4->time_to_live--; - } - else { - plog_info("TTL = 0 => Dropping\n"); - return OUT_DISCARD; - } - uint16_t ipv4_length = rte_be_to_cpu_16(pip4->total_length); - rte_memcpy(((u8*)peth) + sizeof(prox_rte_ether_hdr) + sizeof(prox_rte_ipv4_hdr), - ((u8*)peth) + sizeof(prox_rte_ether_hdr) + - + 2 * sizeof(prox_rte_ipv4_hdr) + 4 + 4 + CIPHER_IV_LENGTH_AES_CBC, ipv4_length - sizeof(prox_rte_ipv4_hdr)); + tbase->flags |= TBASE_FLAG_NEVER_FLUSH; - int len = rte_pktmbuf_pkt_len(mbuf); - rte_pktmbuf_trim(mbuf, len - sizeof(prox_rte_ether_hdr) - ipv4_length); - peth = rte_pktmbuf_mtod(mbuf, prox_rte_ether_hdr *); + uint8_t lcore_id = targ->lconf->id; + char name[64]; + task->handle_esp_finish = handle_enc_finish; + task->handle_esp_ah = handle_esp_ah_enc; + task->len = 0; + task->pkts_in_flight = 0; + sprintf(name, "core_%03u_crypto_pool", lcore_id); + task->crypto_op_pool = rte_crypto_op_pool_create(name, + RTE_CRYPTO_OP_TYPE_SYMMETRIC, targ->nb_mbuf, 128, + MAXIMUM_IV_LENGTH, rte_socket_id()); + plog_info("rte_crypto_op_pool_create nb_elements =%d\n", + targ->nb_mbuf); + PROX_PANIC(task->crypto_op_pool == NULL, "Can't create ENC \ + CRYPTO_OP_POOL\n"); -#if 0 - do_ipv4_swap(task, mbuf); -#else - prox_rte_ether_addr_copy(&task->local_mac, &peth->s_addr); - prox_rte_ether_addr_copy(&task->dst_mac, &peth->d_addr); - //rte_memcpy(peth, task->dst_mac, sizeof(task->dst_mac)); -#endif - prox_ip_cksum(mbuf, pip4, sizeof(prox_rte_ether_hdr), sizeof(prox_rte_ipv4_hdr), 1); + task->cdev_id = get_cdev_id(); - return 0; -} + struct rte_cryptodev_config cdev_conf; + cdev_conf.nb_queue_pairs = 2; + cdev_conf.socket_id = rte_socket_id(); + rte_cryptodev_configure(task->cdev_id, &cdev_conf); -static inline uint8_t handle_esp_ah_dec_finish2(struct task_esp_dec *task, struct rte_mbuf *mbuf) -{ - u8* m = rte_pktmbuf_mtod(mbuf, u8*); - rte_memcpy(m+sizeof(prox_rte_ipv4_hdr)+sizeof(struct prox_esp_hdr)+CIPHER_IV_LENGTH_AES_CBC, - m, sizeof(prox_rte_ether_hdr)); - m = (u8*)rte_pktmbuf_adj(mbuf, sizeof(prox_rte_ipv4_hdr)+sizeof(struct prox_esp_hdr)+CIPHER_IV_LENGTH_AES_CBC); - prox_rte_ipv4_hdr* pip4 = (prox_rte_ipv4_hdr *)(m+sizeof(prox_rte_ether_hdr)); + session_size = rte_cryptodev_sym_get_private_session_size( + task->cdev_id); + plog_info("rte_cryptodev_sym_get_private_session_size=%d\n", + session_size); + sprintf(name, "core_%03u_session_pool", lcore_id); + task->session_pool = rte_cryptodev_sym_session_pool_create(name, + MAX_SESSIONS, + session_size, + POOL_CACHE_SIZE, + 0, rte_socket_id()); + PROX_PANIC(task->session_pool == NULL, "Failed rte_mempool_create\n"); - if (unlikely((pip4->version_ihl >> 4) != 4)) { - plog_info("non IPv4 packet after esp dec %i\n", pip4->version_ihl); - plogdx_info(mbuf, "DEC TX: "); - return OUT_DISCARD; - } - if (pip4->time_to_live) { - pip4->time_to_live--; - } - else { - plog_info("TTL = 0 => Dropping\n"); - return OUT_DISCARD; - } - uint16_t ipv4_length = rte_be_to_cpu_16(pip4->total_length); - int len = rte_pktmbuf_pkt_len(mbuf); - rte_pktmbuf_trim(mbuf, len - sizeof(prox_rte_ether_hdr) - ipv4_length); + task->qp_id=0; + plog_info("enc: task->qp_id=%u\n", task->qp_id); + struct prox_rte_cryptodev_qp_conf qp_conf; + qp_conf.nb_descriptors = 2048; + qp_conf.mp_session = task->session_pool; + prox_rte_cryptodev_queue_pair_setup(task->cdev_id, task->qp_id, + &qp_conf, rte_cryptodev_socket_id(task->cdev_id)); -#if 0 - do_ipv4_swap(task, mbuf); -#else - prox_rte_ether_hdr *peth = rte_pktmbuf_mtod(mbuf, prox_rte_ether_hdr *); - prox_rte_ether_addr_copy(&task->local_mac, &peth->s_addr); - prox_rte_ether_addr_copy(&task->dst_mac, &peth->d_addr); - //rte_memcpy(peth, task->dst_mac, sizeof(task->dst_mac)); -#endif + int ret = rte_cryptodev_start(task->cdev_id); + PROX_PANIC(ret < 0, "Failed to start device\n"); - prox_ip_cksum(mbuf, pip4, sizeof(prox_rte_ether_hdr), sizeof(prox_rte_ipv4_hdr), 1); - return 0; -} + //Setup Cipher Parameters + struct rte_crypto_sym_xform cipher_xform = {0}; + struct rte_crypto_sym_xform auth_xform = {0}; -static int handle_esp_enc_bulk(struct task_base *tbase, struct rte_mbuf **mbufs, uint16_t n_pkts) -{ - struct task_esp_enc *task = (struct task_esp_enc *)tbase; - uint8_t out[MAX_PKT_BURST]; - uint16_t i = 0, nb_rx = 0, nb_enc=0, j = 0; + cipher_xform.type = RTE_CRYPTO_SYM_XFORM_CIPHER; +// cipher_xform.next = &auth_xform; + cipher_xform.next = NULL; //CRYPTO_ONLY - for (uint16_t j = 0; j < n_pkts; ++j) { - out[j] = handle_esp_ah_enc(task, mbufs[j], task->ops_burst[nb_enc]); - if (out[j] != OUT_DISCARD) - ++nb_enc; - } + cipher_xform.cipher.algo = RTE_CRYPTO_CIPHER_AES_CBC; + cipher_xform.cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT; + cipher_xform.cipher.key.data = aes_cbc_key; + cipher_xform.cipher.key.length = CIPHER_KEY_LENGTH_AES_CBC; - if (rte_cryptodev_enqueue_burst(task->cdev_id, task->qp_id, task->ops_burst, nb_enc) != nb_enc) { - plog_info("Error enc enqueue_burst\n"); - return -1; - } + cipher_xform.cipher.iv.offset = IV_OFFSET; + cipher_xform.cipher.iv.length = CIPHER_IV_LENGTH_AES_CBC; + + //Setup HMAC Parameters + auth_xform.type = RTE_CRYPTO_SYM_XFORM_AUTH; + auth_xform.next = NULL; + auth_xform.auth.op = RTE_CRYPTO_AUTH_OP_GENERATE; + auth_xform.auth.algo = RTE_CRYPTO_AUTH_SHA1_HMAC; + auth_xform.auth.key.length = DIGEST_BYTE_LENGTH_SHA1; + auth_xform.auth.key.data = hmac_sha1_key; + auth_xform.auth.digest_length = DIGEST_BYTE_LENGTH_SHA1; + + auth_xform.auth.iv.offset = 0; + auth_xform.auth.iv.length = 0; - do { - nb_rx = rte_cryptodev_dequeue_burst(task->cdev_id, task->qp_id, task->ops_burst+i, nb_enc-i); - i += nb_rx; - } while (i < nb_enc); + task->sess = rte_cryptodev_sym_session_create(task->cdev_id, + &cipher_xform, task->session_pool); + PROX_PANIC(task->sess < 0, "Failed ENC sym_session_create\n"); - return task->base.tx_pkt(&task->base, mbufs, n_pkts, out); + task->local_ipv4 = rte_cpu_to_be_32(targ->local_ipv4); + task->remote_ipv4 = rte_cpu_to_be_32(targ->remote_ipv4); + //memcpy(&task->src_mac, &prox_port_cfg[task->base.tx_params_hw.tx_port_queue->port].eth_addr, sizeof(prox_rte_ether_addr)); + struct prox_port_cfg *port = find_reachable_port(targ); + memcpy(&task->local_mac, &port->eth_addr, sizeof(prox_rte_ether_addr)); + + if (targ->flags & TASK_ARG_DST_MAC_SET){ + memcpy(&task->dst_mac, &targ->edaddr, sizeof(task->dst_mac)); + plog_info("TASK_ARG_DST_MAC_SET ("MAC_BYTES_FMT")\n", + MAC_BYTES(task->dst_mac.addr_bytes)); + //prox_rte_ether_addr_copy(&ptask->dst_mac, &peth->d_addr); + //rte_memcpy(hdr, task->src_dst_mac, sizeof(task->src_dst_mac)); + } } -static int handle_esp_dec_bulk(struct task_base *tbase, struct rte_mbuf **mbufs, uint16_t n_pkts) +static void init_task_esp_dec(struct task_base *tbase, struct task_args *targ) { - struct task_esp_dec *task = (struct task_esp_dec *)tbase; - uint8_t out[MAX_PKT_BURST]; - uint16_t j, nb_dec=0, nb_rx=0; + struct task_esp *task = (struct task_esp *)tbase; + unsigned int session_size; - for (j = 0; j < n_pkts; ++j) { - out[j] = handle_esp_ah_dec(task, mbufs[j], task->ops_burst[nb_dec]); - if (out[j] != OUT_DISCARD) - ++nb_dec; - } + tbase->flags |= TBASE_FLAG_NEVER_FLUSH; + + uint8_t lcore_id = targ->lconf->id; + char name[64]; + task->handle_esp_finish = handle_dec_finish; + task->handle_esp_ah = handle_esp_ah_dec; + task->len = 0; + task->pkts_in_flight = 0; + sprintf(name, "core_%03u_crypto_pool", lcore_id); + task->crypto_op_pool = rte_crypto_op_pool_create(name, + RTE_CRYPTO_OP_TYPE_SYMMETRIC, targ->nb_mbuf, 128, + MAXIMUM_IV_LENGTH, rte_socket_id()); + PROX_PANIC(task->crypto_op_pool == NULL, "Can't create DEC \ + CRYPTO_OP_POOL\n"); + + task->cdev_id = get_cdev_id(); + struct rte_cryptodev_config cdev_conf; + cdev_conf.nb_queue_pairs = 2; + cdev_conf.socket_id = SOCKET_ID_ANY; + cdev_conf.socket_id = rte_socket_id(); + rte_cryptodev_configure(task->cdev_id, &cdev_conf); + + session_size = rte_cryptodev_sym_get_private_session_size( + task->cdev_id); + plog_info("rte_cryptodev_sym_get_private_session_size=%d\n", + session_size); + sprintf(name, "core_%03u_session_pool", lcore_id); + task->session_pool = rte_cryptodev_sym_session_pool_create(name, + MAX_SESSIONS, + session_size, + POOL_CACHE_SIZE, + 0, rte_socket_id()); + PROX_PANIC(task->session_pool == NULL, "Failed rte_mempool_create\n"); + + task->qp_id=0; + plog_info("dec: task->qp_id=%u\n", task->qp_id); + struct prox_rte_cryptodev_qp_conf qp_conf; + qp_conf.nb_descriptors = 2048; + qp_conf.mp_session = task->session_pool; + prox_rte_cryptodev_queue_pair_setup(task->cdev_id, task->qp_id, + &qp_conf, rte_cryptodev_socket_id(task->cdev_id)); + + int ret = rte_cryptodev_start(task->cdev_id); + PROX_PANIC(ret < 0, "Failed to start device\n"); + + //Setup Cipher Parameters + struct rte_crypto_sym_xform cipher_xform = {0}; + struct rte_crypto_sym_xform auth_xform = {0}; - if (rte_cryptodev_enqueue_burst(task->cdev_id, task->qp_id, task->ops_burst, nb_dec) != nb_dec) { - plog_info("Error dec enqueue_burst\n"); - return -1; + cipher_xform.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + cipher_xform.next = NULL; + cipher_xform.cipher.algo = RTE_CRYPTO_CIPHER_AES_CBC; + cipher_xform.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + cipher_xform.cipher.key.data = aes_cbc_key; + cipher_xform.cipher.key.length = CIPHER_KEY_LENGTH_AES_CBC; + + cipher_xform.cipher.iv.offset = IV_OFFSET; + cipher_xform.cipher.iv.length = CIPHER_IV_LENGTH_AES_CBC; + + //Setup HMAC Parameters + auth_xform.type = RTE_CRYPTO_SYM_XFORM_AUTH; + auth_xform.next = &cipher_xform; + auth_xform.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY; + auth_xform.auth.algo = RTE_CRYPTO_AUTH_SHA1_HMAC; + auth_xform.auth.key.length = DIGEST_BYTE_LENGTH_SHA1; + auth_xform.auth.key.data = hmac_sha1_key; + auth_xform.auth.digest_length = DIGEST_BYTE_LENGTH_SHA1; + + auth_xform.auth.iv.offset = 0; + auth_xform.auth.iv.length = 0; + + task->sess = rte_cryptodev_sym_session_create(task->cdev_id, &cipher_xform, + task->session_pool); + PROX_PANIC(task->sess < 0, "Failed DEC sym_session_create\n"); + + task->local_ipv4 = rte_cpu_to_be_32(targ->local_ipv4); + task->remote_ipv4 = rte_cpu_to_be_32(targ->remote_ipv4); + //memcpy(&task->src_mac, &prox_port_cfg[task->base.tx_params_hw.tx_port_queue->port].eth_addr, sizeof(prox_rte_ether_addr)); + struct prox_port_cfg *port = find_reachable_port(targ); + memcpy(&task->local_mac, &port->eth_addr, sizeof(prox_rte_ether_addr)); + + if (targ->flags & TASK_ARG_DST_MAC_SET){ + memcpy(&task->dst_mac, &targ->edaddr, sizeof(task->dst_mac)); + plog_info("TASK_ARG_DST_MAC_SET ("MAC_BYTES_FMT")\n", + MAC_BYTES(task->dst_mac.addr_bytes)); + //prox_rte_ether_addr_copy(&ptask->dst_mac, &peth->d_addr); + //rte_memcpy(hdr, task->src_dst_mac, sizeof(task->src_dst_mac)); } +} - j=0; - do { - nb_rx = rte_cryptodev_dequeue_burst(task->cdev_id, task->qp_id, - task->ops_burst+j, nb_dec-j); - j += nb_rx; - } while (j < nb_dec); - - for (j = 0; j < nb_dec; ++j) { - if (task->ops_burst[j]->status != RTE_CRYPTO_OP_STATUS_SUCCESS){ - plog_info("err: task->ops_burst[%d].status=%d\n", j, task->ops_burst[j]->status); - //!!!TODO!!! find mbuf and discard it!!! - //for now just send it further - //plogdx_info(mbufs[j], "RX: "); - } - if (task->ops_burst[j]->status == RTE_CRYPTO_OP_STATUS_SUCCESS) { - struct rte_mbuf *mbuf = task->ops_burst[j]->sym->m_src; - handle_esp_ah_dec_finish2(task, mbuf);//TODO set out[j] properly +static int crypto_send_burst(struct task_esp *task, uint16_t n) +{ + uint8_t out[MAX_PKT_BURST]; + struct rte_mbuf *mbufs[MAX_PKT_BURST]; + unsigned ret; + unsigned i = 0; + ret = rte_cryptodev_enqueue_burst(task->cdev_id, + task->qp_id, task->ops_burst, n); + task->pkts_in_flight += ret; + if (unlikely(ret < n)) { + for (i = 0; i < (n-ret); i++) { + mbufs[i] = task->ops_burst[ret + i]->sym->m_src; + out[i] = OUT_DISCARD; + rte_crypto_op_free(task->ops_burst[ret + i]); + } + return task->base.tx_pkt(&task->base, mbufs, i, out); + } + return 0; +} + +static int handle_esp_bulk(struct task_base *tbase, struct rte_mbuf **mbufs, + uint16_t n_pkts) +{ + struct task_esp *task = (struct task_esp *)tbase; + uint8_t out[MAX_PKT_BURST]; + uint8_t result = 0; + uint16_t nb_deq = 0, j, idx = 0; + struct rte_mbuf *drop_mbufs[MAX_PKT_BURST]; + struct rte_crypto_op *ops_burst[MAX_PKT_BURST]; + int nbr_tx_pkt = 0; + + if (likely(n_pkts != 0)) { + if (rte_crypto_op_bulk_alloc(task->crypto_op_pool, + RTE_CRYPTO_OP_TYPE_SYMMETRIC, + ops_burst, n_pkts) != n_pkts) { + plog_info("Failed to allocate crypto operations, discarding \ + %d packets\n", n_pkts); + for (j = 0; j < n_pkts; j++) { + out[j] = OUT_DISCARD; + } + nbr_tx_pkt += task->base.tx_pkt(&task->base, mbufs, n_pkts, + out); + } + else { + for (j = 0; j < n_pkts; j++) { + result = task->handle_esp_ah(task, mbufs[j], + ops_burst[j]); + if (result == 0) { + task->ops_burst[task->len] = ops_burst[j]; + task->len++; + /* enough ops to be sent */ + if (task->len == MAX_PKT_BURST) { + nbr_tx_pkt += crypto_send_burst(task, + (uint16_t) MAX_PKT_BURST); + task->len = 0; + } + } + else { + drop_mbufs[idx] = mbufs[j]; + out[idx] = result; + idx++; + rte_crypto_op_free(ops_burst[j]); + plog_info("Failed handle_esp_ah for 1 \ + packet\n"); + } + } + if (idx) nbr_tx_pkt += task->base.tx_pkt(&task->base, + drop_mbufs, idx, out); } + } else if (task->len) { + // No packets where received on the rx queue, but this handle + // function was called anyway since some packets where not yet + // enqueued. Hence they get enqueued here in order to minimize + // latency or in case no new packets will arrive + nbr_tx_pkt += crypto_send_burst(task, task->len); + task->len = 0; } - - return task->base.tx_pkt(&task->base, mbufs, n_pkts, out); + if (task->pkts_in_flight) { + do { + nb_deq = rte_cryptodev_dequeue_burst(task->cdev_id, + task->qp_id, ops_burst, MAX_PKT_BURST); + task->pkts_in_flight -= nb_deq; + for (j = 0; j < nb_deq; j++) { + mbufs[j] = ops_burst[j]->sym->m_src; + out[j] = task->handle_esp_finish(task, mbufs[j], + ops_burst[j]->status); + rte_crypto_op_free(ops_burst[j]); + } + nbr_tx_pkt += task->base.tx_pkt(&task->base, mbufs, nb_deq, + out); + } while (nb_deq == MAX_PKT_BURST); + } + return nbr_tx_pkt; } struct task_init task_init_esp_enc = { .mode = ESP_ENC, .mode_str = "esp_enc", .init = init_task_esp_enc, - .handle = handle_esp_enc_bulk, - .size = sizeof(struct task_esp_enc), + .handle = handle_esp_bulk, + .flag_features = TASK_FEATURE_ZERO_RX, + .size = sizeof(struct task_esp), }; struct task_init task_init_esp_dec = { - .mode = ESP_ENC, + .mode = ESP_DEC, .mode_str = "esp_dec", .init = init_task_esp_dec, - .handle = handle_esp_dec_bulk, - .size = sizeof(struct task_esp_dec), + .handle = handle_esp_bulk, + .flag_features = TASK_FEATURE_ZERO_RX, + .size = sizeof(struct task_esp), }; __attribute__((constructor)) static void reg_task_esp_enc(void) diff --git a/VNFs/DPPD-PROX/handle_gen.c b/VNFs/DPPD-PROX/handle_gen.c index f625e01c..2c8a65c7 100644 --- a/VNFs/DPPD-PROX/handle_gen.c +++ b/VNFs/DPPD-PROX/handle_gen.c @@ -13,6 +13,12 @@ // See the License for the specific language governing permissions and // limitations under the License. */ + +#include <rte_common.h> +#ifndef __rte_cache_aligned +#include <rte_memory.h> +#endif + #include <rte_mbuf.h> #include <pcap.h> #include <string.h> @@ -1775,7 +1781,7 @@ static void init_task_gen(struct task_base *tbase, struct task_args *targ) struct prox_port_cfg *port = find_reachable_port(targ); // TODO: check that all reachable ports have the same mtu... if (port) { - task->cksum_offload = port->requested_tx_offload & (DEV_TX_OFFLOAD_IPV4_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM); + task->cksum_offload = port->requested_tx_offload & (RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | RTE_ETH_TX_OFFLOAD_UDP_CKSUM); task->port = port; task->max_frame_size = port->mtu + PROX_RTE_ETHER_HDR_LEN + 2 * PROX_VLAN_TAG_SIZE; } else { @@ -1900,6 +1906,7 @@ static void init_task_gen(struct task_base *tbase, struct task_args *targ) .key_len = sizeof(union ipv4_5tuple_host), .hash_func = rte_hash_crc, .hash_func_init_val = 0, + .socket_id = task->socket_id, }; plog_info("\t\thash table name = %s\n", hash_params.name); task->flow_id_table = rte_hash_create(&hash_params); diff --git a/VNFs/DPPD-PROX/handle_ipv6_tunnel.c b/VNFs/DPPD-PROX/handle_ipv6_tunnel.c index a99a8f96..1c99eb84 100644 --- a/VNFs/DPPD-PROX/handle_ipv6_tunnel.c +++ b/VNFs/DPPD-PROX/handle_ipv6_tunnel.c @@ -167,7 +167,7 @@ static void init_task_ipv6_tun_base(struct task_ipv6_tun_base* tun_base, struct struct prox_port_cfg *port = find_reachable_port(targ); if (port) { - tun_base->offload_crc = port->requested_tx_offload & (DEV_TX_OFFLOAD_IPV4_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM); + tun_base->offload_crc = port->requested_tx_offload & (RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | RTE_ETH_TX_OFFLOAD_UDP_CKSUM); } } diff --git a/VNFs/DPPD-PROX/handle_irq.c b/VNFs/DPPD-PROX/handle_irq.c index 00c192f6..36aa54e8 100644 --- a/VNFs/DPPD-PROX/handle_irq.c +++ b/VNFs/DPPD-PROX/handle_irq.c @@ -26,7 +26,10 @@ #include "input.h" #define MAX_INTERRUPT_LENGTH 500000 /* Maximum length of an interrupt is (1 / MAX_INTERRUPT_LENGTH) seconds */ + +uint64_t irq_bucket_maxtime_cycles[IRQ_BUCKETS_COUNT]; uint64_t irq_bucket_maxtime_micro[] = {1,5,10,50,100,500,1000,5000,10000,50000,100000,500000,UINT64_MAX}; + /* * This module is not handling any packets. * It loops on rdtsc() and checks whether it has been interrupted diff --git a/VNFs/DPPD-PROX/handle_lb_5tuple.c b/VNFs/DPPD-PROX/handle_lb_5tuple.c index d320ca9d..ec229386 100644 --- a/VNFs/DPPD-PROX/handle_lb_5tuple.c +++ b/VNFs/DPPD-PROX/handle_lb_5tuple.c @@ -14,6 +14,11 @@ // limitations under the License. */ +#include <rte_common.h> +#ifndef __rte_cache_aligned +#include <rte_memory.h> +#endif + #include <rte_hash.h> #include <rte_ether.h> #include <rte_memcpy.h> diff --git a/VNFs/DPPD-PROX/handle_lb_qinq.c b/VNFs/DPPD-PROX/handle_lb_qinq.c index a2f1b1fa..9726edda 100644 --- a/VNFs/DPPD-PROX/handle_lb_qinq.c +++ b/VNFs/DPPD-PROX/handle_lb_qinq.c @@ -104,9 +104,9 @@ static void init_task_lb_qinq(struct task_base *tbase, struct task_args *targ) plog_info("\t\ttask_lb_qinq protocols_mask = 0x%x\n", task->protocols_mask); if (targ->task_init->flag_features & TASK_FEATURE_LUT_QINQ_RSS) - tbase->flags |= BASE_FLAG_LUT_QINQ_RSS; + tbase->flags |= TBASE_FLAG_LUT_QINQ_RSS; if (targ->task_init->flag_features & TASK_FEATURE_LUT_QINQ_HASH) - tbase->flags |= BASE_FLAG_LUT_QINQ_HASH; + tbase->flags |= TBASE_FLAG_LUT_QINQ_HASH; plog_info("\t\ttask_lb_qinq flags = 0x%x\n", tbase->flags); } @@ -275,7 +275,7 @@ struct cpe_packet { static inline uint8_t get_worker(struct task_lb_qinq *task, struct cpe_packet *packet) { uint8_t worker = 0; - if (((struct task_base *)task)->flags & BASE_FLAG_LUT_QINQ_HASH) { + if (((struct task_base *)task)->flags & TBASE_FLAG_LUT_QINQ_HASH) { // Load Balance on Hash of combination of cvlan and svlan uint64_t qinq_net = packet->qd.qinq; qinq_net = qinq_net & 0xFF0F0000FF0F0000; // Mask Proto and QoS bits @@ -286,7 +286,7 @@ static inline uint8_t get_worker(struct task_lb_qinq *task, struct cpe_packet *p worker = rte_hash_crc(&qinq_net,8,0) % task->nb_worker_threads; } plogx_dbg("Sending packet svlan=%x, cvlan=%x, pseudo_qinq=%lx to worker %d\n", rte_bswap16(0xFF0F & packet->qp.qinq_hdr.svlan.vlan_tci), rte_bswap16(0xFF0F & packet->qp.qinq_hdr.cvlan.vlan_tci), qinq_net, worker); - } else if (((struct task_base *)task)->flags & BASE_FLAG_LUT_QINQ_RSS){ + } else if (((struct task_base *)task)->flags & TBASE_FLAG_LUT_QINQ_RSS){ // Load Balance on rss of combination of cvlan and svlan uint32_t qinq = (packet->qp.qinq_hdr.cvlan.vlan_tci & 0xFF0F) << 16; uint32_t rss = toeplitz_hash((uint8_t *)&qinq, 4); diff --git a/VNFs/DPPD-PROX/handle_master.c b/VNFs/DPPD-PROX/handle_master.c index 169c7ca0..58240ba0 100644 --- a/VNFs/DPPD-PROX/handle_master.c +++ b/VNFs/DPPD-PROX/handle_master.c @@ -73,6 +73,8 @@ const char *actions_string[] = { }; +int (*handle_ctrl_plane)(struct task_base *tbase, struct rte_mbuf **mbuf, uint16_t n_pkts) = NULL; + static struct my_arp_t arp_reply = { .htype = 0x100, .ptype = 8, @@ -259,7 +261,7 @@ static inline void handle_arp_request(struct task_base *tbase, struct rte_mbuf * plogx_dbg("\tMaster handling ARP request for ip "IPv4_BYTES_FMT" on port %d which supports random ip\n", IP4(key.ip), key.port); struct rte_ring *ring = task->internal_port_table[port].ring; create_mac(arp, &mac); - mbuf->ol_flags &= ~(PKT_TX_IP_CKSUM|PKT_TX_UDP_CKSUM); + mbuf->ol_flags &= ~(RTE_MBUF_F_TX_IP_CKSUM|RTE_MBUF_F_TX_UDP_CKSUM); build_arp_reply(ether_hdr, &mac, arp); tx_ring(tbase, ring, SEND_ARP_REPLY_FROM_MASTER, mbuf); return; @@ -274,7 +276,7 @@ static inline void handle_arp_request(struct task_base *tbase, struct rte_mbuf * tx_drop(mbuf); } else { struct rte_ring *ring = task->internal_ip_table[ret].ring; - mbuf->ol_flags &= ~(PKT_TX_IP_CKSUM|PKT_TX_UDP_CKSUM); + mbuf->ol_flags &= ~(RTE_MBUF_F_TX_IP_CKSUM|RTE_MBUF_F_TX_UDP_CKSUM); build_arp_reply(ether_hdr, &task->internal_ip_table[ret].mac, arp); tx_ring(tbase, ring, SEND_ARP_REPLY_FROM_MASTER, mbuf); } @@ -339,7 +341,7 @@ static inline void handle_unknown_ip(struct task_base *tbase, struct rte_mbuf *m return; } // We send an ARP request even if one was just sent (and not yet answered) by another task - mbuf->ol_flags &= ~(PKT_TX_IP_CKSUM|PKT_TX_UDP_CKSUM); + mbuf->ol_flags &= ~(RTE_MBUF_F_TX_IP_CKSUM|RTE_MBUF_F_TX_UDP_CKSUM); build_arp_request(mbuf, &task->internal_port_table[port].mac, ip_dst, ip_src, vlan); tx_ring(tbase, ring, SEND_ARP_REQUEST_FROM_MASTER, mbuf); } @@ -368,7 +370,7 @@ static inline void build_icmp_reply_message(struct task_base *tbase, struct rte_ tx_drop(mbuf); } else { struct rte_ring *ring = task->internal_ip_table[ret].ring; - mbuf->ol_flags &= ~(PKT_TX_IP_CKSUM|PKT_TX_UDP_CKSUM); + mbuf->ol_flags &= ~(RTE_MBUF_F_TX_IP_CKSUM|RTE_MBUF_F_TX_UDP_CKSUM); tx_ring(tbase, ring, SEND_ICMP_FROM_MASTER, mbuf); } } @@ -907,6 +909,7 @@ void init_ctrl_plane(struct task_base *tbase) .entries = n_entries, .hash_func = rte_hash_crc, .hash_func_init_val = 0, + .socket_id = socket_id, }; if (prox_cfg.flags & DSF_L3_ENABLED) { hash_params.key_len = sizeof(uint32_t); diff --git a/VNFs/DPPD-PROX/handle_master.h b/VNFs/DPPD-PROX/handle_master.h index 518906ed..dcd0a5f2 100644 --- a/VNFs/DPPD-PROX/handle_master.h +++ b/VNFs/DPPD-PROX/handle_master.h @@ -99,11 +99,11 @@ struct task_master { struct pollfd route_fds; }; -const char *actions_string[MAX_ACTIONS]; +extern const char *actions_string[MAX_ACTIONS]; void init_ctrl_plane(struct task_base *tbase); -int (*handle_ctrl_plane)(struct task_base *tbase, struct rte_mbuf **mbuf, uint16_t n_pkts); +extern int (*handle_ctrl_plane)(struct task_base *tbase, struct rte_mbuf **mbuf, uint16_t n_pkts); static inline void tx_drop(struct rte_mbuf *mbuf) { diff --git a/VNFs/DPPD-PROX/handle_nat.c b/VNFs/DPPD-PROX/handle_nat.c index 83c9ebc5..ad0fcf45 100644 --- a/VNFs/DPPD-PROX/handle_nat.c +++ b/VNFs/DPPD-PROX/handle_nat.c @@ -123,6 +123,7 @@ static int lua_to_hash_nat(struct lua_State *L, enum lua_place from, const char .key_len = sizeof(ip_from), .hash_func = rte_hash_crc, .hash_func_init_val = 0, + .socket_id = socket, }; ret_hash = rte_hash_create(&hash_params); @@ -171,7 +172,7 @@ static void init_task_nat(struct task_base *tbase, struct task_args *targ) PROX_PANIC(ret != 0, "Failed to load NAT table from lua:\n%s\n", get_lua_to_errors()); struct prox_port_cfg *port = find_reachable_port(targ); if (port) { - task->offload_crc = port->requested_tx_offload & (DEV_TX_OFFLOAD_IPV4_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM); + task->offload_crc = port->requested_tx_offload & (RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | RTE_ETH_TX_OFFLOAD_UDP_CKSUM); } } diff --git a/VNFs/DPPD-PROX/handle_qinq_decap4.c b/VNFs/DPPD-PROX/handle_qinq_decap4.c index d74e622e..2a5bfc7f 100644 --- a/VNFs/DPPD-PROX/handle_qinq_decap4.c +++ b/VNFs/DPPD-PROX/handle_qinq_decap4.c @@ -148,7 +148,7 @@ static void init_task_qinq_decap4(struct task_base *tbase, struct task_args *tar struct prox_port_cfg *port = find_reachable_port(targ); if (port) { - task->offload_crc = port->requested_tx_offload & (DEV_TX_OFFLOAD_IPV4_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM); + task->offload_crc = port->requested_tx_offload & (RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | RTE_ETH_TX_OFFLOAD_UDP_CKSUM); } // By default, calling this function 1K times per second => 64K ARP per second max diff --git a/VNFs/DPPD-PROX/handle_qinq_encap4.c b/VNFs/DPPD-PROX/handle_qinq_encap4.c index ffd9356a..0b707b7a 100644 --- a/VNFs/DPPD-PROX/handle_qinq_encap4.c +++ b/VNFs/DPPD-PROX/handle_qinq_encap4.c @@ -152,7 +152,7 @@ static void init_task_qinq_encap4(struct task_base *tbase, struct task_args *tar struct prox_port_cfg *port = find_reachable_port(targ); if (port) { - task->offload_crc = port->requested_tx_offload & (DEV_TX_OFFLOAD_IPV4_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM); + task->offload_crc = port->requested_tx_offload & (RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | RTE_ETH_TX_OFFLOAD_UDP_CKSUM); } /* TODO: check if it is not necessary to limit reverse mapping diff --git a/VNFs/DPPD-PROX/handle_swap.c b/VNFs/DPPD-PROX/handle_swap.c index b7a82d03..503af598 100644 --- a/VNFs/DPPD-PROX/handle_swap.c +++ b/VNFs/DPPD-PROX/handle_swap.c @@ -572,7 +572,7 @@ static void init_task_swap(struct task_base *tbase, struct task_args *targ) struct prox_port_cfg *port = find_reachable_port(targ); if (port) { - task->offload_crc = port->requested_tx_offload & (DEV_TX_OFFLOAD_IPV4_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM); + task->offload_crc = port->requested_tx_offload & (RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | RTE_ETH_TX_OFFLOAD_UDP_CKSUM); } task->store_pkt_id = 0; if (targ->store_max) { diff --git a/VNFs/DPPD-PROX/hash_utils.c b/VNFs/DPPD-PROX/hash_utils.c index ad746d5c..3922ef0f 100644 --- a/VNFs/DPPD-PROX/hash_utils.c +++ b/VNFs/DPPD-PROX/hash_utils.c @@ -14,6 +14,11 @@ // limitations under the License. */ +#include <rte_common.h> +#ifndef __rte_cache_aligned +#include <rte_memory.h> +#endif + #include <string.h> #include <rte_hash_crc.h> #include <rte_table_hash.h> diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/Dockerfile b/VNFs/DPPD-PROX/helper-scripts/rapid/Dockerfile index 2c25f097..fef0fcaf 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/Dockerfile +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/Dockerfile @@ -17,42 +17,101 @@ ################################################## # Build all components in separate builder image # ################################################## -FROM centos:7 as builder -ARG BUILD_DIR=/opt/rapid +FROM ubuntu:20.04 as builder -COPY ./port_info ${BUILD_DIR}/port_info +ARG DPDK_VERSION=22.07 +ENV DPDK_VERSION=${DPDK_VERSION} -COPY ./deploycentostools.sh ${BUILD_DIR}/ -RUN chmod +x ${BUILD_DIR}/deploycentostools.sh \ - && ${BUILD_DIR}/deploycentostools.sh -k deploy +ARG BUILD_DIR="/opt/rapid" +ENV BUILD_DIR=${BUILD_DIR} + +ENV DEBIAN_FRONTEND=noninteractive + +# Install Dependencies +RUN apt update && apt -y install git wget gcc unzip libpcap-dev libncurses5-dev \ + libedit-dev liblua5.3-dev linux-headers-generic iperf3 pciutils \ + libnuma-dev vim tuna wireshark make driverctl openssh-server sudo \ + meson python3-pyelftools pkg-config + +WORKDIR ${BUILD_DIR} + +# Install DPDK +RUN wget http://fast.dpdk.org/rel/dpdk-${DPDK_VERSION}.tar.xz \ + && tar -xf ./dpdk-${DPDK_VERSION}.tar.xz \ + && cd dpdk-${DPDK_VERSION} \ + && meson build -Dlibdir=lib/x86_64-linux-gnu -Denable_driver_sdk=true \ + && ninja -C build install + +WORKDIR ${BUILD_DIR} + +# Install Prox +RUN git clone https://gerrit.opnfv.org/gerrit/samplevnf \ + && cd samplevnf/VNFs/DPPD-PROX \ + && COMMIT_ID=$(git rev-parse HEAD) \ + && echo "${COMMIT_ID}" > ${BUILD_DIR}/commit_id \ + && meson build \ + && ninja -C build \ + && cp ${BUILD_DIR}/samplevnf/VNFs/DPPD-PROX/build/prox ${BUILD_DIR}/prox + +# Build and copy port info app +WORKDIR ${BUILD_DIR}/samplevnf/VNFs/DPPD-PROX/helper-scripts/rapid/port_info +RUN meson build \ + && ninja -C build \ + && cp ${BUILD_DIR}/samplevnf/VNFs/DPPD-PROX/helper-scripts/rapid/port_info/build/port_info_app ${BUILD_DIR}/port_info_app + +RUN ldconfig && pkg-config --modversion libdpdk > ${BUILD_DIR}/dpdk_version +# Create Minimal Install +RUN ldd ${BUILD_DIR}/prox | awk '$2 ~ /=>/ {print $3}' >> ${BUILD_DIR}/list_of_install_components \ + && echo "${BUILD_DIR}/prox" >> ${BUILD_DIR}/list_of_install_components \ + && echo "${BUILD_DIR}/port_info_app" >> ${BUILD_DIR}/list_of_install_components \ + && echo "${BUILD_DIR}/commit_id" >> ${BUILD_DIR}/list_of_install_components \ + && echo "${BUILD_DIR}/dpdk_version" >> ${BUILD_DIR}/list_of_install_components \ + && find /usr/local/lib/x86_64-linux-gnu -not -path '*/\.*' >> ${BUILD_DIR}/list_of_install_components \ + && tar -czvhf ${BUILD_DIR}/install_components.tgz -T ${BUILD_DIR}/list_of_install_components ############################# # Create slim runtime image # ############################# -FROM centos:7 +FROM ubuntu:20.04 + +ARG BUILD_DIR="/opt/rapid" +ENV BUILD_DIR=${BUILD_DIR} -ARG BUILD_DIR=/opt/rapid +ENV DEBIAN_FRONTEND=noninteractive + +# Install Runtime Dependencies +RUN apt update -y +# Install required dynamically linked libraries + required packages +RUN apt -y install sudo openssh-server libatomic1 -COPY ./deploycentostools.sh ${BUILD_DIR}/ COPY --from=builder ${BUILD_DIR}/install_components.tgz ${BUILD_DIR}/install_components.tgz -COPY --from=builder ${BUILD_DIR}/src ${BUILD_DIR}/src -RUN chmod a+rwx ${BUILD_DIR} && chmod +x ${BUILD_DIR}/deploycentostools.sh \ - && ${BUILD_DIR}/deploycentostools.sh -k runtime_image +WORKDIR / +RUN tar -xvf ${BUILD_DIR}/install_components.tgz --skip-old-files +RUN ldconfig +RUN rm ${BUILD_DIR}/install_components.tgz # Expose SSH and PROX ports EXPOSE 22 8474 +RUN useradd -rm -d /home/rapid -s /bin/bash -g root -G sudo -u 1000 rapid \ + && chmod 777 ${BUILD_DIR} \ + && echo 'rapid:rapid' | chpasswd \ + && mkdir /home/rapid/.ssh + # Copy SSH keys -COPY ./rapid_rsa_key.pub /home/centos/.ssh/authorized_keys +COPY ./rapid_rsa_key.pub /home/rapid/.ssh/authorized_keys COPY ./rapid_rsa_key.pub /root/.ssh/authorized_keys -RUN chown centos:centos /home/centos/.ssh/authorized_keys \ - && chmod 600 /home/centos/.ssh/authorized_keys \ +RUN chown rapid:root /home/rapid/.ssh/authorized_keys \ + && chmod 600 /home/rapid/.ssh/authorized_keys \ && chown root:root /root/.ssh/authorized_keys \ && chmod 600 /root/.ssh/authorized_keys +#RUN apt-get clean && apt autoremove --purge +RUN apt-get autoremove -y && apt-get clean all && rm -rf /var/cache/apt + # Copy startup script COPY ./start.sh /start.sh RUN chmod +x /start.sh diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/configs/esp.cfg b/VNFs/DPPD-PROX/helper-scripts/rapid/configs/esp.cfg new file mode 100644 index 00000000..31728daf --- /dev/null +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/configs/esp.cfg @@ -0,0 +1,47 @@ +[lua] +dofile("parameters.lua") + +[eal options] +-n=4 ; force number of memory channels +no-output=no ; disable DPDK debug output +eal=--proc-type auto ${eal} + +[port 0] +name=if0 +mac=hardware +rx desc=2048 +tx desc=2048 +vlan=yes +vdev=esp_tap +local ipv4=$local_ip1 + +[defaults] +mempool size=64K + +[global] +name=${name} + +[core $mcore] +mode=master + +[core $cores] +name=enc +task=0 +mode=esp_enc +sub mode=l3 +remote ipv4=$dest_ip1 +rx port=if0 +tx cores=$altcores task=0 +drop=yes + + +[core $altcores] +name=dec +task=0 +mode=esp_dec +sub mode=l3 +remote ipv4=$dest_ip1 +rx ring=yes +tx port=if0 +drop=yes + diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/helper.lua b/VNFs/DPPD-PROX/helper-scripts/rapid/helper.lua index 1b4c657b..a5633409 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/helper.lua +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/helper.lua @@ -21,7 +21,7 @@ function convertIPToHex(ip) return "IP ADDRESS ERROR" end - local chunks = {ip:match("^(%d+)%.(%d+)%.(%d+)%.(%d+)(\/%d+)$")} + local chunks = {ip:match("^(%d+)%.(%d+)%.(%d+)%.(%d+)(/%d+)$")} if #chunks == 5 then for i,v in ipairs(chunks) do if i < 5 then diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/port_info/meson.build b/VNFs/DPPD-PROX/helper-scripts/rapid/port_info/meson.build new file mode 100644 index 00000000..f2efd667 --- /dev/null +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/port_info/meson.build @@ -0,0 +1,101 @@ +## +## +## Licensed under the Apache License, Version 2.0 (the "License"); +## you may not use this file except in compliance with the License. +## You may obtain a copy of the License at +## +## http://www.apache.org/licenses/LICENSE-2.0 +## +## Unless required by applicable law or agreed to in writing, software +## distributed under the License is distributed on an "AS IS" BASIS, +## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +## See the License for the specific language governing permissions and +## limitations under the License. +## + +project('port-info', 'C', + version: + run_command(['git', 'describe', + '--abbrev=8', '--dirty', '--always']).stdout().strip(), + license: 'Apache', + default_options: ['buildtype=release', 'c_std=gnu99'], + meson_version: '>= 0.47' +) + +cc = meson.get_compiler('c') + +# Configure options for prox +# Grab the DPDK version here "manually" as it is not available in the dpdk_dep +# object +dpdk_version = run_command('pkg-config', '--modversion', 'libdpdk').stdout() + + +cflags = [ + '-DPROGRAM_NAME="port_info_app"', + '-fno-stack-protector', + '-DGRE_TP', + '-D_GNU_SOURCE'] # for PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP + +# Add configured cflags to arguments +foreach arg: cflags + add_project_arguments(arg, language: 'c') +endforeach + +# enable warning flags if they are supported by the compiler +warning_flags = [ + '-Wno-unused', + '-Wno-unused-parameter', + '-Wno-unused-result', + '-Wno-deprecated-declarations'] + +foreach arg: warning_flags + if cc.has_argument(arg) + add_project_arguments(arg, language: 'c') + endif +endforeach + +has_sym_args = [ + [ 'HAVE_LIBEDIT_EL_RFUNC_T', 'histedit.h', + 'el_rfunc_t' ], +] +config = configuration_data() +foreach arg:has_sym_args + config.set(arg[0], cc.has_header_symbol(arg[1], arg[2])) +endforeach +configure_file(output : 'libedit_autoconf.h', configuration : config) + +# All other dependencies +dpdk_dep = dependency('libdpdk', required: true) +tinfo_dep = dependency('tinfo', required: false) +threads_dep = dependency('threads', required: true) +pcap_dep = dependency('pcap', required: true) +libedit_dep = dependency('libedit', required: true) +math_dep = cc.find_library('m', required : false) +dl_dep = cc.find_library('dl', required : true) + +deps = [dpdk_dep, + tinfo_dep, + threads_dep, + pcap_dep, + libedit_dep, + math_dep, + dl_dep] + +# Explicitly add these to the dependency list +deps += [cc.find_library('rte_bus_pci', required: true)] +deps += [cc.find_library('rte_bus_vdev', required: true)] + +if dpdk_version.version_compare('<20.11.0') +deps += [cc.find_library('rte_pmd_ring', required: true)] +else +deps += [cc.find_library('rte_net_ring', required: true)] +endif + +sources = files( + 'port_info.c') + +executable('port_info_app', + sources, + c_args: cflags, + dependencies: deps, + install: true) diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/port_info/port_info.c b/VNFs/DPPD-PROX/helper-scripts/rapid/port_info/port_info.c index 79bd0c0b..917c0636 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/port_info/port_info.c +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/port_info/port_info.c @@ -21,7 +21,11 @@ #include <rte_version.h> static const uint16_t rx_rings = 1, tx_rings = 1; +#if RTE_VERSION < RTE_VERSION_NUM(21,11,0,0) static const struct rte_eth_conf port_conf = { .link_speeds = ETH_LINK_SPEED_AUTONEG }; +#else +static const struct rte_eth_conf port_conf = { .link_speeds = RTE_ETH_LINK_SPEED_AUTONEG }; +#endif static inline int port_info(void) diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/prox_ctrl.py b/VNFs/DPPD-PROX/helper-scripts/rapid/prox_ctrl.py index c1aade6b..8754ebc4 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/prox_ctrl.py +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/prox_ctrl.py @@ -47,8 +47,10 @@ class prox_ctrl(object): on %s, attempt: %d" % (self._ip, attempts)) while True: try: - self.run_cmd('test -e /opt/rapid/system_ready_for_rapid') - break + if (self.run_cmd('test -e /opt/rapid/system_ready_for_rapid \ + && echo exists')): + break + time.sleep(2) except RuntimeWarning as ex: RapidLog.debug("RuntimeWarning %d:\n%s" % (ex.returncode, ex.output.strip())) @@ -103,9 +105,12 @@ class prox_ctrl(object): def scp_put(self, src, dst): self._sshclient.scp_put(src, dst) + RapidLog.info("Copying from {} to {}:{}".format(src, self._ip, dst)) def scp_get(self, src, dst): self._sshclient.scp_get('/home/' + self._user + src, dst) + RapidLog.info("Copying from {}:/home/{}{} to {}".format(self._ip, + self._user, src, dst)) class prox_sock(object): def __init__(self, sock): diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_defaults.py b/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_defaults.py index b56fbe1d..27d2430d 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_defaults.py +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_defaults.py @@ -21,7 +21,7 @@ class RapidDefaults(object): Class to define the test defaults """ test_params = { - 'version' : '2021.03.15', # Please do NOT change, used for debugging + 'version' : '2023.01.16', # Please do NOT change, used for debugging 'environment_file' : 'rapid.env', #Default string for environment 'test_file' : 'tests/basicrapid.test', #Default string for test 'machine_map_file' : 'machine.map', #Default string for machine map file diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_flowsizetest.py b/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_flowsizetest.py index 500be5b4..ea42fc9a 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_flowsizetest.py +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_flowsizetest.py @@ -226,7 +226,14 @@ class FlowSizeTest(RapidTest): # the drop rate is below a treshold, either we want that no # packet has been lost during the test. # This can be specified by putting 0 in the .test file - elif ((iteration_data['drop_rate'] < self.test['drop_rate_threshold']) or (iteration_data['abs_dropped']==self.test['drop_rate_threshold']==0)) and (iteration_data['lat_avg']< self.test['lat_avg_threshold']) and (iteration_data['lat_perc']< self.test['lat_perc_threshold']) and (iteration_data['lat_max'] < self.test['lat_max_threshold'] and iteration_data['mis_ordered'] <= self.test['mis_ordered_threshold']): + elif ((self.get_pps(speed,size) - iteration_data['pps_tx']) / self.get_pps(speed,size)) \ + < self.test['generator_threshold'] and \ + ((iteration_data['drop_rate'] < self.test['drop_rate_threshold']) or \ + (iteration_data['abs_dropped']==self.test['drop_rate_threshold']==0)) and \ + (iteration_data['lat_avg']< self.test['lat_avg_threshold']) and \ + (iteration_data['lat_perc']< self.test['lat_perc_threshold']) and \ + (iteration_data['lat_max'] < self.test['lat_max_threshold'] and \ + iteration_data['mis_ordered'] <= self.test['mis_ordered_threshold']): end_data = copy.deepcopy(iteration_data) end_prefix = copy.deepcopy(iteration_prefix) success = True diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_irqtest.py b/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_irqtest.py index b0516eeb..de7e6ae3 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_irqtest.py +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_irqtest.py @@ -45,7 +45,7 @@ class IrqTest(RapidTest): max_loop_duration = 0 machine_details = {} for machine in self.machines: - buckets=machine.socket.show_irq_buckets(1) + buckets=machine.socket.show_irq_buckets(machine.get_cores()[0]) if max_loop_duration == 0: # First time we go through the loop, we need to initialize # result_details diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_k8s_deployment.py b/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_k8s_deployment.py index bfb81611..1d1112f7 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_k8s_deployment.py +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_k8s_deployment.py @@ -32,7 +32,7 @@ class K8sDeployment: """ LOG_FILE_NAME = "createrapidk8s.log" SSH_PRIVATE_KEY = "./rapid_rsa_key" - SSH_USER = "centos" + SSH_USER = "rapid" POD_YAML_TEMPLATE_FILE_NAME = "pod-rapid.yaml" @@ -111,7 +111,7 @@ class K8sDeployment: pod_name = self._create_config.get( "POD%d" % i, "name") else: - pod_name = "pod-rapid-%d" % i + pod_name = "prox-pod-%d" % i # Search for POD hostname if self._create_config.has_option("POD%d" % i, @@ -121,6 +121,14 @@ class K8sDeployment: else: pod_nodeselector_hostname = None + # Search for POD spec + if self._create_config.has_option("POD%d" % i, + "spec_file_name"): + pod_spec_file_name = self._create_config.get( + "POD%d" % i, "spec_file_name") + else: + pod_spec_file_name = K8sDeployment.POD_YAML_TEMPLATE_FILE_NAME + # Search for POD dataplane static IP if self._create_config.has_option("POD%d" % i, "dp_ip"): @@ -139,6 +147,7 @@ class K8sDeployment: pod = Pod(pod_name, self._namespace) pod.set_nodeselector(pod_nodeselector_hostname) + pod.set_spec_file_name(pod_spec_file_name) pod.set_dp_ip(pod_dp_ip) pod.set_dp_subnet(pod_dp_subnet) pod.set_id(i) @@ -157,7 +166,7 @@ class K8sDeployment: # Create PODs using template from yaml file for pod in self._pods: self._log.info("Creating POD %s...", pod.get_name()) - pod.create_from_yaml(K8sDeployment.POD_YAML_TEMPLATE_FILE_NAME) + pod.create_from_yaml() # Wait for PODs to start for pod in self._pods: @@ -167,6 +176,7 @@ class K8sDeployment: for pod in self._pods: pod.set_ssh_credentials(K8sDeployment.SSH_USER, K8sDeployment.SSH_PRIVATE_KEY) pod.get_sriov_dev_mac() + pod.get_qat_dev() def save_runtime_config(self, config_file_name): self._log.info("Saving config %s for runrapid script...", @@ -203,6 +213,10 @@ class K8sDeployment: "dp_mac1", pod.get_dp_mac()) self._runtime_config.set("M%d" % pod.get_id(), "dp_pci_dev", pod.get_dp_pci_dev()) + if (pod.get_qat_pci_dev()): + for qat_index, qat_device in enumerate(pod.get_qat_pci_dev()): + self._runtime_config.set("M%d" % pod.get_id(), + "qat_pci_dev%d" % qat_index, qat_device) self._runtime_config.set("M%d" % pod.get_id(), "dp_ip1", pod.get_dp_ip() + "/" + pod.get_dp_subnet()) diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_k8s_pod.py b/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_k8s_pod.py index d15fe7ff..beaedd69 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_k8s_pod.py +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_k8s_pod.py @@ -32,6 +32,7 @@ class Pod: _name = "pod" _namespace = "default" _nodeSelector_hostname = None + _spec_filename = None _last_status = None _id = None _admin_ip = None @@ -49,6 +50,7 @@ class Pod: self._name = name self._namespace = namespace self._ssh_client = SSHClient(logger_name = logger_name) + self.qat_vf = [] def __del__(self): """Destroy POD. Do a cleanup. @@ -56,10 +58,11 @@ class Pod: if self._ssh_client is not None: self._ssh_client.disconnect() - def create_from_yaml(self, file_name): + def create_from_yaml(self): """Load POD description from yaml file. """ - with open(path.join(path.dirname(__file__), file_name)) as yaml_file: + with open(path.join(path.dirname(__file__), + self._spec_filename)) as yaml_file: self.body = yaml.safe_load(yaml_file) self.body["metadata"]["name"] = self._name @@ -67,14 +70,16 @@ class Pod: if (self._nodeSelector_hostname is not None): if ("nodeSelector" not in self.body["spec"]): self.body["spec"]["nodeSelector"] = {} - self.body["spec"]["nodeSelector"]["kubernetes.io/hostname"] = self._nodeSelector_hostname + self.body["spec"]["nodeSelector"]["kubernetes.io/hostname"] = \ + self._nodeSelector_hostname self._log.debug("Creating POD, body:\n%s" % self.body) try: self.k8s_CoreV1Api.create_namespaced_pod(body = self.body, namespace = self._namespace) except client.rest.ApiException as e: - self._log.error("Couldn't create POD %s!\n%s\n" % (self._name, e)) + self._log.error("Couldn't create POD %s!\n%s\n" % (self._name, + e)) def terminate(self): """Terminate POD. Close SSH connection. @@ -138,6 +143,9 @@ class Pod: def get_dp_pci_dev(self): return self._sriov_vf + def get_qat_pci_dev(self): + return self.qat_vf + def get_id(self): return self._id @@ -153,6 +161,28 @@ class Pod: self._last_status = pod.status.phase return self._last_status + def get_qat_dev(self): + """Get qat devices if any, assigned by k8s QAT device plugin. + """ + self._log.info("Checking assigned QAT VF for POD %s" % self._name) + ret = self._ssh_client.run_cmd("cat /opt/rapid/k8s_qat_device_plugin_envs") + if ret != 0: + self._log.error("Failed to check assigned QAT VF!" + "Error %s" % self._ssh_client.get_error()) + return -1 + + cmd_output = self._ssh_client.get_output().decode("utf-8").rstrip() + + if cmd_output: + self._log.debug("Before: Using QAT VF %s" % self.qat_vf) + self._log.debug("Environment variable %s" % cmd_output) + for line in cmd_output.splitlines(): + self.qat_vf.append(line.split("=")[1]) + self._log.debug("Using QAT VF %s" % self.qat_vf) + else: + self._log.debug("No QAT devices for this pod") + self.qat_vf = None + def get_sriov_dev_mac(self): """Get assigned by k8s SRIOV network device plugin SRIOV VF devices. Return 0 in case of sucessfull configuration. @@ -173,8 +203,24 @@ class Pod: self._sriov_vf = cmd_output.split(",")[0] self._log.debug("Using first SRIOV VF %s" % self._sriov_vf) - self._log.info("Getting MAC address for assigned SRIOV VF %s" % self._sriov_vf) - self._ssh_client.run_cmd("sudo /opt/rapid/port_info_app -n 4 -w %s" % self._sriov_vf) + # find DPDK version + self._log.info("Checking DPDK version for POD %s" % self._name) + ret = self._ssh_client.run_cmd("cat /opt/rapid/dpdk_version") + if ret != 0: + self._log.error("Failed to check DPDK version" + "Error %s" % self._ssh_client.get_error()) + return -1 + dpdk_version = self._ssh_client.get_output().decode("utf-8").rstrip() + self._log.debug("DPDK version %s" % dpdk_version) + if (dpdk_version >= '20.11.0'): + allow_parameter = 'allow' + else: + allow_parameter = 'pci-whitelist' + + self._log.info("Getting MAC address for assigned SRIOV VF %s" % \ + self._sriov_vf) + self._ssh_client.run_cmd("sudo /opt/rapid/port_info_app -n 4 \ + --{} {}".format(allow_parameter, self._sriov_vf)) if ret != 0: self._log.error("Failed to get MAC address!" "Error %s" % self._ssh_client.get_error()) @@ -204,6 +250,11 @@ class Pod: """ self._nodeSelector_hostname = hostname + def set_spec_file_name(self, file_name): + """Set pod spec filename. + """ + self._spec_filename = file_name + def set_ssh_credentials(self, user, rsa_private_key): """Set SSH credentials for the SSH connection to the POD. """ diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_machine.py b/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_machine.py index e46e46ef..47f858d0 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_machine.py +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_machine.py @@ -20,6 +20,7 @@ from rapid_log import RapidLog from prox_ctrl import prox_ctrl import os import re +import uuid class RapidMachine(object): """ @@ -41,8 +42,11 @@ class RapidMachine(object): while True: ip_key = 'dp_ip{}'.format(index) mac_key = 'dp_mac{}'.format(index) - if ip_key in machine_params.keys() and mac_key in machine_params.keys(): - dp_port = {'ip': machine_params[ip_key], 'mac' : machine_params[mac_key]} + if ip_key in machine_params.keys(): + if mac_key in machine_params.keys(): + dp_port = {'ip': machine_params[ip_key], 'mac' : machine_params[mac_key]} + else: + dp_port = {'ip': machine_params[ip_key], 'mac' : None} self.dp_ports.append(dict(dp_port)) self.dpdk_port_index.append(index - 1) index += 1 @@ -51,8 +55,6 @@ class RapidMachine(object): self.machine_params = machine_params self.vim = vim self.cpu_mapping = None - self.numa_nodes = [ 0 ] - self.socket_mem_mb = '512' if 'config_file' in self.machine_params.keys(): PROXConfigfile = open (self.machine_params['config_file'], 'r') PROXConfig = PROXConfigfile.read() @@ -79,7 +81,17 @@ class RapidMachine(object): def read_cpuset(self): """Read list of cpus on which we allowed to execute """ - cmd = 'cat /sys/fs/cgroup/cpuset/cpuset.cpus' + cpu_set_file = '/sys/fs/cgroup/cpuset.cpus' + cmd = 'test -e {0} && echo exists'.format(cpu_set_file) + if (self._client.run_cmd(cmd).decode().rstrip()): + cmd = 'cat {}'.format(cpu_set_file) + else: + cpu_set_file = '/sys/fs/cgroup/cpuset/cpuset.cpus' + cmd = 'test -e {0} && echo exists'.format(cpu_set_file) + if (self._client.run_cmd(cmd).decode().rstrip()): + cmd = 'cat {}'.format(cpu_set_file) + else: + RapidLog.critical('{Cannot determine cpuset') cpuset_cpus = self._client.run_cmd(cmd).decode().rstrip() RapidLog.debug('{} ({}): Allocated cpuset: {}'.format(self.name, self.ip, cpuset_cpus)) self.cpu_mapping = self.expand_list_format(cpuset_cpus) @@ -118,24 +130,10 @@ class RapidMachine(object): RapidLog.debug('{} ({}): cores {} remapped to {}'.format(self.name, self.ip, self.machine_params['cores'], cpus_remapped)) self.machine_params['cores'] = cpus_remapped - def read_cpuset_mems(self): - """Read list of NUMA nodes on which we allowed to allocate memory - """ - cmd = 'cat /sys/fs/cgroup/cpuset/cpuset.mems' - cpuset_mems = self._client.run_cmd(cmd).decode().rstrip() - RapidLog.debug('{} ({}): Allowed NUMA nodes: {}'.format(self.name, self.ip, cpuset_mems)) - self.numa_nodes = self.expand_list_format(cpuset_mems) - RapidLog.debug('{} ({}): Expanded allowed NUMA nodes: {}'.format(self.name, self.ip, self.numa_nodes)) - - def get_prox_socket_mem_str(self): - socket_mem_str = '' - for node in range(self.numa_nodes[-1] + 1): - if node in self.numa_nodes: - socket_mem_str += self.socket_mem_mb + ',' - else: - socket_mem_str += '0,' - socket_mem_str = socket_mem_str[:-1] - return socket_mem_str + if 'altcores' in self.machine_params.keys(): + cpus_remapped = self.remap_cpus(self.machine_params['altcores']) + RapidLog.debug('{} ({}): altcores {} remapped to {}'.format(self.name, self.ip, self.machine_params['altcores'], cpus_remapped)) + self.machine_params['altcores'] = cpus_remapped def devbind(self): # Script to bind the right network interface to the poll mode driver @@ -158,22 +156,47 @@ class RapidMachine(object): LuaFile.write('local_ip{}="{}"\n'.format(index, dp_port['ip'])) LuaFile.write('local_hex_ip{}=convertIPToHex(local_ip{})\n'.format(index, index)) if self.vim in ['kubernetes']: - socket_mem_str = self.get_prox_socket_mem_str() - RapidLog.debug('{} ({}): PROX socket mem str: {}'.format(self.name, self.ip, socket_mem_str)) - LuaFile.write("eal=\"--socket-mem=%s --file-prefix %s --pci-whitelist %s\"\n" % (socket_mem_str, self.name, self.machine_params['dp_pci_dev'])) + cmd = 'cat /opt/rapid/dpdk_version' + dpdk_version = self._client.run_cmd(cmd).decode().rstrip() + if (dpdk_version >= '20.11.0'): + allow_parameter = 'allow' + else: + allow_parameter = 'pci-whitelist' + eal_line = 'eal=\"--file-prefix {}{} --{} {} --force-max-simd-bitwidth=512'.format( + self.name, str(uuid.uuid4()), allow_parameter, + self.machine_params['dp_pci_dev']) + looking_for_qat = True + index = 0 + while (looking_for_qat): + if 'qat_pci_dev{}'.format(index) in self.machine_params: + eal_line += ' --{} {}'.format(allow_parameter, + self.machine_params['qat_pci_dev{}'.format(index)]) + index += 1 + else: + looking_for_qat = False + eal_line += '"\n' + LuaFile.write(eal_line) else: LuaFile.write("eal=\"\"\n") if 'mcore' in self.machine_params.keys(): - LuaFile.write('mcore="%s"\n'% ','.join(map(str, self.machine_params['mcore']))) + LuaFile.write('mcore="%s"\n'% ','.join(map(str, + self.machine_params['mcore']))) if 'cores' in self.machine_params.keys(): - LuaFile.write('cores="%s"\n'% ','.join(map(str, self.machine_params['cores']))) + LuaFile.write('cores="%s"\n'% ','.join(map(str, + self.machine_params['cores']))) + if 'altcores' in self.machine_params.keys(): + LuaFile.write('altcores="%s"\n'% ','.join(map(str, + self.machine_params['altcores']))) if 'ports' in self.machine_params.keys(): - LuaFile.write('ports="%s"\n'% ','.join(map(str, self.machine_params['ports']))) + LuaFile.write('ports="%s"\n'% ','.join(map(str, + self.machine_params['ports']))) if 'dest_ports' in self.machine_params.keys(): for index, dest_port in enumerate(self.machine_params['dest_ports'], start = 1): LuaFile.write('dest_ip{}="{}"\n'.format(index, dest_port['ip'])) LuaFile.write('dest_hex_ip{}=convertIPToHex(dest_ip{})\n'.format(index, index)) - LuaFile.write('dest_hex_mac{}="{}"\n'.format(index , dest_port['mac'].replace(':',' '))) + if dest_port['mac']: + LuaFile.write('dest_hex_mac{}="{}"\n'.format(index , + dest_port['mac'].replace(':',' '))) if 'gw_vm' in self.machine_params.keys(): for index, gw_ip in enumerate(self.machine_params['gw_ips'], start = 1): @@ -193,7 +216,6 @@ class RapidMachine(object): self.devbind() if self.vim in ['kubernetes']: self.read_cpuset() - self.read_cpuset_mems() self.remap_all_cpus() _, prox_config_file_name = os.path.split(self. machine_params['config_file']) diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_parser.py b/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_parser.py index 445aed4b..143323b8 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_parser.py +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_parser.py @@ -63,12 +63,6 @@ class RapidConfigParser(object): test_params['user'] = config.get('ssh', 'user') if config.has_option('ssh', 'key'): test_params['key'] = config.get('ssh', 'key') - if test_params['user'] in ['rapid']: - if test_params['key'] != 'rapid_rsa_key': - RapidLog.debug(("Key file {} for user {} overruled by key file:" - " rapid_rsa_key").format(test_params['key'], - test_params['user'])) - test_params['key'] = 'rapid_rsa_key' else: test_params['key'] = None if config.has_option('ssh', 'password'): @@ -91,7 +85,7 @@ class RapidConfigParser(object): 'flowsize','warmupflowsize','warmuptime', 'steps']: test[option] = int(testconfig.get(section, option)) elif option in ['startspeed', 'step', 'drop_rate_threshold', - 'lat_avg_threshold','lat_perc_threshold', + 'generator_threshold','lat_avg_threshold','lat_perc_threshold', 'lat_max_threshold','accuracy','maxr','maxz', 'ramp_step','warmupspeed','mis_ordered_threshold']: test[option] = float(testconfig.get(section, option)) @@ -99,11 +93,12 @@ class RapidConfigParser(object): test[option] = testconfig.get(section, option) tests.append(dict(test)) for test in tests: - if test['test'] in ['flowsizetest','TST009test']: + if test['test'] in ['flowsizetest', 'TST009test', 'increment_till_fail']: if 'drop_rate_threshold' not in test.keys(): test['drop_rate_threshold'] = 0 - latency_thresholds = ['lat_avg_threshold','lat_perc_threshold','lat_max_threshold','mis_ordered_threshold'] - for threshold in latency_thresholds: + thresholds = ['generator_threshold','lat_avg_threshold', \ + 'lat_perc_threshold','lat_max_threshold','mis_ordered_threshold'] + for threshold in thresholds: if threshold not in test.keys(): test[threshold] = inf test_params['tests'] = tests @@ -142,7 +137,8 @@ class RapidConfigParser(object): for option in options: if option in ['prox_socket','prox_launch_exit','monitor']: machine[option] = testconfig.getboolean(section, option) - elif option in ['mcore', 'cores', 'gencores','latcores']: + elif option in ['mcore', 'cores', 'gencores', 'latcores', + 'altcores']: machine[option] = ast.literal_eval(testconfig.get( section, option)) elif option in ['bucket_size_exp']: @@ -169,10 +165,13 @@ class RapidConfigParser(object): while True: dp_ip_key = 'dp_ip{}'.format(index) dp_mac_key = 'dp_mac{}'.format(index) - if dp_ip_key in machines[int(machine['dest_vm'])-1].keys() and \ - dp_mac_key in machines[int(machine['dest_vm'])-1].keys(): - dp_port = {'ip': machines[int(machine['dest_vm'])-1][dp_ip_key], - 'mac' : machines[int(machine['dest_vm'])-1][dp_mac_key]} + if dp_ip_key in machines[int(machine['dest_vm'])-1].keys(): + if dp_mac_key in machines[int(machine['dest_vm'])-1].keys(): + dp_port = {'ip': machines[int(machine['dest_vm'])-1][dp_ip_key], + 'mac' : machines[int(machine['dest_vm'])-1][dp_mac_key]} + else: + dp_port = {'ip': machines[int(machine['dest_vm'])-1][dp_ip_key], + 'mac' : None} dp_ports.append(dict(dp_port)) index += 1 else: diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_test.py b/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_test.py index 6badfb45..deba695f 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_test.py +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/rapid_test.py @@ -261,6 +261,7 @@ class RapidTest(object): 'lat_perc' : '', 'lat_max' : '', 'abs_drop_rate' : '', + 'mis_ordered' : '', 'drop_rate' : ''} RapidLog.info(self.report_result(flow_number, size, iteration_data, iteration_prefix )) @@ -346,6 +347,7 @@ class RapidTest(object): 'lat_perc' : '', 'lat_max' : '', 'abs_drop_rate' : '', + 'mis_ordered' : '', 'drop_rate' : ''} RapidLog.info(self.report_result(flow_number, size, time_loop_data, time_loop_prefix)) diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/runrapid.py b/VNFs/DPPD-PROX/helper-scripts/rapid/runrapid.py index 7a1c8eb5..7ec270a1 100755 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/runrapid.py +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/runrapid.py @@ -59,12 +59,20 @@ class RapidTestManager(object): def run_tests(self, test_params): test_params = RapidConfigParser.parse_config(test_params) - RapidLog.debug(test_params) monitor_gen = monitor_sut = False background_machines = [] sut_machine = gen_machine = None configonly = test_params['configonly'] + machine_names = [] + machine_counter = {} for machine_params in test_params['machines']: + if machine_params['name'] not in machine_names: + machine_names.append(machine_params['name']) + machine_counter[machine_params['name']] = 1 + else: + machine_counter[machine_params['name']] += 1 + machine_params['name'] = '{}_{}'.format(machine_params['name'], + machine_counter[machine_params['name']]) if 'gencores' in machine_params.keys(): machine = RapidGeneratorMachine(test_params['key'], test_params['user'], test_params['password'], @@ -94,6 +102,7 @@ class RapidTestManager(object): if machine_params['prox_socket']: sut_machine = machine self.machines.append(machine) + RapidLog.debug(test_params) try: prox_executor = concurrent.futures.ThreadPoolExecutor(max_workers=len(self.machines)) self.future_to_prox = {prox_executor.submit(machine.start_prox): machine for machine in self.machines} diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/start.sh b/VNFs/DPPD-PROX/helper-scripts/rapid/start.sh index 7fbeedf8..78772dd2 100755 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/start.sh +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/start.sh @@ -17,7 +17,8 @@ function save_k8s_envs() { - printenv | grep "PCIDEVICE_INTEL_COM" > /opt/rapid/k8s_sriov_device_plugin_envs + printenv | grep "PCIDEVICE" > /opt/rapid/k8s_sriov_device_plugin_envs + printenv | grep "QAT[0-9]" > /opt/rapid/k8s_qat_device_plugin_envs } function create_tun() @@ -34,6 +35,9 @@ create_tun touch /opt/rapid/system_ready_for_rapid # Start SSH server in background -/usr/sbin/sshd +echo "mkdir -p /var/run/sshd" >> /etc/rc.local +service ssh start -exec sleep infinity +echo "rapid ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers + +sleep infinity diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/tests/encrypt.test b/VNFs/DPPD-PROX/helper-scripts/rapid/tests/encrypt.test new file mode 100644 index 00000000..bc5e96b8 --- /dev/null +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/tests/encrypt.test @@ -0,0 +1,70 @@ +## +## Copyright (c) 2023 luc.provoost@gmail.com +## +## Licensed under the Apache License, Version 2.0 (the "License"); +## you may not use this file except in compliance with the License. +## You may obtain a copy of the License at +## +## http://www.apache.org/licenses/LICENSE-2.0 +## +## Unless required by applicable law or agreed to in writing, software +## distributed under the License is distributed on an "AS IS" BASIS, +## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +## See the License for the specific language governing permissions and +## limitations under the License. +## +# CHECK README IN THIS DIRECTORY FOR MORE EXPLANATION +# ON PARAMETERS IN THIS FILE + +[TestParameters] +name = EncryptionDecryption +number_of_tests = 1 +total_number_of_test_machines = 2 +lat_percentile = 99 + +[TestM1] +name = Generator +config_file = configs/gen.cfg +dest_vm = 2 +mcore = [0] +gencores = [1] +latcores = [3] +bucket_size_exp = 16 +#prox_launch_exit = false + +[TestM2] +name = Encrypt +config_file = configs/esp.cfg +dest_vm = 1 +mcore = [0] +cores = [1] +altcores=[2] +#prox_socket = true +#prox_launch_exit = false + +[test1] +test=flowsizetest +warmupflowsize=512 +warmupimix=[64] +warmupspeed=1 +warmuptime=2 +# Each element in the imix list will result in a separate test. Each element +# is on its turn a list of packet sizes which will be used during one test +# execution. If you only want to test 1 size, define a list with only one +# element. +#imixs=[[64],[64,250,800,800]] +imixs=[[1500],[512],[256],[128]] +# the number of flows in the list need to be powers of 2, max 2^30 +# If not a power of 2, we will use the lowest power of 2 that is larger than +# the requested number of flows. e.g. 9 will result in 16 flows +flows=[64] +# Setting one of the following thresholds to infinity (inf) +# results in the criterion not being evaluated to rate the test as succesful +drop_rate_threshold = 0.5 +lat_avg_threshold = inf +lat_perc_threshold = inf +lat_max_threshold = inf +accuracy = 5 +startspeed = 250 +#ramp_step = 1 + diff --git a/VNFs/DPPD-PROX/lconf.h b/VNFs/DPPD-PROX/lconf.h index 8ac1112e..897e6b37 100644 --- a/VNFs/DPPD-PROX/lconf.h +++ b/VNFs/DPPD-PROX/lconf.h @@ -17,6 +17,11 @@ #ifndef _LCONF_H_ #define _LCONF_H_ +#include <rte_common.h> +#ifndef __rte_cache_aligned +#include <rte_memory.h> +#endif + #include "task_init.h" #include "stats.h" @@ -100,8 +105,8 @@ static inline void lconf_flush_all_queues(struct lcore_cfg *lconf) for (uint8_t task_id = 0; task_id < lconf->n_tasks_all; ++task_id) { task = lconf->tasks_all[task_id]; - if (!(task->flags & FLAG_TX_FLUSH) || (task->flags & FLAG_NEVER_FLUSH)) { - task->flags |= FLAG_TX_FLUSH; + if (!(task->flags & TBASE_FLAG_TX_FLUSH) || (task->flags & TBASE_FLAG_NEVER_FLUSH)) { + task->flags |= TBASE_FLAG_TX_FLUSH; continue; } lconf->flush_queues[task_id](task); diff --git a/VNFs/DPPD-PROX/main.c b/VNFs/DPPD-PROX/main.c index ebb861c0..61abe6e6 100644 --- a/VNFs/DPPD-PROX/main.c +++ b/VNFs/DPPD-PROX/main.c @@ -116,7 +116,7 @@ static void check_mixed_normal_pipeline(void) } } -static void check_zero_rx(void) +static void check_no_rx(void) { struct lcore_cfg *lconf = NULL; struct task_args *targ; @@ -228,7 +228,7 @@ static void check_cfg_consistent(void) { check_nb_mbuf(); check_missing_rx(); - check_zero_rx(); + check_no_rx(); check_mixed_normal_pipeline(); } @@ -307,7 +307,7 @@ static void configure_if_tx_queues(struct task_args *targ, uint8_t socket) } #else if (chain_flag_always_set(targ, TASK_FEATURE_TXQ_FLAGS_NOOFFLOADS)) { - prox_port_cfg[if_port].requested_tx_offload &= ~(DEV_TX_OFFLOAD_IPV4_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM); + prox_port_cfg[if_port].requested_tx_offload &= ~(RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | RTE_ETH_TX_OFFLOAD_UDP_CKSUM); } #endif } @@ -425,12 +425,12 @@ static void configure_tx_queue_flags(void) prox_port_cfg[if_port].tx_conf.txq_flags |= ETH_TXQ_FLAGS_NOREFCOUNT; } #else - /* Set the DEV_TX_OFFLOAD_MBUF_FAST_FREE flag if none of + /* Set the RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE flag if none of the tasks up to the task transmitting to the port use refcnt and per-queue all mbufs comes from the same mempool. */ if (chain_flag_never_set(targ, TASK_FEATURE_TXQ_FLAGS_REFCOUNT)) { if (chain_flag_never_set(targ, TASK_FEATURE_TXQ_FLAGS_MULTIPLE_MEMPOOL)) - prox_port_cfg[if_port].requested_tx_offload |= DEV_TX_OFFLOAD_MBUF_FAST_FREE; + prox_port_cfg[if_port].requested_tx_offload |= RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE; } #endif } @@ -455,7 +455,7 @@ static void configure_multi_segments(void) #else // We enable "multi segment" if at least one task requires it in the chain of tasks. if (chain_flag_sometimes_set(targ, TASK_FEATURE_TXQ_FLAGS_MULTSEGS)) { - prox_port_cfg[if_port].requested_tx_offload |= DEV_TX_OFFLOAD_MULTI_SEGS; + prox_port_cfg[if_port].requested_tx_offload |= RTE_ETH_TX_OFFLOAD_MULTI_SEGS; } #endif } diff --git a/VNFs/DPPD-PROX/meson.build b/VNFs/DPPD-PROX/meson.build index 217a9679..48251e8d 100644 --- a/VNFs/DPPD-PROX/meson.build +++ b/VNFs/DPPD-PROX/meson.build @@ -26,6 +26,10 @@ project('dppd-prox', 'C', cc = meson.get_compiler('c') # Configure options for prox +# Grab the DPDK version here "manually" as it is not available in the dpdk_dep +# object +dpdk_version = run_command('pkg-config', '--modversion', 'libdpdk').stdout() + if get_option('bng_qinq').enabled() add_project_arguments('-DUSE_QINQ', language: 'c') endif @@ -119,6 +123,7 @@ ncurses_dep = dependency('ncurses', required: true) ncursesw_dep = dependency('ncursesw', required: true) libedit_dep = dependency('libedit', required: true) math_dep = cc.find_library('m', required : false) +dl_dep = cc.find_library('dl', required : true) deps = [dpdk_dep, tinfo_dep, @@ -128,13 +133,11 @@ deps = [dpdk_dep, ncursesw_dep, libedit_dep, math_dep, + dl_dep, lua_dep] -# Grab the DPDK version here "manually" as it is not available in the dpdk_dep -# object -dpdk_version = run_command('pkg-config', '--modversion', 'libdpdk').stdout() - # Explicitly add these to the dependency list +deps += [cc.find_library('rte_bus_pci', required: true)] deps += [cc.find_library('rte_bus_vdev', required: true)] if dpdk_version.version_compare('<20.11.0') @@ -147,7 +150,7 @@ sources = files( 'task_init.c', 'handle_aggregator.c', 'handle_nop.c', 'handle_irq.c', 'handle_arp.c', 'handle_impair.c', 'handle_lat.c', 'handle_qos.c', 'handle_qinq_decap4.c', 'handle_routing.c', 'handle_untag.c', - 'handle_mplstag.c', 'handle_qinq_decap6.c', 'rw_reg.c', + 'handle_mplstag.c', 'handle_qinq_decap6.c', 'handle_lb_qinq.c', 'handle_lb_pos.c', 'handle_lb_net.c', 'handle_qinq_encap4.c', 'handle_qinq_encap6.c', 'handle_classify.c', 'handle_l2fwd.c', 'handle_swap.c', 'handle_police.c', 'handle_acl.c', @@ -172,8 +175,11 @@ sources = files( 'stats_cons_cli.c', 'stats_parser.c', 'hash_set.c', 'prox_lua.c', 'prox_malloc.c', 'prox_ipv6.c', 'prox_compat.c', 'handle_nsh.c') +sources += files('rw_reg.c') + # Include a couple of source files depending on DPDK support -if cc.find_library('rte_pmd_aesni', required: false).found() +if cc.find_library('rte_crypto_ipsec_mb', required: false).found() + add_project_arguments('-DRTE_LIBRTE_PMD_AESNI_MB', language: 'c') sources += files('handle_esp.c') else warning('Building w/o IPSEC support') diff --git a/VNFs/DPPD-PROX/meson_options.txt b/VNFs/DPPD-PROX/meson_options.txt index 199d31fc..afc2be7e 100644 --- a/VNFs/DPPD-PROX/meson_options.txt +++ b/VNFs/DPPD-PROX/meson_options.txt @@ -1,6 +1,6 @@ #Keep the options sorted alphabetically -#option('crc', type: 'string', value: 'soft') option('bng_qinq', type: 'feature', value: 'enabled') +option('crc', type: 'string', value: 'hard') option('dbg', type: 'boolean', value: false) option('gen_decap_ipv6_to_ipv4_cksum', type: 'feature', value: 'enabled') option('hw_direct_stats', type: 'feature', value: 'enabled') diff --git a/VNFs/DPPD-PROX/packet_utils.c b/VNFs/DPPD-PROX/packet_utils.c index 0189a1e0..95ce7abc 100644 --- a/VNFs/DPPD-PROX/packet_utils.c +++ b/VNFs/DPPD-PROX/packet_utils.c @@ -492,6 +492,7 @@ void task_init_l3(struct task_base *tbase, struct task_args *targ) .key_len = sizeof(uint32_t), .hash_func = rte_hash_crc, .hash_func_init_val = 0, + .socket_id = socket_id, }; if (targ->flags & TASK_ARG_L3) { plog_info("\t\tInitializing L3 (IPv4)\n"); diff --git a/VNFs/DPPD-PROX/prox_args.c b/VNFs/DPPD-PROX/prox_args.c index 0ef1f8b1..dc212494 100644 --- a/VNFs/DPPD-PROX/prox_args.c +++ b/VNFs/DPPD-PROX/prox_args.c @@ -610,18 +610,18 @@ static int get_port_cfg(unsigned sindex, char *str, void *data) if (parse_bool(&val, pkey)) { return -1; } -#if defined(DEV_RX_OFFLOAD_CRC_STRIP) +#if defined(RTE_ETH_RX_OFFLOAD_CRC_STRIP) if (val) - cfg->requested_rx_offload |= DEV_RX_OFFLOAD_CRC_STRIP; + cfg->requested_rx_offload |= RTE_ETH_RX_OFFLOAD_CRC_STRIP; else - cfg->requested_rx_offload &= ~DEV_RX_OFFLOAD_CRC_STRIP; + cfg->requested_rx_offload &= ~RTE_ETH_RX_OFFLOAD_CRC_STRIP; #else -#if defined (DEV_RX_OFFLOAD_KEEP_CRC) +#if defined (RTE_ETH_RX_OFFLOAD_KEEP_CRC) if (val) - cfg->requested_rx_offload &= ~DEV_RX_OFFLOAD_KEEP_CRC; + cfg->requested_rx_offload &= ~RTE_ETH_RX_OFFLOAD_KEEP_CRC; else + cfg->requested_rx_offload |= RTE_ETH_RX_OFFLOAD_KEEP_CRC; #endif - cfg->requested_rx_offload |= DEV_RX_OFFLOAD_KEEP_CRC; #endif } @@ -635,11 +635,11 @@ static int get_port_cfg(unsigned sindex, char *str, void *data) return -1; } if (val) { - cfg->requested_rx_offload |= DEV_RX_OFFLOAD_VLAN_STRIP; - cfg->requested_tx_offload |= DEV_TX_OFFLOAD_VLAN_INSERT; + cfg->requested_rx_offload |= RTE_ETH_RX_OFFLOAD_VLAN_STRIP; + cfg->requested_tx_offload |= RTE_ETH_TX_OFFLOAD_VLAN_INSERT; } else { - cfg->requested_rx_offload &= ~DEV_RX_OFFLOAD_VLAN_STRIP; - cfg->requested_tx_offload &= ~DEV_TX_OFFLOAD_VLAN_INSERT; + cfg->requested_rx_offload &= ~RTE_ETH_RX_OFFLOAD_VLAN_STRIP; + cfg->requested_tx_offload &= ~RTE_ETH_TX_OFFLOAD_VLAN_INSERT; } #else plog_warn("vlan option not supported : update DPDK at least to 18.08 to support this option\n"); @@ -655,10 +655,14 @@ static int get_port_cfg(unsigned sindex, char *str, void *data) // A frame of 1526 bytes (1500 bytes mtu, 14 bytes hdr, 4 bytes crc and 8 bytes vlan) // should not be considered as a jumbo frame. However rte_ethdev.c considers that // the max_rx_pkt_len for a non jumbo frame is 1518 +#if RTE_VERSION < RTE_VERSION_NUM(21,11,0,0) cfg->port_conf.rxmode.max_rx_pkt_len = cfg->mtu + PROX_RTE_ETHER_HDR_LEN + PROX_RTE_ETHER_CRC_LEN; - if (cfg->port_conf.rxmode.max_rx_pkt_len > PROX_RTE_ETHER_MAX_LEN) { - cfg->requested_rx_offload |= DEV_RX_OFFLOAD_JUMBO_FRAME; - } + if (cfg->port_conf.rxmode.max_rx_pkt_len > PROX_RTE_ETHER_MAX_LEN) +#else + cfg->port_conf.rxmode.mtu = cfg->mtu; + if (cfg->port_conf.rxmode.mtu > PROX_MTU) +#endif + cfg->requested_rx_offload |= RTE_ETH_RX_OFFLOAD_JUMBO_FRAME; } } @@ -668,8 +672,8 @@ static int get_port_cfg(unsigned sindex, char *str, void *data) return -1; } if (val) { - cfg->port_conf.rxmode.mq_mode = ETH_MQ_RX_RSS; - cfg->port_conf.rx_adv_conf.rss_conf.rss_hf = ETH_RSS_IPV4; + cfg->port_conf.rxmode.mq_mode = RTE_ETH_MQ_RX_RSS; + cfg->port_conf.rx_adv_conf.rss_conf.rss_hf = RTE_ETH_RSS_IPV4; } } else if (STR_EQ(str, "rx_ring")) { @@ -2296,10 +2300,14 @@ int prox_setup_rte(const char *prog_name) sprintf(rte_arg[++argc], "-c%s", tmp); rte_argv[argc] = rte_arg[argc]; #if RTE_VERSION >= RTE_VERSION_NUM(1,8,0,0) + uint32_t master_core = prox_cfg.master; if (prox_cfg.flags & DSF_USE_DUMMY_CPU_TOPO) - sprintf(rte_arg[++argc], "--master-lcore=%u", 0); - else - sprintf(rte_arg[++argc], "--master-lcore=%u", prox_cfg.master); + master_core = 0; +#if RTE_VERSION < RTE_VERSION_NUM(21,11,0,0) + sprintf(rte_arg[++argc], "--master-lcore=%u", master_core); +#else + sprintf(rte_arg[++argc], "--main-lcore=%u", master_core); +#endif rte_argv[argc] = rte_arg[argc]; #else /* For old DPDK versions, the master core had to be the first diff --git a/VNFs/DPPD-PROX/prox_cksum.c b/VNFs/DPPD-PROX/prox_cksum.c index 11d37a6c..401191f6 100644 --- a/VNFs/DPPD-PROX/prox_cksum.c +++ b/VNFs/DPPD-PROX/prox_cksum.c @@ -91,14 +91,14 @@ static inline void prox_write_tcp_pseudo_hdr(prox_rte_tcp_hdr *tcp, uint16_t len inline void prox_ip_udp_cksum(struct rte_mbuf *mbuf, prox_rte_ipv4_hdr *pip, uint16_t l2_len, uint16_t l3_len, int cksum_offload) { - prox_ip_cksum(mbuf, pip, l2_len, l3_len, cksum_offload & DEV_TX_OFFLOAD_IPV4_CKSUM); + prox_ip_cksum(mbuf, pip, l2_len, l3_len, cksum_offload & RTE_ETH_TX_OFFLOAD_IPV4_CKSUM); uint32_t l4_len = rte_bswap16(pip->total_length) - l3_len; if (pip->next_proto_id == IPPROTO_UDP) { prox_rte_udp_hdr *udp = (prox_rte_udp_hdr *)(((uint8_t*)pip) + l3_len); #ifndef SOFT_CRC - if (cksum_offload & DEV_TX_OFFLOAD_UDP_CKSUM) { - mbuf->ol_flags |= PKT_TX_UDP_CKSUM; + if (cksum_offload & RTE_ETH_TX_OFFLOAD_UDP_CKSUM) { + mbuf->ol_flags |= RTE_MBUF_F_TX_UDP_CKSUM; prox_write_udp_pseudo_hdr(udp, l4_len, pip->src_addr, pip->dst_addr); } else #endif @@ -106,9 +106,9 @@ inline void prox_ip_udp_cksum(struct rte_mbuf *mbuf, prox_rte_ipv4_hdr *pip, uin } else if (pip->next_proto_id == IPPROTO_TCP) { prox_rte_tcp_hdr *tcp = (prox_rte_tcp_hdr *)(((uint8_t*)pip) + l3_len); #ifndef SOFT_CRC - if (cksum_offload & DEV_TX_OFFLOAD_TCP_CKSUM) { + if (cksum_offload & RTE_ETH_TX_OFFLOAD_TCP_CKSUM) { prox_write_tcp_pseudo_hdr(tcp, l4_len, pip->src_addr, pip->dst_addr); - mbuf->ol_flags |= PKT_TX_UDP_CKSUM; + mbuf->ol_flags |= RTE_MBUF_F_TX_UDP_CKSUM; } else #endif prox_tcp_cksum_sw(tcp, l4_len, pip->src_addr, pip->dst_addr); diff --git a/VNFs/DPPD-PROX/prox_cksum.h b/VNFs/DPPD-PROX/prox_cksum.h index 03be595a..d4ac5a6b 100644 --- a/VNFs/DPPD-PROX/prox_cksum.h +++ b/VNFs/DPPD-PROX/prox_cksum.h @@ -42,7 +42,7 @@ static void prox_ip_cksum_hw(struct rte_mbuf *mbuf, uint16_t l2_len, uint16_t l3 #else mbuf->tx_offload = CALC_TX_OL(l2_len, l3_len); #endif - mbuf->ol_flags |= PKT_TX_IP_CKSUM; + mbuf->ol_flags |= RTE_MBUF_F_TX_IP_CKSUM; } void prox_ip_cksum_sw(prox_rte_ipv4_hdr *buf); diff --git a/VNFs/DPPD-PROX/prox_compat.h b/VNFs/DPPD-PROX/prox_compat.h index 404ce9ed..d4b7f247 100644 --- a/VNFs/DPPD-PROX/prox_compat.h +++ b/VNFs/DPPD-PROX/prox_compat.h @@ -153,16 +153,6 @@ static void *prox_rte_table_create(struct prox_rte_table_params *params, int soc #define rte_cryptodev_sym_get_private_session_size rte_cryptodev_get_private_session_size #endif -#ifndef DEV_RX_OFFLOAD_JUMBO_FRAME -#define DEV_RX_OFFLOAD_JUMBO_FRAME 0x00000800 -#endif - -#ifndef DEV_RX_OFFLOAD_KEEP_CRC -#ifndef DEV_RX_OFFLOAD_CRC_STRIP -#define DEV_RX_OFFLOAD_CRC_STRIP 0x00001000 -#endif -#endif - #if RTE_VERSION < RTE_VERSION_NUM(19,2,0,0) #define RTE_COLOR_GREEN e_RTE_METER_GREEN #define RTE_COLOR_YELLOW e_RTE_METER_YELLOW @@ -210,7 +200,7 @@ typedef struct icmp_hdr prox_rte_icmp_hdr; #define PROX_RTE_IS_IPV4_MCAST IS_IPV4_MCAST #define prox_rte_is_same_ether_addr is_same_ether_addr #define prox_rte_is_zero_ether_addr is_zero_ether_addr -#else +#else // >= 19.08 #define PROX_RTE_ETHER_CRC_LEN RTE_ETHER_CRC_LEN #define PROX_RTE_ETHER_MIN_LEN RTE_ETHER_MIN_LEN @@ -229,7 +219,16 @@ typedef struct icmp_hdr prox_rte_icmp_hdr; typedef struct rte_ipv6_hdr prox_rte_ipv6_hdr; typedef struct rte_ipv4_hdr prox_rte_ipv4_hdr; typedef struct rte_ether_addr prox_rte_ether_addr; +#if RTE_VERSION < RTE_VERSION_NUM(21,11,0,0) typedef struct rte_ether_hdr prox_rte_ether_hdr; +#else +typedef struct prox_rte_ether_hdr +{ + struct rte_ether_addr d_addr; /**< Destination address. */ + struct rte_ether_addr s_addr; /**< Source address. */ + rte_be16_t ether_type; /**< Frame type. */ +} __rte_aligned(2) prox_rte_ether_hdr; +#endif typedef struct rte_vlan_hdr prox_rte_vlan_hdr; typedef struct rte_vxlan_gpe_hdr prox_rte_vxlan_gpe_hdr; typedef struct rte_udp_hdr prox_rte_udp_hdr; @@ -286,4 +285,378 @@ static int prox_rte_cryptodev_queue_pair_setup(uint8_t dev_id, uint16_t queue_pa #define prox_rte_eth_dev_count_avail() rte_eth_dev_count_avail() #endif +// deal with RTE_DEPRECATED symbols + +#if RTE_VERSION < RTE_VERSION_NUM(20,11,0,0) +#define SKIP_MAIN SKIP_MASTER +#define CALL_MAIN CALL_MASTER +#define RTE_DEVTYPE_ALLOWED RTE_DEVTYPE_WHITELISTED_PCI +#define RTE_DEVTYPE_BLOCKED RTE_DEVTYPE_BLACKLISTED_PCI +#define RTE_LCORE_FOREACH_WORKER RTE_LCORE_FOREACH_SLAVE +#if RTE_VERSION >= RTE_VERSION_NUM(17,8,0,0) +#define RTE_DEV_ALLOWED RTE_DEV_WHITELISTED +#define RTE_DEV_BLOCKED RTE_DEV_BLACKLISTED +#define RTE_BUS_SCAN_ALLOWLIST RTE_BUS_SCAN_WHITELIST +#define RTE_BUS_SCAN_BLOCKLIST RTE_BUS_SCAN_BLACKLIST +#endif +#endif + +#if RTE_VERSION < RTE_VERSION_NUM(21,5,0,0) +#define RTE_PCI_ANY_ID PCI_ANY_ID +#define PKT_RX_OUTER_IP_CKSUM_BAD PKT_RX_EIP_CKSUM_BAD +#endif + +#if RTE_VERSION < RTE_VERSION_NUM(21,11,0,0) +#define RTE_MEMPOOL_HEADER_SIZE MEMPOOL_HEADER_SIZE +#define RTE_MBUF_F_RX_RSS_HASH PKT_RX_RSS_HASH +#define RTE_MBUF_F_RX_FDIR PKT_RX_FDIR +#define RTE_MBUF_F_RX_OUTER_IP_CKSUM_BAD PKT_RX_OUTER_IP_CKSUM_BAD +#define RTE_MBUF_F_RX_IP_CKSUM_BAD PKT_RX_IP_CKSUM_BAD +#define RTE_MBUF_F_RX_L4_CKSUM_BAD PKT_RX_L4_CKSUM_BAD +#define RTE_MBUF_F_RX_IEEE1588_PTP PKT_RX_IEEE1588_PTP +#define RTE_MBUF_F_RX_IEEE1588_TMST PKT_RX_IEEE1588_TMST +#define RTE_MBUF_F_RX_FDIR_ID PKT_RX_FDIR_ID +#define RTE_MBUF_F_RX_FDIR_FLX PKT_RX_FDIR_FLX +#define RTE_MBUF_F_TX_QINQ PKT_TX_QINQ_PKT +#define RTE_MBUF_F_TX_TCP_SEG PKT_TX_TCP_SEG +#define RTE_MBUF_F_TX_IEEE1588_TMST PKT_TX_IEEE1588_TMST +#define RTE_MBUF_F_TX_L4_NO_CKSUM PKT_TX_L4_NO_CKSUM +#define RTE_MBUF_F_TX_TCP_CKSUM PKT_TX_TCP_CKSUM +#define RTE_MBUF_F_TX_SCTP_CKSUM PKT_TX_SCTP_CKSUM +#define RTE_MBUF_F_TX_UDP_CKSUM PKT_TX_UDP_CKSUM +#define RTE_MBUF_F_TX_L4_MASK PKT_TX_L4_MASK +#define RTE_MBUF_F_TX_IP_CKSUM PKT_TX_IP_CKSUM +#define RTE_MBUF_F_TX_IPV4 PKT_TX_IPV4 +#define RTE_MBUF_F_TX_IPV6 PKT_TX_IPV6 +#define RTE_MBUF_F_TX_VLAN PKT_TX_VLAN_PKT +#define RTE_MBUF_F_TX_OUTER_IP_CKSUM PKT_TX_OUTER_IP_CKSUM +#define RTE_MBUF_F_TX_OUTER_IPV4 PKT_TX_OUTER_IPV4 +#define RTE_MBUF_F_TX_OUTER_IPV6 PKT_TX_OUTER_IPV6 +#define RTE_MBUF_F_INDIRECT IND_ATTACHED_MBUF +#define RTE_ETH_LINK_SPEED_AUTONEG ETH_LINK_SPEED_AUTONEG +#define RTE_ETH_LINK_SPEED_FIXED ETH_LINK_SPEED_FIXED +#define RTE_ETH_LINK_SPEED_10M_HD ETH_LINK_SPEED_10M_HD +#define RTE_ETH_LINK_SPEED_10M ETH_LINK_SPEED_10M +#define RTE_ETH_LINK_SPEED_100M_HD ETH_LINK_SPEED_100M_HD +#define RTE_ETH_LINK_SPEED_100M ETH_LINK_SPEED_100M +#define RTE_ETH_LINK_SPEED_1G ETH_LINK_SPEED_1G +#define RTE_ETH_LINK_SPEED_2_5G ETH_LINK_SPEED_2_5G +#define RTE_ETH_LINK_SPEED_5G ETH_LINK_SPEED_5G +#define RTE_ETH_LINK_SPEED_10G ETH_LINK_SPEED_10G +#define RTE_ETH_LINK_SPEED_20G ETH_LINK_SPEED_20G +#define RTE_ETH_LINK_SPEED_25G ETH_LINK_SPEED_25G +#define RTE_ETH_LINK_SPEED_40G ETH_LINK_SPEED_40G +#define RTE_ETH_LINK_SPEED_50G ETH_LINK_SPEED_50G +#define RTE_ETH_LINK_SPEED_56G ETH_LINK_SPEED_56G +#define RTE_ETH_LINK_SPEED_100G ETH_LINK_SPEED_100G +#define RTE_ETH_SPEED_NUM_NONE ETH_SPEED_NUM_NONE +#define RTE_ETH_SPEED_NUM_10M ETH_SPEED_NUM_10M +#define RTE_ETH_SPEED_NUM_100M ETH_SPEED_NUM_100M +#define RTE_ETH_SPEED_NUM_1G ETH_SPEED_NUM_1G +#define RTE_ETH_SPEED_NUM_2_5G ETH_SPEED_NUM_2_5G +#define RTE_ETH_SPEED_NUM_5G ETH_SPEED_NUM_5G +#define RTE_ETH_SPEED_NUM_10G ETH_SPEED_NUM_10G +#define RTE_ETH_SPEED_NUM_20G ETH_SPEED_NUM_20G +#define RTE_ETH_SPEED_NUM_25G ETH_SPEED_NUM_25G +#define RTE_ETH_SPEED_NUM_40G ETH_SPEED_NUM_40G +#define RTE_ETH_SPEED_NUM_50G ETH_SPEED_NUM_50G +#define RTE_ETH_SPEED_NUM_56G ETH_SPEED_NUM_56G +#define RTE_ETH_SPEED_NUM_100G ETH_SPEED_NUM_100G +#define RTE_ETH_LINK_HALF_DUPLEX ETH_LINK_HALF_DUPLEX +#define RTE_ETH_LINK_FULL_DUPLEX ETH_LINK_FULL_DUPLEX +#define RTE_ETH_LINK_DOWN ETH_LINK_DOWN +#define RTE_ETH_LINK_UP ETH_LINK_UP +#define RTE_ETH_LINK_FIXED ETH_LINK_FIXED +#define RTE_ETH_LINK_AUTONEG ETH_LINK_AUTONEG +#define RTE_ETH_MQ_RX_RSS_FLAG ETH_MQ_RX_RSS_FLAG +#define RTE_ETH_MQ_RX_DCB_FLAG ETH_MQ_RX_DCB_FLAG +#define RTE_ETH_MQ_RX_VMDQ_FLAG ETH_MQ_RX_VMDQ_FLAG +#define RTE_ETH_MQ_RX_NONE ETH_MQ_RX_NONE +#define RTE_ETH_MQ_RX_RSS ETH_MQ_RX_RSS +#define RTE_ETH_MQ_RX_DCB ETH_MQ_RX_DCB +#define RTE_ETH_MQ_RX_DCB_RSS ETH_MQ_RX_DCB_RSS +#define RTE_ETH_MQ_RX_VMDQ_ONLY ETH_MQ_RX_VMDQ_ONLY +#define RTE_ETH_MQ_RX_VMDQ_RSS ETH_MQ_RX_VMDQ_RSS +#define RTE_ETH_MQ_RX_VMDQ_DCB ETH_MQ_RX_VMDQ_DCB +#define RTE_ETH_MQ_RX_VMDQ_DCB_RSS ETH_MQ_RX_VMDQ_DCB_RSS +#define RTE_ETH_MQ_TX_NONE ETH_MQ_TX_NONE +#define RTE_ETH_MQ_TX_DCB ETH_MQ_TX_DCB +#define RTE_ETH_MQ_TX_VMDQ_DCB ETH_MQ_TX_VMDQ_DCB +#define RTE_ETH_MQ_TX_VMDQ_ONLY ETH_MQ_TX_VMDQ_ONLY +#define RTE_ETH_VLAN_TYPE_UNKNOWN ETH_VLAN_TYPE_UNKNOWN +#define RTE_ETH_VLAN_TYPE_INNER ETH_VLAN_TYPE_INNER +#define RTE_ETH_VLAN_TYPE_OUTER ETH_VLAN_TYPE_OUTER +#define RTE_ETH_VLAN_TYPE_MAX ETH_VLAN_TYPE_MAX +#define RTE_ETH_RSS_IPV4 ETH_RSS_IPV4 +#define RTE_ETH_RSS_FRAG_IPV4 ETH_RSS_FRAG_IPV4 +#define RTE_ETH_RSS_NONFRAG_IPV4_TCP ETH_RSS_NONFRAG_IPV4_TCP +#define RTE_ETH_RSS_NONFRAG_IPV4_UDP ETH_RSS_NONFRAG_IPV4_UDP +#define RTE_ETH_RSS_NONFRAG_IPV4_SCTP ETH_RSS_NONFRAG_IPV4_SCTP +#define RTE_ETH_RSS_NONFRAG_IPV4_OTHER ETH_RSS_NONFRAG_IPV4_OTHER +#define RTE_ETH_RSS_IPV6 ETH_RSS_IPV6 +#define RTE_ETH_RSS_FRAG_IPV6 ETH_RSS_FRAG_IPV6 +#define RTE_ETH_RSS_NONFRAG_IPV6_TCP ETH_RSS_NONFRAG_IPV6_TCP +#define RTE_ETH_RSS_NONFRAG_IPV6_UDP ETH_RSS_NONFRAG_IPV6_UDP +#define RTE_ETH_RSS_NONFRAG_IPV6_SCTP ETH_RSS_NONFRAG_IPV6_SCTP +#define RTE_ETH_RSS_NONFRAG_IPV6_OTHER ETH_RSS_NONFRAG_IPV6_OTHER +#define RTE_ETH_RSS_L2_PAYLOAD ETH_RSS_L2_PAYLOAD +#define RTE_ETH_RSS_IPV6_EX ETH_RSS_IPV6_EX +#define RTE_ETH_RSS_IPV6_TCP_EX ETH_RSS_IPV6_TCP_EX +#define RTE_ETH_RSS_IPV6_UDP_EX ETH_RSS_IPV6_UDP_EX +#define RTE_ETH_RSS_IP ETH_RSS_IP +#define RTE_ETH_RSS_UDP ETH_RSS_UDP +#define RTE_ETH_RSS_TCP ETH_RSS_TCP +#define RTE_ETH_RSS_SCTP ETH_RSS_SCTP +#define RTE_ETH_RSS_PROTO_MASK ETH_RSS_PROTO_MASK +#define RTE_ETH_RSS_RETA_SIZE_64 ETH_RSS_RETA_SIZE_64 +#define RTE_ETH_RSS_RETA_SIZE_128 ETH_RSS_RETA_SIZE_128 +#define RTE_ETH_RSS_RETA_SIZE_512 ETH_RSS_RETA_SIZE_512 +#define RTE_ETH_RETA_GROUP_SIZE RTE_RETA_GROUP_SIZE +#define RTE_ETH_VMDQ_MAX_VLAN_FILTERS ETH_VMDQ_MAX_VLAN_FILTERS +#define RTE_ETH_DCB_NUM_USER_PRIORITIES ETH_DCB_NUM_USER_PRIORITIES +#define RTE_ETH_VMDQ_DCB_NUM_QUEUES ETH_VMDQ_DCB_NUM_QUEUES +#define RTE_ETH_DCB_NUM_QUEUES ETH_DCB_NUM_QUEUES +#define RTE_ETH_DCB_PG_SUPPORT ETH_DCB_PG_SUPPORT +#define RTE_ETH_DCB_PFC_SUPPORT ETH_DCB_PFC_SUPPORT +#define RTE_ETH_VLAN_STRIP_OFFLOAD ETH_VLAN_STRIP_OFFLOAD +#define RTE_ETH_VLAN_FILTER_OFFLOAD ETH_VLAN_FILTER_OFFLOAD +#define RTE_ETH_VLAN_EXTEND_OFFLOAD ETH_VLAN_EXTEND_OFFLOAD +#define RTE_ETH_VLAN_STRIP_MASK ETH_VLAN_STRIP_MASK +#define RTE_ETH_VLAN_FILTER_MASK ETH_VLAN_FILTER_MASK +#define RTE_ETH_VLAN_EXTEND_MASK ETH_VLAN_EXTEND_MASK +#define RTE_ETH_VLAN_ID_MAX ETH_VLAN_ID_MAX +#define RTE_ETH_NUM_RECEIVE_MAC_ADDR ETH_NUM_RECEIVE_MAC_ADDR +#define RTE_ETH_VMDQ_NUM_UC_HASH_ARRAY ETH_VMDQ_NUM_UC_HASH_ARRAY +#define RTE_ETH_VMDQ_ACCEPT_UNTAG ETH_VMDQ_ACCEPT_UNTAG +#define RTE_ETH_VMDQ_ACCEPT_HASH_MC ETH_VMDQ_ACCEPT_HASH_MC +#define RTE_ETH_VMDQ_ACCEPT_HASH_UC ETH_VMDQ_ACCEPT_HASH_UC +#define RTE_ETH_VMDQ_ACCEPT_BROADCAST ETH_VMDQ_ACCEPT_BROADCAST +#define RTE_ETH_VMDQ_ACCEPT_MULTICAST ETH_VMDQ_ACCEPT_MULTICAST +#define RTE_ETH_4_TCS ETH_4_TCS +#define RTE_ETH_8_TCS ETH_8_TCS +#define RTE_ETH_8_POOLS ETH_8_POOLS +#define RTE_ETH_16_POOLS ETH_16_POOLS +#define RTE_ETH_32_POOLS ETH_32_POOLS +#define RTE_ETH_64_POOLS ETH_64_POOLS +#define RTE_ETH_FC_NONE RTE_FC_NONE +#define RTE_ETH_FC_RX_PAUSE RTE_FC_RX_PAUSE +#define RTE_ETH_FC_TX_PAUSE RTE_FC_TX_PAUSE +#define RTE_ETH_FC_FULL RTE_FC_FULL +#define RTE_ETH_TUNNEL_TYPE_NONE RTE_TUNNEL_TYPE_NONE +#define RTE_ETH_TUNNEL_TYPE_VXLAN RTE_TUNNEL_TYPE_VXLAN +#define RTE_ETH_TUNNEL_TYPE_GENEVE RTE_TUNNEL_TYPE_GENEVE +#define RTE_ETH_TUNNEL_TYPE_TEREDO RTE_TUNNEL_TYPE_TEREDO +#define RTE_ETH_TUNNEL_TYPE_NVGRE RTE_TUNNEL_TYPE_NVGRE +#define RTE_ETH_TUNNEL_TYPE_IP_IN_GRE RTE_TUNNEL_TYPE_IP_IN_GRE +#define RTE_ETH_L2_TUNNEL_TYPE_E_TAG RTE_L2_TUNNEL_TYPE_E_TAG +#define RTE_ETH_TUNNEL_TYPE_MAX RTE_TUNNEL_TYPE_MAX +#define RTE_ETH_FDIR_PBALLOC_64K RTE_FDIR_PBALLOC_64K +#define RTE_ETH_FDIR_PBALLOC_128K RTE_FDIR_PBALLOC_128K +#define RTE_ETH_FDIR_PBALLOC_256K RTE_FDIR_PBALLOC_256K +#define RTE_ETH_RX_OFFLOAD_VLAN_STRIP DEV_RX_OFFLOAD_VLAN_STRIP +#define RTE_ETH_RX_OFFLOAD_IPV4_CKSUM DEV_RX_OFFLOAD_IPV4_CKSUM +#define RTE_ETH_RX_OFFLOAD_UDP_CKSUM DEV_RX_OFFLOAD_UDP_CKSUM +#define RTE_ETH_RX_OFFLOAD_TCP_CKSUM DEV_RX_OFFLOAD_TCP_CKSUM +#define RTE_ETH_RX_OFFLOAD_TCP_LRO DEV_RX_OFFLOAD_TCP_LRO +#define RTE_ETH_RX_OFFLOAD_QINQ_STRIP DEV_RX_OFFLOAD_QINQ_STRIP +#define RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM +#define RTE_ETH_TX_OFFLOAD_VLAN_INSERT DEV_TX_OFFLOAD_VLAN_INSERT +#define RTE_ETH_TX_OFFLOAD_IPV4_CKSUM DEV_TX_OFFLOAD_IPV4_CKSUM +#define RTE_ETH_TX_OFFLOAD_UDP_CKSUM DEV_TX_OFFLOAD_UDP_CKSUM +#define RTE_ETH_TX_OFFLOAD_TCP_CKSUM DEV_TX_OFFLOAD_TCP_CKSUM +#define RTE_ETH_TX_OFFLOAD_SCTP_CKSUM DEV_TX_OFFLOAD_SCTP_CKSUM +#define RTE_ETH_TX_OFFLOAD_TCP_TSO DEV_TX_OFFLOAD_TCP_TSO +#define RTE_ETH_TX_OFFLOAD_UDP_TSO DEV_TX_OFFLOAD_UDP_TSO +#define RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM +#define RTE_ETH_TX_OFFLOAD_QINQ_INSERT DEV_TX_OFFLOAD_QINQ_INSERT +#define RTE_ETH_DCB_NUM_TCS ETH_DCB_NUM_TCS +#define RTE_ETH_MAX_VMDQ_POOL ETH_MAX_VMDQ_POOL +#if RTE_VERSION >= RTE_VERSION_NUM(16,7,0,0) +#define RTE_MEMPOOL_REGISTER_OPS MEMPOOL_REGISTER_OPS +#define RTE_MBUF_F_RX_VLAN_STRIPPED PKT_RX_VLAN_STRIPPED +#define RTE_MBUF_F_RX_QINQ_STRIPPED PKT_RX_QINQ_STRIPPED +#define RTE_ETH_RSS_PORT ETH_RSS_PORT +#define RTE_ETH_RSS_VXLAN ETH_RSS_VXLAN +#define RTE_ETH_RSS_GENEVE ETH_RSS_GENEVE +#define RTE_ETH_RSS_NVGRE ETH_RSS_NVGRE +#define RTE_ETH_RSS_TUNNEL ETH_RSS_TUNNEL +#define RTE_ETH_RSS_RETA_SIZE_256 ETH_RSS_RETA_SIZE_256 +#endif +#if RTE_VERSION >= RTE_VERSION_NUM(16,11,0,0) +#define RTE_MBUF_F_RX_IP_CKSUM_MASK PKT_RX_IP_CKSUM_MASK +#define RTE_MBUF_F_RX_IP_CKSUM_UNKNOWN PKT_RX_IP_CKSUM_UNKNOWN +#define RTE_MBUF_F_RX_IP_CKSUM_GOOD PKT_RX_IP_CKSUM_GOOD +#define RTE_MBUF_F_RX_IP_CKSUM_NONE PKT_RX_IP_CKSUM_NONE +#define RTE_MBUF_F_RX_L4_CKSUM_MASK PKT_RX_L4_CKSUM_MASK +#define RTE_MBUF_F_RX_L4_CKSUM_UNKNOWN PKT_RX_L4_CKSUM_UNKNOWN +#define RTE_MBUF_F_RX_L4_CKSUM_GOOD PKT_RX_L4_CKSUM_GOOD +#define RTE_MBUF_F_RX_L4_CKSUM_NONE PKT_RX_L4_CKSUM_NONE +#define RTE_MBUF_F_RX_LRO PKT_RX_LRO +#define RTE_MBUF_F_TX_TUNNEL_VXLAN PKT_TX_TUNNEL_VXLAN +#define RTE_MBUF_F_TX_TUNNEL_GRE PKT_TX_TUNNEL_GRE +#define RTE_MBUF_F_TX_TUNNEL_IPIP PKT_TX_TUNNEL_IPIP +#define RTE_MBUF_F_TX_TUNNEL_GENEVE PKT_TX_TUNNEL_GENEVE +#define RTE_MBUF_F_TX_TUNNEL_MASK PKT_TX_TUNNEL_MASK +#define RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO DEV_TX_OFFLOAD_VXLAN_TNL_TSO +#define RTE_ETH_TX_OFFLOAD_GRE_TNL_TSO DEV_TX_OFFLOAD_GRE_TNL_TSO +#define RTE_ETH_TX_OFFLOAD_IPIP_TNL_TSO DEV_TX_OFFLOAD_IPIP_TNL_TSO +#define RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO DEV_TX_OFFLOAD_GENEVE_TNL_TSO +#endif +#if RTE_VERSION >= RTE_VERSION_NUM(17,2,0,0) +#define RTE_MBUF_F_TX_MACSEC PKT_TX_MACSEC +#define RTE_MBUF_F_TX_OFFLOAD_MASK PKT_TX_OFFLOAD_MASK +#define RTE_ETH_RX_OFFLOAD_MACSEC_STRIP DEV_RX_OFFLOAD_MACSEC_STRIP +#define RTE_ETH_TX_OFFLOAD_MACSEC_INSERT DEV_TX_OFFLOAD_MACSEC_INSERT +#endif +#if RTE_VERSION >= RTE_VERSION_NUM(17,8,0,0) +#define RTE_MBUF_F_TX_TUNNEL_MPLSINUDP PKT_TX_TUNNEL_MPLSINUDP +#define RTE_ETH_TX_OFFLOAD_MT_LOCKFREE DEV_TX_OFFLOAD_MT_LOCKFREE +#endif +#if RTE_VERSION >= RTE_VERSION_NUM(17,11,0,0) +#define RTE_MBUF_F_RX_VLAN PKT_RX_VLAN +#define RTE_MBUF_F_RX_SEC_OFFLOAD PKT_RX_SEC_OFFLOAD +#define RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED PKT_RX_SEC_OFFLOAD_FAILED +#define RTE_MBUF_F_RX_QINQ PKT_RX_QINQ +#define RTE_MBUF_F_TX_SEC_OFFLOAD PKT_TX_SEC_OFFLOAD +#define RTE_ETH_RX_OFFLOAD_HEADER_SPLIT DEV_RX_OFFLOAD_HEADER_SPLIT +#define RTE_ETH_RX_OFFLOAD_VLAN_FILTER DEV_RX_OFFLOAD_VLAN_FILTER +#define RTE_ETH_RX_OFFLOAD_VLAN_EXTEND DEV_RX_OFFLOAD_VLAN_EXTEND +#define RTE_ETH_RX_OFFLOAD_SCATTER DEV_RX_OFFLOAD_SCATTER +#define RTE_ETH_RX_OFFLOAD_TIMESTAMP DEV_RX_OFFLOAD_TIMESTAMP +#define RTE_ETH_RX_OFFLOAD_SECURITY DEV_RX_OFFLOAD_SECURITY +#define RTE_ETH_RX_OFFLOAD_CHECKSUM DEV_RX_OFFLOAD_CHECKSUM +#define RTE_ETH_RX_OFFLOAD_VLAN DEV_RX_OFFLOAD_VLAN +#define RTE_ETH_TX_OFFLOAD_MULTI_SEGS DEV_TX_OFFLOAD_MULTI_SEGS +#define RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE DEV_TX_OFFLOAD_MBUF_FAST_FREE +#define RTE_ETH_TX_OFFLOAD_SECURITY DEV_TX_OFFLOAD_SECURITY +#endif +#if RTE_VERSION >= RTE_VERSION_NUM(18,2,0,0) +#define RTE_MBUF_F_TX_UDP_SEG PKT_TX_UDP_SEG +#endif +#if RTE_VERSION >= RTE_VERSION_NUM(18,5,0,0) +#define RTE_MBUF_F_TX_TUNNEL_VXLAN_GPE PKT_TX_TUNNEL_VXLAN_GPE +#define RTE_MBUF_F_TX_TUNNEL_IP PKT_TX_TUNNEL_IP +#define RTE_MBUF_F_TX_TUNNEL_UDP PKT_TX_TUNNEL_UDP +#define RTE_MBUF_F_EXTERNAL EXT_ATTACHED_MBUF +#define RTE_ETH_TX_OFFLOAD_UDP_TNL_TSO DEV_TX_OFFLOAD_UDP_TNL_TSO +#define RTE_ETH_TX_OFFLOAD_IP_TNL_TSO DEV_TX_OFFLOAD_IP_TNL_TSO +#endif +#if RTE_VERSION >= RTE_VERSION_NUM(18,11,0,0) +#define RTE_MBUF_F_RX_OUTER_L4_CKSUM_MASK PKT_RX_OUTER_L4_CKSUM_MASK +#define RTE_MBUF_F_RX_OUTER_L4_CKSUM_UNKNOWN PKT_RX_OUTER_L4_CKSUM_UNKNOWN +#define RTE_MBUF_F_RX_OUTER_L4_CKSUM_BAD PKT_RX_OUTER_L4_CKSUM_BAD +#define RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD PKT_RX_OUTER_L4_CKSUM_GOOD +#define RTE_MBUF_F_RX_OUTER_L4_CKSUM_INVALID PKT_RX_OUTER_L4_CKSUM_INVALID +#define RTE_MBUF_F_TX_OUTER_UDP_CKSUM PKT_TX_OUTER_UDP_CKSUM +#define RTE_ETH_RX_OFFLOAD_SCTP_CKSUM DEV_RX_OFFLOAD_SCTP_CKSUM +#define RTE_ETH_RX_OFFLOAD_OUTER_UDP_CKSUM DEV_RX_OFFLOAD_OUTER_UDP_CKSUM +#define RTE_ETH_TX_OFFLOAD_OUTER_UDP_CKSUM DEV_TX_OFFLOAD_OUTER_UDP_CKSUM +#endif +#if RTE_VERSION >= RTE_VERSION_NUM(19,5,0,0) +#define RTE_ETH_TUNNEL_TYPE_VXLAN_GPE RTE_TUNNEL_TYPE_VXLAN_GPE +#endif +#if RTE_VERSION >= RTE_VERSION_NUM(19,8,0,0) +#define RTE_ETH_QINQ_STRIP_OFFLOAD ETH_QINQ_STRIP_OFFLOAD +#define RTE_ETH_QINQ_STRIP_MASK ETH_QINQ_STRIP_MASK +#endif +#if RTE_VERSION >= RTE_VERSION_NUM(19,11,0,0) +#define RTE_MBUF_DYNFLAG_RX_METADATA PKT_RX_DYNF_METADATA +#define RTE_MBUF_DYNFLAG_TX_METADATA PKT_TX_DYNF_METADATA +#define RTE_MBUF_F_FIRST_FREE PKT_FIRST_FREE +#define RTE_MBUF_F_LAST_FREE PKT_LAST_FREE +#define RTE_MBUF_F_TX_TUNNEL_GTP PKT_TX_TUNNEL_GTP +#define RTE_ETH_RSS_GTPU ETH_RSS_GTPU +#define RTE_ETH_RSS_L3_SRC_ONLY ETH_RSS_L3_SRC_ONLY +#define RTE_ETH_RSS_L3_DST_ONLY ETH_RSS_L3_DST_ONLY +#define RTE_ETH_RSS_L4_SRC_ONLY ETH_RSS_L4_SRC_ONLY +#define RTE_ETH_RSS_L4_DST_ONLY ETH_RSS_L4_DST_ONLY +#define RTE_ETH_RX_OFFLOAD_RSS_HASH DEV_RX_OFFLOAD_RSS_HASH +#endif +#if RTE_VERSION >= RTE_VERSION_NUM(20,5,0,0) +#define RTE_ETH_LINK_SPEED_200G ETH_LINK_SPEED_200G +#define RTE_ETH_SPEED_NUM_200G ETH_SPEED_NUM_200G +#define RTE_ETH_RSS_ETH ETH_RSS_ETH +#define RTE_ETH_RSS_S_VLAN ETH_RSS_S_VLAN +#define RTE_ETH_RSS_C_VLAN ETH_RSS_C_VLAN +#define RTE_ETH_RSS_ESP ETH_RSS_ESP +#define RTE_ETH_RSS_AH ETH_RSS_AH +#define RTE_ETH_RSS_L2TPV3 ETH_RSS_L2TPV3 +#define RTE_ETH_RSS_PFCP ETH_RSS_PFCP +#define RTE_ETH_RSS_L2_SRC_ONLY ETH_RSS_L2_SRC_ONLY +#define RTE_ETH_RSS_L2_DST_ONLY ETH_RSS_L2_DST_ONLY +#define RTE_ETH_RSS_VLAN ETH_RSS_VLAN +#endif +#if RTE_VERSION >= RTE_VERSION_NUM(20,8,0,0) +#define RTE_ETH_RSS_PPPOE ETH_RSS_PPPOE +#define RTE_ETH_RSS_IPV6_PRE32 ETH_RSS_IPV6_PRE32 +#define RTE_ETH_RSS_IPV6_PRE40 ETH_RSS_IPV6_PRE40 +#define RTE_ETH_RSS_IPV6_PRE48 ETH_RSS_IPV6_PRE48 +#define RTE_ETH_RSS_IPV6_PRE56 ETH_RSS_IPV6_PRE56 +#define RTE_ETH_RSS_IPV6_PRE64 ETH_RSS_IPV6_PRE64 +#define RTE_ETH_RSS_IPV6_PRE96 ETH_RSS_IPV6_PRE96 +#define RTE_ETH_RSS_IPV6_PRE32_UDP ETH_RSS_IPV6_PRE32_UDP +#define RTE_ETH_RSS_IPV6_PRE40_UDP ETH_RSS_IPV6_PRE40_UDP +#define RTE_ETH_RSS_IPV6_PRE48_UDP ETH_RSS_IPV6_PRE48_UDP +#define RTE_ETH_RSS_IPV6_PRE56_UDP ETH_RSS_IPV6_PRE56_UDP +#define RTE_ETH_RSS_IPV6_PRE64_UDP ETH_RSS_IPV6_PRE64_UDP +#define RTE_ETH_RSS_IPV6_PRE96_UDP ETH_RSS_IPV6_PRE96_UDP +#define RTE_ETH_RSS_IPV6_PRE32_TCP ETH_RSS_IPV6_PRE32_TCP +#define RTE_ETH_RSS_IPV6_PRE40_TCP ETH_RSS_IPV6_PRE40_TCP +#define RTE_ETH_RSS_IPV6_PRE48_TCP ETH_RSS_IPV6_PRE48_TCP +#define RTE_ETH_RSS_IPV6_PRE56_TCP ETH_RSS_IPV6_PRE56_TCP +#define RTE_ETH_RSS_IPV6_PRE64_TCP ETH_RSS_IPV6_PRE64_TCP +#define RTE_ETH_RSS_IPV6_PRE96_TCP ETH_RSS_IPV6_PRE96_TCP +#define RTE_ETH_RSS_IPV6_PRE32_SCTP ETH_RSS_IPV6_PRE32_SCTP +#define RTE_ETH_RSS_IPV6_PRE40_SCTP ETH_RSS_IPV6_PRE40_SCTP +#define RTE_ETH_RSS_IPV6_PRE48_SCTP ETH_RSS_IPV6_PRE48_SCTP +#define RTE_ETH_RSS_IPV6_PRE56_SCTP ETH_RSS_IPV6_PRE56_SCTP +#define RTE_ETH_RSS_IPV6_PRE64_SCTP ETH_RSS_IPV6_PRE64_SCTP +#define RTE_ETH_RSS_IPV6_PRE96_SCTP ETH_RSS_IPV6_PRE96_SCTP +#define RTE_ETH_TX_OFFLOAD_SEND_ON_TIMESTAMP DEV_TX_OFFLOAD_SEND_ON_TIMESTAMP +#endif +#if RTE_VERSION >= RTE_VERSION_NUM(20,11,0,0) +#define RTE_ETH_SPEED_NUM_UNKNOWN ETH_SPEED_NUM_UNKNOWN +#define RTE_ETH_RSS_ECPRI ETH_RSS_ECPRI +#define RTE_ETH_RSS_LEVEL_PMD_DEFAULT ETH_RSS_LEVEL_PMD_DEFAULT +#define RTE_ETH_RSS_LEVEL_OUTERMOST ETH_RSS_LEVEL_OUTERMOST +#define RTE_ETH_RSS_LEVEL_INNERMOST ETH_RSS_LEVEL_INNERMOST +#define RTE_ETH_RSS_LEVEL_MASK ETH_RSS_LEVEL_MASK +#define RTE_ETH_RSS_LEVEL ETH_RSS_LEVEL +#endif +#if RTE_VERSION >= RTE_VERSION_NUM(21,2,0,0) +#define RTE_ETH_RSS_MPLS ETH_RSS_MPLS +#define RTE_ETH_TUNNEL_TYPE_ECPRI RTE_TUNNEL_TYPE_ECPRI +#endif + +#ifndef DEV_RX_OFFLOAD_JUMBO_FRAME +#define RTE_ETH_RX_OFFLOAD_JUMBO_FRAME 0x00000800 +#else +#define RTE_ETH_RX_OFFLOAD_JUMBO_FRAME DEV_RX_OFFLOAD_JUMBO_FRAME +#endif + +#ifndef DEV_RX_OFFLOAD_KEEP_CRC +#ifndef DEV_RX_OFFLOAD_CRC_STRIP +#define RTE_ETH_RX_OFFLOAD_CRC_STRIP 0x00001000 +#else +#define RTE_ETH_RX_OFFLOAD_CRC_STRIP DEV_RX_OFFLOAD_CRC_STRIP +#endif +#define RTE_ETH_RX_OFFLOAD_KEEP_CRC _force_error_if_defined_ +#undef RTE_ETH_RX_OFFLOAD_KEEP_CRC + +#else +#ifndef DEV_RX_OFFLOAD_CRC_STRIP +#define RTE_ETH_RX_OFFLOAD_CRC_STRIP _force_error_if_defined_ +#undef RTE_ETH_RX_OFFLOAD_CRC_STRIP +#else +#define RTE_ETH_RX_OFFLOAD_CRC_STRIP DEV_RX_OFFLOAD_CRC_STRIP +#endif +#define RTE_ETH_RX_OFFLOAD_KEEP_CRC DEV_RX_OFFLOAD_KEEP_CRC +#endif + +#else // >= 21.11 +#define RTE_ETH_RX_OFFLOAD_JUMBO_FRAME RTE_BIT64(11) +#define RTE_ETH_RX_OFFLOAD_CRC_STRIP _force_error_if_defined_ +#undef RTE_ETH_RX_OFFLOAD_CRC_STRIP +#endif + #endif // _PROX_COMPAT_H diff --git a/VNFs/DPPD-PROX/prox_globals.h b/VNFs/DPPD-PROX/prox_globals.h index b09f3a52..7463ded5 100644 --- a/VNFs/DPPD-PROX/prox_globals.h +++ b/VNFs/DPPD-PROX/prox_globals.h @@ -18,6 +18,7 @@ #define MAX_TASKS_PER_CORE 8 #define MAX_SOCKETS 64 #define MAX_NAME_SIZE 64 +#define MAX_NAME_BUFFER_SIZE 128 #define MAX_PROTOCOLS 3 #define MAX_RINGS_PER_TASK (MAX_WT_PER_LB*MAX_PROTOCOLS) #define MAX_WT_PER_LB 64 diff --git a/VNFs/DPPD-PROX/prox_ipv6.c b/VNFs/DPPD-PROX/prox_ipv6.c index 90538230..f8ec147f 100644 --- a/VNFs/DPPD-PROX/prox_ipv6.c +++ b/VNFs/DPPD-PROX/prox_ipv6.c @@ -141,7 +141,7 @@ void build_router_advertisement(struct rte_mbuf *mbuf, prox_rte_ether_addr *s_ad { prox_rte_ether_hdr *peth = rte_pktmbuf_mtod(mbuf, prox_rte_ether_hdr *); init_mbuf_seg(mbuf); - mbuf->ol_flags &= ~(PKT_TX_IP_CKSUM|PKT_TX_UDP_CKSUM); // Software calculates the checksum + mbuf->ol_flags &= ~(RTE_MBUF_F_TX_IP_CKSUM|RTE_MBUF_F_TX_UDP_CKSUM); // Software calculates the checksum memcpy(peth->d_addr.addr_bytes, &prox_cfg.all_nodes_mac_addr, sizeof(prox_rte_ether_addr)); memcpy(peth->s_addr.addr_bytes, s_addr, sizeof(prox_rte_ether_addr)); @@ -191,7 +191,7 @@ void build_router_sollicitation(struct rte_mbuf *mbuf, prox_rte_ether_addr *s_ad prox_rte_ether_hdr *peth = rte_pktmbuf_mtod(mbuf, prox_rte_ether_hdr *); init_mbuf_seg(mbuf); - mbuf->ol_flags &= ~(PKT_TX_IP_CKSUM|PKT_TX_UDP_CKSUM); // Software calculates the checksum + mbuf->ol_flags &= ~(RTE_MBUF_F_TX_IP_CKSUM|RTE_MBUF_F_TX_UDP_CKSUM); // Software calculates the checksum memcpy(peth->d_addr.addr_bytes, &prox_cfg.all_routers_mac_addr, sizeof(prox_rte_ether_addr)); memcpy(peth->s_addr.addr_bytes, s_addr, sizeof(prox_rte_ether_addr)); @@ -225,7 +225,7 @@ void build_neighbour_sollicitation(struct rte_mbuf *mbuf, prox_rte_ether_addr *s set_mcast_mac_from_ipv6(&mac_dst, dst); init_mbuf_seg(mbuf); - mbuf->ol_flags &= ~(PKT_TX_IP_CKSUM|PKT_TX_UDP_CKSUM); // Software calculates the checksum + mbuf->ol_flags &= ~(RTE_MBUF_F_TX_IP_CKSUM|RTE_MBUF_F_TX_UDP_CKSUM); // Software calculates the checksum memcpy(peth->d_addr.addr_bytes, &mac_dst, sizeof(prox_rte_ether_addr)); memcpy(peth->s_addr.addr_bytes, s_addr, sizeof(prox_rte_ether_addr)); @@ -263,7 +263,7 @@ void build_neighbour_advertisement(struct task_base *tbase, struct rte_mbuf *mbu uint8_t port_id = get_port(mbuf); init_mbuf_seg(mbuf); - mbuf->ol_flags &= ~(PKT_TX_IP_CKSUM|PKT_TX_UDP_CKSUM); // Software calculates the checksum + mbuf->ol_flags &= ~(RTE_MBUF_F_TX_IP_CKSUM|RTE_MBUF_F_TX_UDP_CKSUM); // Software calculates the checksum prox_rte_ipv6_hdr *ipv6_hdr = prox_set_vlan_ipv6(peth, vlan); diff --git a/VNFs/DPPD-PROX/prox_port_cfg.c b/VNFs/DPPD-PROX/prox_port_cfg.c index b93198df..3b7f778d 100644 --- a/VNFs/DPPD-PROX/prox_port_cfg.c +++ b/VNFs/DPPD-PROX/prox_port_cfg.c @@ -227,10 +227,10 @@ void init_rte_dev(int use_dummy_devices) } if (port_cfg->vdev[0]) { - char name[MAX_NAME_SIZE], tap[MAX_NAME_SIZE]; + char name[MAX_NAME_BUFFER_SIZE], tap[MAX_NAME_SIZE]; snprintf(tap, MAX_NAME_SIZE, "net_tap%d", port_id); #if (RTE_VERSION > RTE_VERSION_NUM(17,5,0,1)) - snprintf(name, MAX_NAME_SIZE, "iface=%s", port_cfg->vdev); + snprintf(name, MAX_NAME_BUFFER_SIZE, "iface=%s", port_cfg->vdev); rc = rte_vdev_init(tap, name); #else PROX_PANIC(1, "vdev not supported in DPDK < 17.05\n"); @@ -248,7 +248,7 @@ void init_rte_dev(int use_dummy_devices) for (uint32_t tag_id = 0; tag_id < prox_port_cfg[port_id].n_vlans; tag_id++) { prox_port_cfg[vdev_port_id].vlan_tags[tag_id] = prox_port_cfg[port_id].vlan_tags[tag_id]; char command[1024]; - snprintf(prox_port_cfg[vdev_port_id].names[tag_id], MAX_NAME_SIZE, "%s_%d", port_cfg->vdev, prox_port_cfg[port_id].vlan_tags[tag_id]); + snprintf(prox_port_cfg[vdev_port_id].names[tag_id], MAX_NAME_BUFFER_SIZE, "%s_%d", port_cfg->vdev, prox_port_cfg[port_id].vlan_tags[tag_id]); sprintf(command, "ip link add link %s name %s type vlan id %d", port_cfg->vdev, prox_port_cfg[vdev_port_id].names[tag_id], prox_port_cfg[port_id].vlan_tags[tag_id]); system(command); plog_info("\tRunning %s\n", command); @@ -385,6 +385,11 @@ void init_rte_dev(int use_dummy_devices) if ((ptr = strstr(port_cfg->short_name, "_pmd")) != NULL) { *ptr = '\x0'; } + // Set socket for vdev device identical to socket of corresponding port + if (prox_port_cfg[port_id].is_vdev) { + prox_port_cfg[port_id].socket = prox_port_cfg[prox_port_cfg[port_id].dpdk_mapping].socket; + continue; + } #if RTE_VERSION < RTE_VERSION_NUM(18,5,0,0) pci_dev = dev_info.pci_dev; @@ -414,20 +419,20 @@ void init_rte_dev(int use_dummy_devices) } // In DPDK 18.08 vmxnet3 reports it supports IPV4 checksum, but packets does not go through when IPv4 cksum is enabled - if ((!strcmp(port_cfg->short_name, "vmxnet3")) && (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_IPV4_CKSUM)) { + if ((!strcmp(port_cfg->short_name, "vmxnet3")) && (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_IPV4_CKSUM)) { plog_info("\t\tDisabling IPV4 cksum on vmxnet3\n"); - port_cfg->disabled_tx_offload |= DEV_TX_OFFLOAD_IPV4_CKSUM; + port_cfg->disabled_tx_offload |= RTE_ETH_TX_OFFLOAD_IPV4_CKSUM; } - if ((!strcmp(port_cfg->short_name, "vmxnet3")) && (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_UDP_CKSUM)) { + if ((!strcmp(port_cfg->short_name, "vmxnet3")) && (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_UDP_CKSUM)) { plog_info("\t\tDisabling UDP cksum on vmxnet3\n"); - port_cfg->disabled_tx_offload |= DEV_TX_OFFLOAD_UDP_CKSUM; + port_cfg->disabled_tx_offload |= RTE_ETH_TX_OFFLOAD_UDP_CKSUM; } // Some OVS versions reports that they support UDP offload and no IPv4 offload, but fails when UDP offload is enabled if ((!strcmp(port_cfg->short_name, "virtio")) && - ((port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_IPV4_CKSUM) == 0) && - (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_UDP_CKSUM)) { + ((port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_IPV4_CKSUM) == 0) && + (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_UDP_CKSUM)) { plog_info("\t\tDisabling UDP cksum on virtio\n"); - port_cfg->disabled_tx_offload |= DEV_TX_OFFLOAD_UDP_CKSUM; + port_cfg->disabled_tx_offload |= RTE_ETH_TX_OFFLOAD_UDP_CKSUM; } } } @@ -481,84 +486,86 @@ static void print_port_capa(struct prox_port_cfg *port_cfg) #if RTE_VERSION >= RTE_VERSION_NUM(18,8,0,1) plog_info("\t\tRX offload capa = 0x%lx = ", port_cfg->dev_info.rx_offload_capa); - if (port_cfg->dev_info.rx_offload_capa & DEV_RX_OFFLOAD_VLAN_STRIP) + if (port_cfg->dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_VLAN_STRIP) plog_info("VLAN STRIP | "); - if (port_cfg->dev_info.rx_offload_capa & DEV_RX_OFFLOAD_IPV4_CKSUM) + if (port_cfg->dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_IPV4_CKSUM) plog_info("IPV4 CKSUM | "); - if (port_cfg->dev_info.rx_offload_capa & DEV_RX_OFFLOAD_UDP_CKSUM) + if (port_cfg->dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_UDP_CKSUM) plog_info("UDP CKSUM | "); - if (port_cfg->dev_info.rx_offload_capa & DEV_RX_OFFLOAD_TCP_CKSUM) + if (port_cfg->dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_TCP_CKSUM) plog_info("TCP CKSUM | "); - if (port_cfg->dev_info.rx_offload_capa & DEV_RX_OFFLOAD_TCP_LRO) + if (port_cfg->dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_TCP_LRO) plog_info("TCP LRO | "); - if (port_cfg->dev_info.rx_offload_capa & DEV_RX_OFFLOAD_QINQ_STRIP) + if (port_cfg->dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_QINQ_STRIP) plog_info("QINQ STRIP | "); - if (port_cfg->dev_info.rx_offload_capa & DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM) + if (port_cfg->dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_OUTER_IPV4_CKSUM) plog_info("OUTER_IPV4_CKSUM | "); - if (port_cfg->dev_info.rx_offload_capa & DEV_RX_OFFLOAD_MACSEC_STRIP) + if (port_cfg->dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_MACSEC_STRIP) plog_info("MACSEC STRIP | "); - if (port_cfg->dev_info.rx_offload_capa & DEV_RX_OFFLOAD_HEADER_SPLIT) +#if defined(RTE_ETH_RX_OFFLOAD_HEADER_SPLIT) + if (port_cfg->dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_HEADER_SPLIT) plog_info("HEADER SPLIT | "); - if (port_cfg->dev_info.rx_offload_capa & DEV_RX_OFFLOAD_VLAN_FILTER) +#endif + if (port_cfg->dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_VLAN_FILTER) plog_info("VLAN FILTER | "); - if (port_cfg->dev_info.rx_offload_capa & DEV_RX_OFFLOAD_VLAN_EXTEND) + if (port_cfg->dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_VLAN_EXTEND) plog_info("VLAN EXTEND | "); - if (port_cfg->dev_info.rx_offload_capa & DEV_RX_OFFLOAD_JUMBO_FRAME) + if (port_cfg->dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_JUMBO_FRAME) plog_info("JUMBO FRAME | "); -#if defined(DEV_RX_OFFLOAD_CRC_STRIP) - if (port_cfg->dev_info.rx_offload_capa & DEV_RX_OFFLOAD_CRC_STRIP) +#if defined(RTE_ETH_RX_OFFLOAD_CRC_STRIP) + if (port_cfg->dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_CRC_STRIP) plog_info("CRC STRIP | "); #endif -#if defined(DEV_RX_OFFLOAD_KEEP_CRC) - if (port_cfg->dev_info.rx_offload_capa & DEV_RX_OFFLOAD_KEEP_CRC) +#if defined(RTE_ETH_RX_OFFLOAD_KEEP_CRC) + if (port_cfg->dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_KEEP_CRC) plog_info("KEEP CRC | "); #endif - if (port_cfg->dev_info.rx_offload_capa & DEV_RX_OFFLOAD_SCATTER) + if (port_cfg->dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_SCATTER) plog_info("SCATTER | "); - if (port_cfg->dev_info.rx_offload_capa & DEV_RX_OFFLOAD_TIMESTAMP) + if (port_cfg->dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_TIMESTAMP) plog_info("TIMESTAMP | "); - if (port_cfg->dev_info.rx_offload_capa & DEV_RX_OFFLOAD_SECURITY) + if (port_cfg->dev_info.rx_offload_capa & RTE_ETH_RX_OFFLOAD_SECURITY) plog_info("SECURITY "); plog_info("\n"); plog_info("\t\tTX offload capa = 0x%lx = ", port_cfg->dev_info.tx_offload_capa); - if (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_VLAN_INSERT) + if (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_VLAN_INSERT) plog_info("VLAN INSERT | "); - if (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_IPV4_CKSUM) + if (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_IPV4_CKSUM) plog_info("IPV4 CKSUM | "); - if (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_UDP_CKSUM) + if (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_UDP_CKSUM) plog_info("UDP CKSUM | "); - if (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_TCP_CKSUM) + if (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_TCP_CKSUM) plog_info("TCP CKSUM | "); - if (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_SCTP_CKSUM) + if (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_SCTP_CKSUM) plog_info("SCTP CKSUM | "); - if (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_TCP_TSO) + if (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_TCP_TSO) plog_info("TCP TS0 | "); - if (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_UDP_TSO) + if (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_UDP_TSO) plog_info("UDP TSO | "); - if (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM) + if (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_OUTER_IPV4_CKSUM) plog_info("OUTER IPV4 CKSUM | "); - if (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_QINQ_INSERT) + if (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_QINQ_INSERT) plog_info("QINQ INSERT | "); - if (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_VXLAN_TNL_TSO) + if (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_VXLAN_TNL_TSO) plog_info("VLAN TNL TSO | "); - if (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_GRE_TNL_TSO) + if (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_GRE_TNL_TSO) plog_info("GRE TNL TSO | "); - if (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_IPIP_TNL_TSO) + if (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_IPIP_TNL_TSO) plog_info("IPIP TNL TSO | "); - if (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_GENEVE_TNL_TSO) + if (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_GENEVE_TNL_TSO) plog_info("GENEVE TNL TSO | "); - if (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MACSEC_INSERT) + if (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_MACSEC_INSERT) plog_info("MACSEC INSERT | "); - if (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MT_LOCKFREE) + if (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_MT_LOCKFREE) plog_info("MT LOCKFREE | "); - if (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_MULTI_SEGS) + if (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_MULTI_SEGS) plog_info("MULTI SEG | "); - if (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_SECURITY) + if (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_SECURITY) plog_info("SECURITY | "); - if (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_UDP_TNL_TSO) + if (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_UDP_TNL_TSO) plog_info("UDP TNL TSO | "); - if (port_cfg->dev_info.tx_offload_capa & DEV_TX_OFFLOAD_IP_TNL_TSO) + if (port_cfg->dev_info.tx_offload_capa & RTE_ETH_TX_OFFLOAD_IP_TNL_TSO) plog_info("IP TNL TSO | "); plog_info("\n"); @@ -583,30 +590,30 @@ static void get_max_link_speed(struct prox_port_cfg *port_cfg) // or rte_eth_link_get_nowait) might be reported too late // and might result in wrong exrapolation, and hence should not be used // for extrapolation purposes - if (port_cfg->dev_info.speed_capa & ETH_LINK_SPEED_100G) - port_cfg->max_link_speed = ETH_SPEED_NUM_100G; - else if (port_cfg->dev_info.speed_capa & ETH_LINK_SPEED_56G) - port_cfg->max_link_speed = ETH_SPEED_NUM_56G; - else if (port_cfg->dev_info.speed_capa & ETH_LINK_SPEED_50G) - port_cfg->max_link_speed = ETH_SPEED_NUM_50G; - else if (port_cfg->dev_info.speed_capa & ETH_LINK_SPEED_40G) - port_cfg->max_link_speed = ETH_SPEED_NUM_40G; - else if (port_cfg->dev_info.speed_capa & ETH_LINK_SPEED_25G) - port_cfg->max_link_speed = ETH_SPEED_NUM_25G; - else if (port_cfg->dev_info.speed_capa & ETH_LINK_SPEED_20G) - port_cfg->max_link_speed = ETH_SPEED_NUM_20G; - else if (port_cfg->dev_info.speed_capa & ETH_LINK_SPEED_10G) - port_cfg->max_link_speed = ETH_SPEED_NUM_10G; - else if (port_cfg->dev_info.speed_capa & ETH_LINK_SPEED_5G) - port_cfg->max_link_speed = ETH_SPEED_NUM_5G; - else if (port_cfg->dev_info.speed_capa & ETH_LINK_SPEED_2_5G) - port_cfg->max_link_speed = ETH_SPEED_NUM_2_5G; - else if (port_cfg->dev_info.speed_capa & ETH_LINK_SPEED_1G) - port_cfg->max_link_speed = ETH_SPEED_NUM_1G; - else if (port_cfg->dev_info.speed_capa & (ETH_LINK_SPEED_100M_HD | ETH_LINK_SPEED_100M)) - port_cfg->max_link_speed = ETH_SPEED_NUM_100M; - else if (port_cfg->dev_info.speed_capa & (ETH_LINK_SPEED_10M_HD | ETH_LINK_SPEED_10M)) - port_cfg->max_link_speed = ETH_SPEED_NUM_10M; + if (port_cfg->dev_info.speed_capa & RTE_ETH_LINK_SPEED_100G) + port_cfg->max_link_speed = RTE_ETH_SPEED_NUM_100G; + else if (port_cfg->dev_info.speed_capa & RTE_ETH_LINK_SPEED_56G) + port_cfg->max_link_speed = RTE_ETH_SPEED_NUM_56G; + else if (port_cfg->dev_info.speed_capa & RTE_ETH_LINK_SPEED_50G) + port_cfg->max_link_speed = RTE_ETH_SPEED_NUM_50G; + else if (port_cfg->dev_info.speed_capa & RTE_ETH_LINK_SPEED_40G) + port_cfg->max_link_speed = RTE_ETH_SPEED_NUM_40G; + else if (port_cfg->dev_info.speed_capa & RTE_ETH_LINK_SPEED_25G) + port_cfg->max_link_speed = RTE_ETH_SPEED_NUM_25G; + else if (port_cfg->dev_info.speed_capa & RTE_ETH_LINK_SPEED_20G) + port_cfg->max_link_speed = RTE_ETH_SPEED_NUM_20G; + else if (port_cfg->dev_info.speed_capa & RTE_ETH_LINK_SPEED_10G) + port_cfg->max_link_speed = RTE_ETH_SPEED_NUM_10G; + else if (port_cfg->dev_info.speed_capa & RTE_ETH_LINK_SPEED_5G) + port_cfg->max_link_speed = RTE_ETH_SPEED_NUM_5G; + else if (port_cfg->dev_info.speed_capa & RTE_ETH_LINK_SPEED_2_5G) + port_cfg->max_link_speed = RTE_ETH_SPEED_NUM_2_5G; + else if (port_cfg->dev_info.speed_capa & RTE_ETH_LINK_SPEED_1G) + port_cfg->max_link_speed = RTE_ETH_SPEED_NUM_1G; + else if (port_cfg->dev_info.speed_capa & (RTE_ETH_LINK_SPEED_100M_HD | RTE_ETH_LINK_SPEED_100M)) + port_cfg->max_link_speed = RTE_ETH_SPEED_NUM_100M; + else if (port_cfg->dev_info.speed_capa & (RTE_ETH_LINK_SPEED_10M_HD | RTE_ETH_LINK_SPEED_10M)) + port_cfg->max_link_speed = RTE_ETH_SPEED_NUM_10M; } #endif @@ -665,13 +672,13 @@ static void init_port(struct prox_port_cfg *port_cfg) if (port_cfg->n_rxq > 1) { // Enable RSS if multiple receive queues if (strcmp(port_cfg->short_name, "virtio")) { - port_cfg->port_conf.rxmode.mq_mode |= ETH_MQ_RX_RSS; + port_cfg->port_conf.rxmode.mq_mode |= RTE_ETH_MQ_RX_RSS; port_cfg->port_conf.rx_adv_conf.rss_conf.rss_key = toeplitz_init_key; port_cfg->port_conf.rx_adv_conf.rss_conf.rss_key_len = TOEPLITZ_KEY_LEN; #if RTE_VERSION >= RTE_VERSION_NUM(2,0,0,0) - port_cfg->port_conf.rx_adv_conf.rss_conf.rss_hf = ETH_RSS_IP|ETH_RSS_UDP; + port_cfg->port_conf.rx_adv_conf.rss_conf.rss_hf = RTE_ETH_RSS_IP|RTE_ETH_RSS_UDP; #else - port_cfg->port_conf.rx_adv_conf.rss_conf.rss_hf = ETH_RSS_IPV4|ETH_RSS_NONF_IPV4_UDP; + port_cfg->port_conf.rx_adv_conf.rss_conf.rss_hf = RTE_ETH_RSS_IPV4|ETH_RSS_NONF_IPV4_UDP; #endif } } @@ -681,37 +688,37 @@ static void init_port(struct prox_port_cfg *port_cfg) port_cfg->port_conf.rx_adv_conf.rss_conf.rss_hf &= port_cfg->dev_info.flow_type_rss_offloads; #endif if (strcmp(port_cfg->short_name, "virtio")) { - plog_info("\t\t Enabling RSS rss_hf = 0x%lx (requested 0x%llx, supported 0x%lx)\n", port_cfg->port_conf.rx_adv_conf.rss_conf.rss_hf, ETH_RSS_IP|ETH_RSS_UDP, port_cfg->dev_info.flow_type_rss_offloads); + plog_info("\t\t Enabling RSS rss_hf = 0x%lx (requested 0x%llx, supported 0x%lx)\n", port_cfg->port_conf.rx_adv_conf.rss_conf.rss_hf, RTE_ETH_RSS_IP|RTE_ETH_RSS_UDP, port_cfg->dev_info.flow_type_rss_offloads); } else { plog_info("\t\t Not enabling RSS on virtio port"); } // rxmode such as hw src strip #if RTE_VERSION >= RTE_VERSION_NUM(18,8,0,1) -#if defined (DEV_RX_OFFLOAD_CRC_STRIP) - CONFIGURE_RX_OFFLOAD(DEV_RX_OFFLOAD_CRC_STRIP); +#if defined (RTE_ETH_RX_OFFLOAD_CRC_STRIP) + CONFIGURE_RX_OFFLOAD(RTE_ETH_RX_OFFLOAD_CRC_STRIP); #endif -#if defined (DEV_RX_OFFLOAD_KEEP_CRC) - CONFIGURE_RX_OFFLOAD(DEV_RX_OFFLOAD_KEEP_CRC); +#if defined (RTE_ETH_RX_OFFLOAD_KEEP_CRC) + CONFIGURE_RX_OFFLOAD(RTE_ETH_RX_OFFLOAD_KEEP_CRC); #endif - CONFIGURE_RX_OFFLOAD(DEV_RX_OFFLOAD_JUMBO_FRAME); - CONFIGURE_RX_OFFLOAD(DEV_RX_OFFLOAD_VLAN_STRIP); + CONFIGURE_RX_OFFLOAD(RTE_ETH_RX_OFFLOAD_JUMBO_FRAME); + CONFIGURE_RX_OFFLOAD(RTE_ETH_RX_OFFLOAD_VLAN_STRIP); #else - if (port_cfg->requested_rx_offload & DEV_RX_OFFLOAD_CRC_STRIP) { + if (port_cfg->requested_rx_offload & RTE_ETH_RX_OFFLOAD_CRC_STRIP) { port_cfg->port_conf.rxmode.hw_strip_crc = 1; } - if (port_cfg->requested_rx_offload & DEV_RX_OFFLOAD_JUMBO_FRAME) { + if (port_cfg->requested_rx_offload & RTE_ETH_RX_OFFLOAD_JUMBO_FRAME) { port_cfg->port_conf.rxmode.jumbo_frame = 1; } #endif // IPV4, UDP, SCTP Checksums #if RTE_VERSION >= RTE_VERSION_NUM(18,8,0,1) - CONFIGURE_TX_OFFLOAD(DEV_TX_OFFLOAD_IPV4_CKSUM); - CONFIGURE_TX_OFFLOAD(DEV_TX_OFFLOAD_UDP_CKSUM); - CONFIGURE_TX_OFFLOAD(DEV_TX_OFFLOAD_VLAN_INSERT); + CONFIGURE_TX_OFFLOAD(RTE_ETH_TX_OFFLOAD_IPV4_CKSUM); + CONFIGURE_TX_OFFLOAD(RTE_ETH_TX_OFFLOAD_UDP_CKSUM); + CONFIGURE_TX_OFFLOAD(RTE_ETH_TX_OFFLOAD_VLAN_INSERT); #else - if ((port_cfg->dev_info.tx_offload_capa & (DEV_TX_OFFLOAD_IPV4_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM)) == 0) { + if ((port_cfg->dev_info.tx_offload_capa & (RTE_ETH_TX_OFFLOAD_IPV4_CKSUM | RTE_ETH_TX_OFFLOAD_UDP_CKSUM)) == 0) { port_cfg->tx_conf.txq_flags |= ETH_TXQ_FLAGS_NOOFFLOADS; plog_info("\t\tDisabling TX offloads as pmd reports that it does not support them)\n"); } @@ -722,7 +729,7 @@ static void init_port(struct prox_port_cfg *port_cfg) #endif // Multi Segments #if RTE_VERSION >= RTE_VERSION_NUM(18,8,0,1) - CONFIGURE_TX_OFFLOAD(DEV_TX_OFFLOAD_MULTI_SEGS); + CONFIGURE_TX_OFFLOAD(RTE_ETH_TX_OFFLOAD_MULTI_SEGS); #else if (!strcmp(port_cfg->short_name, "vmxnet3")) { port_cfg->tx_conf.txq_flags |= ETH_TXQ_FLAGS_NOMULTSEGS; @@ -740,7 +747,7 @@ static void init_port(struct prox_port_cfg *port_cfg) // Refcount #if RTE_VERSION >= RTE_VERSION_NUM(18,8,0,1) - CONFIGURE_TX_OFFLOAD(DEV_TX_OFFLOAD_MBUF_FAST_FREE); + CONFIGURE_TX_OFFLOAD(RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE); #else if (port_cfg->tx_conf.txq_flags & ETH_TXQ_FLAGS_NOREFCOUNT) plog_info("\t\tEnabling No refcnt on port %d\n", port_id); @@ -848,7 +855,7 @@ static void init_port(struct prox_port_cfg *port_cfg) if (link.link_status) { plog_info("Link Up - speed %'u Mbps - %s\n", link.link_speed, - (link.link_duplex == ETH_LINK_FULL_DUPLEX) ? + (link.link_duplex == RTE_ETH_LINK_FULL_DUPLEX) ? "full-duplex" : "half-duplex"); } else { diff --git a/VNFs/DPPD-PROX/prox_port_cfg.h b/VNFs/DPPD-PROX/prox_port_cfg.h index 24fd8aac..82d58f76 100644 --- a/VNFs/DPPD-PROX/prox_port_cfg.h +++ b/VNFs/DPPD-PROX/prox_port_cfg.h @@ -21,9 +21,13 @@ #include <rte_ether.h> #include <rte_ethdev.h> #include <rte_version.h> +#if RTE_VERSION >= RTE_VERSION_NUM(22,11,0,0) +#include <bus_pci_driver.h> // Please configure DPDK with meson option -Denable_driver_sdk=true +#else #if RTE_VERSION >= RTE_VERSION_NUM(17,11,0,0) #include <rte_bus_pci.h> #endif +#endif #include <rte_pci.h> #include "prox_compat.h" @@ -57,7 +61,7 @@ struct prox_port_cfg { uint32_t mtu; enum addr_type type; prox_rte_ether_addr eth_addr; /* port MAC address */ - char names[PROX_MAX_VLAN_TAGS][MAX_NAME_SIZE]; + char names[PROX_MAX_VLAN_TAGS][MAX_NAME_BUFFER_SIZE]; char vdev[MAX_NAME_SIZE]; char short_name[MAX_NAME_SIZE]; char driver_name[MAX_NAME_SIZE]; diff --git a/VNFs/DPPD-PROX/prox_shared.c b/VNFs/DPPD-PROX/prox_shared.c index 52f4eb18..de26441d 100644 --- a/VNFs/DPPD-PROX/prox_shared.c +++ b/VNFs/DPPD-PROX/prox_shared.c @@ -55,6 +55,7 @@ static void prox_sh_create_hash(struct prox_shared *ps, size_t size) { param.entries = size; param.name = get_sh_name(); + param.socket_id = rte_socket_id(); ps->hash = rte_hash_create(¶m); PROX_PANIC(ps->hash == NULL, "Failed to create hash table for shared data"); ps->size = size; diff --git a/VNFs/DPPD-PROX/rw_reg.c b/VNFs/DPPD-PROX/rw_reg.c index a0e59085..b4f6c214 100644 --- a/VNFs/DPPD-PROX/rw_reg.c +++ b/VNFs/DPPD-PROX/rw_reg.c @@ -14,6 +14,10 @@ // limitations under the License. */ +#include <rte_version.h> +#if RTE_VERSION >= RTE_VERSION_NUM(21,11,0,0) +#include <ethdev_driver.h> // Please configure DPDK with meson option -Denable_driver_sdk=true +#endif #include <rte_ethdev.h> #include "rw_reg.h" diff --git a/VNFs/DPPD-PROX/stats_irq.h b/VNFs/DPPD-PROX/stats_irq.h index 71ff80f7..9a3f6c2f 100644 --- a/VNFs/DPPD-PROX/stats_irq.h +++ b/VNFs/DPPD-PROX/stats_irq.h @@ -51,7 +51,7 @@ struct irq_task_stats { struct irq_rt_stats *stats; }; -uint64_t irq_bucket_maxtime_cycles[IRQ_BUCKETS_COUNT]; +extern uint64_t irq_bucket_maxtime_cycles[IRQ_BUCKETS_COUNT]; extern uint64_t irq_bucket_maxtime_micro[]; void stats_irq_reset(void); diff --git a/VNFs/DPPD-PROX/stats_port.c b/VNFs/DPPD-PROX/stats_port.c index 124c849e..fb6cf10a 100644 --- a/VNFs/DPPD-PROX/stats_port.c +++ b/VNFs/DPPD-PROX/stats_port.c @@ -18,6 +18,9 @@ #include <stdio.h> #include <rte_version.h> +#if RTE_VERSION >= RTE_VERSION_NUM(21,11,0,0) +#include <ethdev_driver.h> // Please configure DPDK with meson option -Denable_driver_sdk=true +#endif #include <rte_ethdev.h> #include <rte_cycles.h> #include <rte_byteorder.h> @@ -289,16 +292,16 @@ static void nic_read_stats(uint8_t port_id) dropped by the nic". Note that in case CRC is stripped on ixgbe, the CRC bytes are not counted. */ -#if defined (DEV_RX_OFFLOAD_CRC_STRIP) - if (prox_port_cfg[port_id].requested_rx_offload & DEV_RX_OFFLOAD_CRC_STRIP) +#if defined (RTE_ETH_RX_OFFLOAD_CRC_STRIP) + if (prox_port_cfg[port_id].requested_rx_offload & RTE_ETH_RX_OFFLOAD_CRC_STRIP) stats->rx_bytes = eth_stat.ibytes + (24 * eth_stat.ipackets - 20 * (eth_stat.ierrors + eth_stat.imissed)); else stats->rx_bytes = eth_stat.ibytes + (20 * eth_stat.ipackets - 20 * (eth_stat.ierrors + eth_stat.imissed)); #else -#if defined (DEV_RX_OFFLOAD_KEEP_CRC) - if (prox_port_cfg[port_id].requested_rx_offload & DEV_RX_OFFLOAD_KEEP_CRC) +#if defined (RTE_ETH_RX_OFFLOAD_KEEP_CRC) + if (prox_port_cfg[port_id].requested_rx_offload & RTE_ETH_RX_OFFLOAD_KEEP_CRC) stats->rx_bytes = eth_stat.ibytes + (20 * eth_stat.ipackets - 20 * (eth_stat.ierrors + eth_stat.imissed)); else diff --git a/VNFs/DPPD-PROX/stats_task.h b/VNFs/DPPD-PROX/stats_task.h index 7dc54eab..001ebbc7 100644 --- a/VNFs/DPPD-PROX/stats_task.h +++ b/VNFs/DPPD-PROX/stats_task.h @@ -17,6 +17,11 @@ #ifndef _STATS_TASK_H_ #define _STATS_TASK_H_ +#include <rte_common.h> +#ifndef __rte_cache_aligned +#include <rte_memory.h> +#endif + #include <inttypes.h> #include "clock.h" diff --git a/VNFs/DPPD-PROX/task_base.h b/VNFs/DPPD-PROX/task_base.h index e0c87b03..89e5bb9d 100644 --- a/VNFs/DPPD-PROX/task_base.h +++ b/VNFs/DPPD-PROX/task_base.h @@ -56,11 +56,11 @@ #define TASK_FEATURE_RX_ALL 0x8000 #define TASK_FEATURE_TXQ_FLAGS_MULTIPLE_MEMPOOL 0x20000 -#define FLAG_TX_FLUSH 0x01 -#define FLAG_NEVER_FLUSH 0x02 +#define TBASE_FLAG_TX_FLUSH 0x01 +#define TBASE_FLAG_NEVER_FLUSH 0x02 // Task specific flags -#define BASE_FLAG_LUT_QINQ_HASH 0x08 -#define BASE_FLAG_LUT_QINQ_RSS 0x10 +#define TBASE_FLAG_LUT_QINQ_HASH 0x08 +#define TBASE_FLAG_LUT_QINQ_RSS 0x10 #define OUT_DISCARD 0xFF #define OUT_HANDLED 0xFE diff --git a/VNFs/DPPD-PROX/task_init.c b/VNFs/DPPD-PROX/task_init.c index 9fc05623..97f7188c 100644 --- a/VNFs/DPPD-PROX/task_init.c +++ b/VNFs/DPPD-PROX/task_init.c @@ -302,7 +302,7 @@ static size_t init_rx_tx_rings_ports(struct task_args *targ, struct task_base *t tbase->tx_pkt = targ->nb_txrings ? tx_pkt_no_drop_never_discard_sw1 : tx_pkt_no_drop_never_discard_hw1_lat_opt; } if ((targ->nb_txrings) || ((targ->task_init->flag_features & TASK_FEATURE_THROUGHPUT_OPT) == 0)) - tbase->flags |= FLAG_NEVER_FLUSH; + tbase->flags |= TBASE_FLAG_NEVER_FLUSH; else targ->lconf->flush_queues[targ->task] = flush_function(targ); } @@ -316,7 +316,7 @@ static size_t init_rx_tx_rings_ports(struct task_args *targ, struct task_base *t else { tbase->tx_pkt = targ->nb_txrings ? tx_pkt_no_drop_sw1 : tx_pkt_no_drop_hw1; } - tbase->flags |= FLAG_NEVER_FLUSH; + tbase->flags |= TBASE_FLAG_NEVER_FLUSH; } } else { @@ -352,7 +352,7 @@ struct task_base *init_task_struct(struct task_args *targ) offset += t->size; if (targ->nb_txrings == 0 && targ->nb_txports == 0) - tbase->flags |= FLAG_NEVER_FLUSH; + tbase->flags |= TBASE_FLAG_NEVER_FLUSH; offset = init_rx_tx_rings_ports(targ, tbase, offset); tbase->aux = (struct task_base_aux *)(((uint8_t *)tbase) + offset); diff --git a/VNFs/DPPD-PROX/thread_generic.c b/VNFs/DPPD-PROX/thread_generic.c index 14fb943e..39964dea 100644 --- a/VNFs/DPPD-PROX/thread_generic.c +++ b/VNFs/DPPD-PROX/thread_generic.c @@ -213,7 +213,6 @@ int thread_generic(struct lcore_cfg *lconf) next[task_id] = t->handle_bulk(t, mbufs, nb_rx); } } - } } return 0; diff --git a/VNFs/DPPD-PROX/tx_pkt.c b/VNFs/DPPD-PROX/tx_pkt.c index 551056af..cd62cc54 100644 --- a/VNFs/DPPD-PROX/tx_pkt.c +++ b/VNFs/DPPD-PROX/tx_pkt.c @@ -286,7 +286,7 @@ void flush_queues_hw(struct task_base *tbase) } } - tbase->flags &= ~FLAG_TX_FLUSH; + tbase->flags &= ~TBASE_FLAG_TX_FLUSH; } void flush_queues_sw(struct task_base *tbase) @@ -303,7 +303,7 @@ void flush_queues_sw(struct task_base *tbase) ring_enq_drop(tbase->tx_params_sw.tx_rings[i], tbase->ws_mbuf->mbuf[i] + (cons & WS_MBUF_MASK), prod - cons, tbase); } } - tbase->flags &= ~FLAG_TX_FLUSH; + tbase->flags &= ~TBASE_FLAG_TX_FLUSH; } void flush_queues_no_drop_hw(struct task_base *tbase) @@ -321,7 +321,7 @@ void flush_queues_no_drop_hw(struct task_base *tbase) } } - tbase->flags &= ~FLAG_TX_FLUSH; + tbase->flags &= ~TBASE_FLAG_TX_FLUSH; } void flush_queues_no_drop_sw(struct task_base *tbase) @@ -338,7 +338,7 @@ void flush_queues_no_drop_sw(struct task_base *tbase) ring_enq_no_drop(tbase->tx_params_sw.tx_rings[i], tbase->ws_mbuf->mbuf[i] + (cons & WS_MBUF_MASK), prod - cons, tbase); } } - tbase->flags &= ~FLAG_TX_FLUSH; + tbase->flags &= ~TBASE_FLAG_TX_FLUSH; } /* "try" functions try to send packets to sw/hw w/o failing or blocking; @@ -427,7 +427,7 @@ int tx_pkt_no_drop_never_discard_hw1_thrpt_opt(struct task_base *tbase, struct r cons = tbase->ws_mbuf->idx[0].cons; if ((uint16_t)(prod - cons)){ - tbase->flags &= ~FLAG_TX_FLUSH; + tbase->flags &= ~TBASE_FLAG_TX_FLUSH; tbase->ws_mbuf->idx[0].prod = 0; tbase->ws_mbuf->idx[0].cons = 0; ret+= txhw_no_drop(&tbase->tx_params_hw.tx_port_queue[0], tbase->ws_mbuf->mbuf[0] + (cons & WS_MBUF_MASK), (uint16_t)(prod - cons), tbase); @@ -456,7 +456,7 @@ int tx_pkt_never_discard_hw1_thrpt_opt(struct task_base *tbase, struct rte_mbuf cons = tbase->ws_mbuf->idx[0].cons; if ((uint16_t)(prod - cons)){ - tbase->flags &= ~FLAG_TX_FLUSH; + tbase->flags &= ~TBASE_FLAG_TX_FLUSH; tbase->ws_mbuf->idx[0].prod = 0; tbase->ws_mbuf->idx[0].cons = 0; ret+= txhw_drop(&tbase->tx_params_hw.tx_port_queue[0], tbase->ws_mbuf->mbuf[0] + (cons & WS_MBUF_MASK), (uint16_t)(prod - cons), tbase); @@ -595,7 +595,7 @@ int tx_pkt_no_drop_hw(struct task_base *tbase, struct rte_mbuf **mbufs, uint16_t cons = tbase->ws_mbuf->idx[i].cons; if (((uint16_t)(prod - cons)) >= MAX_PKT_BURST) { - tbase->flags &= ~FLAG_TX_FLUSH; + tbase->flags &= ~TBASE_FLAG_TX_FLUSH; tbase->ws_mbuf->idx[i].cons = cons + MAX_PKT_BURST; ret+= txhw_no_drop(&tbase->tx_params_hw.tx_port_queue[i], tbase->ws_mbuf->mbuf[i] + (cons & WS_MBUF_MASK), MAX_PKT_BURST, tbase); } @@ -616,7 +616,7 @@ int tx_pkt_no_drop_sw(struct task_base *tbase, struct rte_mbuf **mbufs, uint16_t cons = tbase->ws_mbuf->idx[i].cons; if (((uint16_t)(prod - cons)) >= MAX_PKT_BURST) { - tbase->flags &= ~FLAG_TX_FLUSH; + tbase->flags &= ~TBASE_FLAG_TX_FLUSH; tbase->ws_mbuf->idx[i].cons = cons + MAX_PKT_BURST; ret += ring_enq_no_drop(tbase->tx_params_sw.tx_rings[i], tbase->ws_mbuf->mbuf[i] + (cons & WS_MBUF_MASK), MAX_PKT_BURST, tbase); } @@ -637,7 +637,7 @@ int tx_pkt_hw(struct task_base *tbase, struct rte_mbuf **mbufs, uint16_t n_pkts, cons = tbase->ws_mbuf->idx[i].cons; if (((uint16_t)(prod - cons)) >= MAX_PKT_BURST) { - tbase->flags &= ~FLAG_TX_FLUSH; + tbase->flags &= ~TBASE_FLAG_TX_FLUSH; tbase->ws_mbuf->idx[i].cons = cons + MAX_PKT_BURST; ret += txhw_drop(&tbase->tx_params_hw.tx_port_queue[i], tbase->ws_mbuf->mbuf[i] + (cons & WS_MBUF_MASK), MAX_PKT_BURST, tbase); } @@ -657,7 +657,7 @@ int tx_pkt_sw(struct task_base *tbase, struct rte_mbuf **mbufs, uint16_t n_pkts, cons = tbase->ws_mbuf->idx[i].cons; if (((uint16_t)(prod - cons)) >= MAX_PKT_BURST) { - tbase->flags &= ~FLAG_TX_FLUSH; + tbase->flags &= ~TBASE_FLAG_TX_FLUSH; tbase->ws_mbuf->idx[i].cons = cons + MAX_PKT_BURST; ret+= ring_enq_drop(tbase->tx_params_sw.tx_rings[i], tbase->ws_mbuf->mbuf[i] + (cons & WS_MBUF_MASK), MAX_PKT_BURST, tbase); } diff --git a/VNFs/DPPD-PROX/version.h b/VNFs/DPPD-PROX/version.h index fe5fcbfc..355a5dcb 100644 --- a/VNFs/DPPD-PROX/version.h +++ b/VNFs/DPPD-PROX/version.h @@ -19,7 +19,7 @@ /* PROGRAM_NAME defined through Makefile */ #define VERSION_MAJOR 0 // Pre-production -#define VERSION_MINOR 2005 // 20.05 i.e. May 2020 +#define VERSION_MINOR 2212 // 22.12 i.e. December 2022 #define VERSION_REV 0 static inline char *VERSION_STR(void) diff --git a/docs/conf.py b/docs/conf.py index 3c4453e7..4821d0aa 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -1 +1,4 @@ from docs_conf.conf import * +linkcheck_ignore = [ + r'https://trex-tgn.cisco.com/', + ] diff --git a/docs/release/release-notes/release-notes.rst b/docs/release/release-notes/release-notes.rst index 80f63e32..f9daae50 100644 --- a/docs/release/release-notes/release-notes.rst +++ b/docs/release/release-notes/release-notes.rst @@ -14,7 +14,7 @@ OPNFV Jerma Release OPNFV Hunter Release -====================== +==================== This Hunter release provides *SampleVNF* as a approx VNF repository for VNF/NFVI testing, characterization and OPNFV feature testing, automated on @@ -30,7 +30,7 @@ OPNFV platform, including: * Results -* Automated SampleVNF test suit in OPNFV Yardstick_ Project +* Automated SampleVNF test suit in OPNFV Yardstick Project * SampleVNF source code @@ -42,7 +42,7 @@ For Hunter release, the *SampleVNF* supported: | *CGNAPT* | Carrier Grade Network Address and port Translation .5.0 | v0.1.0 | +----------------+---------------------------------------------------------+-------------------+ | *Prox* | Packet pROcessing eXecution engine | v0.40.0 | -| | acts as traffic generator, L3FWD, L2FWD, BNG etc | | +| | acts as traffic generator, L3FWD, L2FWD, BNG etc | | +----------------+---------------------------------------------------------+-------------------+ | *vACL* | Access Control List | v0.1.0 | +----------------+---------------------------------------------------------+-------------------+ @@ -68,14 +68,16 @@ testing: * Network - rfc2544, rfc3511, latency, http_test etc -The *SampleVNF* is developed in the OPNFV community, by the SampleVNF_ team. +The *SampleVNF* is developed in the OPNFV community, by the SampleVNF team. The *Network Service Benchmarking* SampleVNF Characterization Testing tool is a part of the Yardstick Project. .. note:: The test case description template used for the SampleVNF in yardstick - test cases is based on the document ETSI-ISG NFV-TST001_; the results report template + test cases is based on the document `ETSI GS NFV-TST 001`_; the results report template used for the SampleVNF test results is based on the IEEE Std 829-2008. +.. _ETSI GS NFV-TST 001: https://portal.etsi.org/webapp/workprogram/Report_WorkItem.asp?WKI_ID=46009 + Release Data ------------ @@ -98,6 +100,7 @@ Release Data +--------------------------------------+--------------------------------------+ | **Purpose of the delivery** | Hunter alignment to Released | | | bug-fixes for the following: | +| | | | | - Memory leak | | | - minimum latency | | | - Increase default mbuf size and | @@ -266,10 +269,10 @@ Open JIRA tickets Useful links ------------ - - wiki project page: https://wiki.opnfv.org/display/SAM + - wiki project page: https://wiki-old.opnfv.org/display/SAM - wiki SampleVNF Hunter release planing page: https://wiki.opnfv.org/display/SAM/G+-+Release+SampleVNF+planning - - SampleVNF repo: https://git.opnfv.org/cgit/samplevnf + - SampleVNF repo: https://git.opnfv.org/samplevnf/ - SampleVNF IRC chanel: #opnfv-samplevnf diff --git a/docs/release/results/overview.rst b/docs/release/results/overview.rst index df04d327..5a2f2b8a 100644 --- a/docs/release/results/overview.rst +++ b/docs/release/results/overview.rst @@ -6,7 +6,7 @@ SampleVNF test tesult document overview ======================================= -.. _`SampleVNF user guide`: artifacts.opnfv.org/samplevnf/docs/userguide/index.html +.. _`SampleVNF user guide`: http://artifacts.opnfv.org/samplevnf/docs/testing_user_userguide/index.html This document provides an overview of the results of test cases developed by the OPNFV SampleVNF Project & test cases executed part of yardstick diff --git a/docs/testing/developer/design/02-Get_started_Guide.rst b/docs/testing/developer/design/02-Get_started_Guide.rst index c8f35ed3..2a9806b5 100644 --- a/docs/testing/developer/design/02-Get_started_Guide.rst +++ b/docs/testing/developer/design/02-Get_started_Guide.rst @@ -6,7 +6,7 @@ ==================================== Get started as a SampleVNF developer -=================================== +==================================== .. _SampleVNF: https://wiki.opnfv.org/samplevnf .. _Gerrit: https://www.gerritcodereview.com/ diff --git a/docs/testing/developer/design/04-SampleVNF_Design.rst b/docs/testing/developer/design/04-SampleVNF_Design.rst index a3332e27..f813a297 100644 --- a/docs/testing/developer/design/04-SampleVNF_Design.rst +++ b/docs/testing/developer/design/04-SampleVNF_Design.rst @@ -348,7 +348,7 @@ transmit takes packets from worker thread in a dedicated ring and sent to the hardware queue. Master pipeline -^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^ This component does not process any packets and should configure with Core 0, to save cores for other components which processes traffic. The component is responsible for: @@ -359,7 +359,7 @@ is responsible for: 4. ARP and ICMP are handled here. Load Balancer pipeline -^^^^^^^^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^^^^^^^^ Load balancer is part of the Multi-Threaded CGMAPT release which distributes the flows to Multiple ACL worker threads. @@ -371,7 +371,7 @@ affinity of flows to worker threads. Tuple can be modified/configured using configuration file vCGNAPT - Static ------------------- +---------------- The vCGNAPT component performs translation of private IP & port to public IP & port at egress side and public IP & port to private IP & port at Ingress side @@ -383,7 +383,7 @@ match will be taken a default action. The default action may result in drop of the packets. vCGNAPT- Dynamic ------------------ +---------------- The vCGNAPT component performs translation of private IP & port to public IP & port at egress side and public IP & port to private IP & port at Ingress side @@ -399,11 +399,13 @@ Dynamic vCGNAPT acts as static one too, we can do NAT entries statically. Static NAT entries port range must not conflict to dynamic NAT port range. vCGNAPT Static Topology ----------------------- +----------------------- -IXIA(Port 0)-->(Port 0)VNF(Port 1)-->(Port 1) IXIA +IXIA(Port 0)-->(Port 0)VNF(Port 1)-->(Port 1)IXIA operation: + Egress --> The packets sent out from ixia(port 0) will be CGNAPTed to ixia(port 1). + Igress --> The packets sent out from ixia(port 1) will be CGNAPTed to ixia(port 0). vCGNAPT Dynamic Topology (UDP_REPLAY) @@ -411,9 +413,11 @@ vCGNAPT Dynamic Topology (UDP_REPLAY) IXIA(Port 0)-->(Port 0)VNF(Port 1)-->(Port 0)UDP_REPLAY operation: + Egress --> The packets sent out from ixia will be CGNAPTed to L3FWD/L4REPLAY. + Ingress --> The L4REPLAY upon reception of packets (Private to Public Network), - will immediately replay back the traffic to IXIA interface. (Pub -->Priv). + will immediately replay back the traffic to IXIA interface. (Pub -->Priv). How to run L4Replay ------------------- @@ -431,7 +435,7 @@ vACL - Design ============= Introduction --------------- +------------ This application implements Access Control List (ACL). ACL is typically used for rule based policy enforcement. It restricts access to a destination IP address/port based on various header fields, such as source IP address/port, @@ -439,12 +443,12 @@ destination IP address/port and protocol. It is built on top of DPDK and uses the packet framework infrastructure. Scope ------- +----- This application provides a standalone DPDK based high performance ACL Virtual Network Function implementation. High Level Design ------------------- +----------------- The ACL Filter performs bulk filtering of incoming packets based on rules in current ruleset, discarding any packets not permitted by the rules. The mechanisms needed for building the rule database and performing lookups are @@ -460,12 +464,12 @@ The Input and Output FIFOs will be implemented using DPDK Ring Buffers. The DPDK ACL example: -http://dpdk.org/doc/guides/sample_app_ug/l3_forward_access_ctrl.html +http://doc.dpdk.org/guides/sample_app_ug/l3_forward.html #figure-ipv4-acl-rule contains a suitable syntax and parser for ACL rules. Components of ACL ------------------- +----------------- In ACL, each component is constructed as a packet framework. It includes Master pipeline component, driver, load balancer pipeline component and ACL worker pipeline component. A pipeline framework is a collection of input ports, @@ -607,27 +611,33 @@ Edge Router has the following functionalities in Upstream. Update the packet color in MPLS EXP field in each MPLS header. Components of vPE -------------------- +----------------- The vPE has downstream and upstream pipelines controlled by Master component. -Edge router processes two different types of traffic through pipelines -I. Downstream (Core-to-Customer) - 1. Receives TCP traffic from core - 2. Routes the packet based on the routing rules - 3. Performs traffic scheduling based on the traffic profile - a. Qos scheduling is performed using token bucket algorithm - SVLAN, CVLAN, DSCP fields are used to determine transmission priority. - 4. Appends QinQ label in each outgoing packet. -II. Upstream (Customer-to-Core) - 1. Receives QinQ labelled TCP packets from Customer - 2. Removes the QinQ label - 3. Classifies the flow using QinQ label and apply Qos metering - a. 1st stage Qos metering is performed with flow ID using trTCM algorithm - b. 2nd stage Qos metering is performed with flow ID and traffic class using - trTCM algorithm - c. traffic class maps to DSCP field in the packet. - 4. Routes the packet based on the routing rules - 5. Appends two MPLS labels in each outgoing packet. +Edge router processes two different types of traffic through pipelines: + +I) Downstream (Core-to-Customer) + + 1. Receives TCP traffic from core + 2. Routes the packet based on the routing rules + 3. Performs traffic scheduling based on the traffic profile + + a. Qos scheduling is performed using token bucket algorithm. + SVLAN, CVLAN, DSCP fields are used to determine transmission priority. + 4. Appends QinQ label in each outgoing packet. + +II) Upstream (Customer-to-Core) + + 1. Receives QinQ labelled TCP packets from Customer + 2. Removes the QinQ label + 3. Classifies the flow using QinQ label and apply Qos metering + + a. 1st stage Qos metering is performed with flow ID using trTCM algorithm + b. 2nd stage Qos metering is performed with flow ID and traffic class using + trTCM algorithm + c. traffic class maps to DSCP field in the packet. + 4. Routes the packet based on the routing rules + 5. Appends two MPLS labels in each outgoing packet. Master Component ^^^^^^^^^^^^^^^^ @@ -635,7 +645,8 @@ Master Component The Master component is part of all the IP Pipeline applications. This component does not process any packets and should configure with Core0, to save cores for other components which processes traffic. The component -is responsible for +is responsible for: + 1. Initializing each component of the Pipeline application in different threads 2. Providing CLI shell for the user 3. Propagating the commands from user to the corresponding components. @@ -656,7 +667,7 @@ To run the VNF, execute the following: Prox - Packet pROcessing eXecution engine -========================================== +========================================= Introduction ------------ diff --git a/docs/testing/developer/requirements/03-Requirements.rst b/docs/testing/developer/requirements/03-Requirements.rst index 25798606..97b1813f 100644 --- a/docs/testing/developer/requirements/03-Requirements.rst +++ b/docs/testing/developer/requirements/03-Requirements.rst @@ -13,7 +13,7 @@ Requirements .. _SampleVNF: https://wiki.opnfv.org/samplevnf .. _Technical_Briefs: https://wiki.opnfv.org/display/SAM/Technical+Briefs+of+VNFs -Supported Test setup: +Supported Test setup -------------------- The device under test (DUT) consists of a system following diff --git a/docs/testing/user/userguide/01-introduction.rst b/docs/testing/user/userguide/01-introduction.rst index 4ddde201..4ddde201 100755..100644 --- a/docs/testing/user/userguide/01-introduction.rst +++ b/docs/testing/user/userguide/01-introduction.rst diff --git a/docs/testing/user/userguide/01-prox_documentation.rst b/docs/testing/user/userguide/01-prox_documentation.rst index 0fbee344..12c740da 100644 --- a/docs/testing/user/userguide/01-prox_documentation.rst +++ b/docs/testing/user/userguide/01-prox_documentation.rst @@ -1,4 +1,4 @@ Testing with PROX ================= -The PROX documentation can be found in `Prox - Packet pROcessing eXecution engine <https://wiki.opnfv.org/x/AAa9>`_ -How to use PROX with the rapid pyton scripts can be found in `Rapid scripting <https://wiki.opnfv.org/x/OwM-Ag>`_ +The PROX documentation can be found in `Prox - Packet pROcessing eXecution engine <https://wiki-old.opnfv.org/x/AAa9>`_ +How to use PROX with the rapid pyton scripts can be found in `Rapid scripting <https://wiki-old.opnfv.org/x/OwM-Ag>`_ diff --git a/docs/testing/user/userguide/03-architecture.rst b/docs/testing/user/userguide/03-architecture.rst index 08e1b2f2..bdc51d3f 100755..100644 --- a/docs/testing/user/userguide/03-architecture.rst +++ b/docs/testing/user/userguide/03-architecture.rst @@ -37,8 +37,8 @@ validating the sample VNFs through OPEN SOURCE VNF approximations and test tools The VNFs belongs to this project are never meant for field deployment. All the VNF source code part of this project requires Apache License Version 2.0. -Supported deployment: ----------------------- +Supported deployment +-------------------- * Bare-Metal - All VNFs can run on a Bare-Metal DUT * Standalone Virtualization(SV): All VNFs can run on SV like VPP as switch, ovs, ovs-dpdk, srioc @@ -47,7 +47,6 @@ Supported deployment: VNF supported ------------- - Carrier Grade Network Address Translation (CG-NAT) VNF - :: The Carrier Grade Network Address and port Translation (vCG-NAPT) is a VNF approximation extending the life of the service providers IPv4 network infrastructure and mitigate IPv4 address exhaustion by using address and @@ -55,23 +54,19 @@ VNF supported It also supports the connectivity between the IPv6 access network to IPv4 data network using the IPv6 to IPv4 address translation and vice versa. - Firewall (vFW) VNF - :: The Virtual Firewall (vFW) is a VNF approximation serving as a state full L3/L4 packet filter with connection tracking enabled for TCP, UDP and ICMP. The VNF could be a part of Network Services (industry use-cases) deployed to secure the enterprise network from un-trusted network. - Access Control List (vACL) VNF - :: The vACL vNF is implemented as a DPDK application using VNF Infrastructure Library (VIL). The VIL implements common VNF internal, optimized for Intel Architecture functions like load balancing between cores, IPv4/IPv6 stack features, and interface to NFV infrastructure like OVS or SRIOV. - UDP_Replay - :: The UDP Replay is implemented as a DPDK application using VNF Infrastructure Library (VIL). Performs as a refelector of all the traffic on given port. - Prox - Packet pROcessing eXecution engine. - :: Packet pROcessing eXecution Engine (PROX) which is a DPDK application. PROX can do operations on packets in a highly configurable manner. The PROX application is also displaying performance statistics that can @@ -142,14 +137,15 @@ The following features were verified by SampleVNF test cases: Test Framework -------------- -.. _Yardstick_NSB: http://artifacts.opnfv.org/yardstick/docs/testing_user_userguide/index.html#document-13-nsb-overview +.. _Yardstick_NSB: http://artifacts.opnfv.org/yardstick/docs/testing_user_userguide/index.html#document-11-nsb-overview +.. _ETSI GS NFV-TST 001: https://portal.etsi.org/webapp/workprogram/Report_WorkItem.asp?WKI_ID=46009 SampleVNF Test Infrastructure (NSB (Yardstick_NSB_)) in yardstick helps to facilitate consistent/repeatable methodologies for characterizing & validating the sample VNFs (:term:`VNF`) through OPEN SOURCE VNF approximations. -Network Service Benchmarking in yardstick framework follows ETSI GS NFV-TST001_ +Network Service Benchmarking in yardstick framework follows `ETSI GS NFV-TST 001`_ to verify/characterize both :term:`NFVI` & :term:`VNF` For more inforamtion refer, Yardstick_NSB_ diff --git a/docs/testing/user/userguide/03-installation.rst b/docs/testing/user/userguide/03-installation.rst index 8fb89744..4407b276 100644 --- a/docs/testing/user/userguide/03-installation.rst +++ b/docs/testing/user/userguide/03-installation.rst @@ -30,15 +30,17 @@ Prerequisites ------------- Supported Test setup -^^^^^^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^^^^^^ The device under test (DUT) is an NFVI instance on which we can deploy PROX instances. -A PROX instance is a machine that +A PROX instance is a machine that: + * has a management interface that can be reached from the test container * has one or more data plane interfaces on a dataplane network. * can be a container, a VM or a bare metal machine. We just need to be able to ssh into the PROX machine from the test container. * is optimized for data plane traffic. * will measure the throughput that is offered through its dataplane interface(s) + There are no requirements on the NFVI instance itself. Of course, the measured throughput will depend heavily on the NFVI characteristics. In this release, we are supporting an automated deployment of the PROX instance on an NFVI that diff --git a/docs/testing/user/userguide/04-running_the_test.rst b/docs/testing/user/userguide/04-running_the_test.rst index 5ffa58ca..3d3a1e6c 100644 --- a/docs/testing/user/userguide/04-running_the_test.rst +++ b/docs/testing/user/userguide/04-running_the_test.rst @@ -7,7 +7,7 @@ Running the test ================ .. _NFV-TST009: https://docbox.etsi.org/ISG/NFV/open/Publications_pdf/Specs-Reports/NFV-TST%20009v3.2.1%20-%20GS%20-%20NFVI_Benchmarks.pdf -.. _TST009_Throughput_64B_64F.test: https://github.com/opnfv/samplevnf/blob/master/VNFs/DPPD-PROX/helper-scripts/rapid/TST009_Throughput_64B_64F.test +.. _TST009_Throughput_64B_64F.test: https://github.com/opnfv/samplevnf/blob/master/VNFs/DPPD-PROX/helper-scripts/rapid/tests/TST009_Throughput_64B_64F.test .. _rapid_location: https://github.com/opnfv/samplevnf/blob/master/VNFs/DPPD-PROX/helper-scripts/rapid/ Overview @@ -124,6 +124,7 @@ not something to start with. gencores = [1] latcores = [3] #bucket_size_exp = 12 + testy ^^^^^ In the testy sections, where y denotes the index of the test, the test that will diff --git a/docs/testing/user/userguide/05-How_to_run_SampleVNFs.rst b/docs/testing/user/userguide/05-How_to_run_SampleVNFs.rst index 7ba25fe1..28da0ebd 100644 --- a/docs/testing/user/userguide/05-How_to_run_SampleVNFs.rst +++ b/docs/testing/user/userguide/05-How_to_run_SampleVNFs.rst @@ -17,6 +17,7 @@ The device under test (DUT) consists of a system following; * Specific Intel Network Interface Cards (NICs) * BIOS settings noting those that updated from the basic settings * DPDK build configuration settings, and commands used for tests + Connected to the DUT is an IXIA* or Software Traffic generator like pktgen or TRex, simulation platform to generate packet traffic to the DUT ports and determine the throughput/latency at the tester side. @@ -103,17 +104,16 @@ The connectivity could be (TG_2(UDP_Replay) reflects all the traffic on the given port) * Bare-Metal - Refer: http://fast.dpdk.org/doc/pdf-guides/ to setup the DUT for VNF to run + Refer: http://fast.dpdk.org/doc/pdf-guides/ to setup the DUT for VNF to run * Standalone Virtualization - PHY-VM-PHY + * SRIOV - Refer below link to setup sriov - https://software.intel.com/en-us/articles/using-sr-iov-to-share-an-ethernet-port-among-multiple-vms + https://software.intel.com/en-us/articles/using-sr-iov-to-share-an-ethernet-port-among-multiple-vms * OVS_DPDK - Refer below link to setup ovs-dpdk - http://docs.openvswitch.org/en/latest/intro/install/general/ - http://docs.openvswitch.org/en/latest/intro/install/dpdk/ + http://docs.openvswitch.org/en/latest/intro/install/general/ + http://docs.openvswitch.org/en/latest/intro/install/dpdk/ * Openstack Use any OPNFV installer to deploy the openstack. @@ -132,19 +132,21 @@ Step 0: Preparing hardware connection Step 1: Setting up Traffic generator (TRex) TRex Software preparations - ************************** * Install the OS (Bare metal Linux, not VM!) * Obtain the latest TRex package: wget https://trex-tgn.cisco.com/trex/release/latest * Untar the package: tar -xzf latest * Change dir to unzipped TRex * Create config file using command: sudo python dpdk_setup_ports.py -i + In case of Ubuntu 16 need python3 + See paragraph config creation for detailed step-by-step + (Refer: https://trex-tgn.cisco.com/trex/doc/trex_stateless_bench.html) Build SampleVNFs ------------------ +---------------- Step 2: Procedure to build SampleVNFs @@ -487,7 +489,7 @@ step 4: Run Test using traffic geneator UDP_Replay - How to run ----------------------------------------- +----------------------- Step 3: Bind the datapath ports to DPDK @@ -532,7 +534,7 @@ step 4: Run Test using traffic geneator For more details refer: https://trex-tgn.cisco.com/trex/doc/trex_stateless_bench.html PROX - How to run ------------------- +----------------- Description ^^^^^^^^^^^ @@ -654,7 +656,7 @@ PROX COMMANDS AND SCREENS +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+ | version | Show version | | +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+ - | port_stats <port id> | Print rate for no_mbufs, ierrors, rx_bytes, tx_bytes, rx_pkts, | | + | port_stats <port id> | Print rate for no_mbufs, ierrors, rx_bytes, tx_bytes, rx_pkts, | | | | tx_pkts and totals for RX, TX, no_mbufs ierrors for port <port id> | | +----------------------------------------------+---------------------------------------------------------------------------+----------------------------+ @@ -941,7 +943,7 @@ PROX Compiation installation * cd samplevnf * export RTE_SDK=`pwd`/dpdk * export RTE_TARGET=x86_64-native-linuxapp-gcc -* git clone http://dpdk.org/git/dpdk +* git clone git://dpdk.org/dpdk * cd dpdk * git checkout v17.05 * make install T=$RTE_TARGET diff --git a/docs/testing/user/userguide/06-How_to_use_REST_api.rst b/docs/testing/user/userguide/06-How_to_use_REST_api.rst index b8c0cbea..ba768d78 100644 --- a/docs/testing/user/userguide/06-How_to_use_REST_api.rst +++ b/docs/testing/user/userguide/06-How_to_use_REST_api.rst @@ -3,12 +3,12 @@ .. http://creativecommons.org/licenses/by/4.0 .. (c) opnfv, national center of scientific research "demokritos" and others. -======================================================== +======== REST API -======================================================== +======== Introduction ---------------- +------------ As the internet industry progresses creating REST API becomes more concrete with emerging best Practices. RESTful web services don’t follow a prescribed standard except fpr the protocol that is used which is HTTP, its important @@ -26,7 +26,7 @@ Here are important points to be considered: always same no matter how many times these operations are invoked. * PUT and POST operation are nearly same with the difference lying only in the result where PUT operation is idempotent and POST - operation can cause different result. + operation can cause different result. REST API in SampleVNF @@ -45,7 +45,7 @@ REST api on VNF’s will help adapting with the new automation techniques being adapted in yardstick. Web server integration with VNF’s ----------------------------------- +--------------------------------- In order to implement REST api’s in VNF one of the first task is to identify a simple web server that needs to be integrated with VNF’s. @@ -150,7 +150,7 @@ API Usage --------- Run time Usage -^^^^^^^^^^^^^^ +============== An application(say vFW) with REST API support is run as follows with just PORT MASK as input. The following environment variables @@ -182,6 +182,7 @@ samplevnf directory). 2. Check the Link IP's using the REST API (vCGNAPT/vACL/vFW) :: + e.g curl <IP>/vnf/config/link This would indicate the number of links enabled. You should enable all the links @@ -194,6 +195,7 @@ samplevnf directory). 3. Now that links are enabled we can configure IP's using link method as follows (vCGNAPT/vACL/vFW) :: + e.g curl -X POST -H "Content-Type:application/json" -d '{"ipv4":"<IP to be configured>","depth":"24"}' http://<IP>/vnf/config/link/0 curl -X POST -H "Content-Type:application/json" -d '{"ipv4":"IP to be configured","depth":"24"}' @@ -207,6 +209,7 @@ samplevnf directory). 4. Adding arp entries we can use this method (vCGNAPT/vACL/vFW) :: + /vnf/config/arp e.g @@ -220,15 +223,17 @@ samplevnf directory). 5. Adding route entries we can use this method (vCGNAPT/vACL/vFW) :: + /vnf/config/route e.g curl -X POST -H "Content-Type:application/json" -d '{"type":"net", "depth":"8", "nhipv4":"202.16.100.20", - "portid":"0"}' http://10.223.166.240/vnf/config/route + "portid":"0"}' http://10.223.166.240/vnf/config/route curl -X POST -H "Content-Type:application/json" -d '{"type":"net", "depth":8", "nhipv4":"172.16.100.20", "portid":"1"}' http://10.223.166.240/vnf/config/route 5. In order to load the rules a script file needs to be posting a script.(vACL/vFW) :: + /vnf/config/rules/load Typical example for loading a script file is shown below @@ -239,12 +244,14 @@ samplevnf directory). 6. The following REST api's for runtime configuring through a script (vCGNAPT Only) :: + /vnf/config/rules/clear /vnf/config/nat /vnf/config/nat/load 7. For debug purpose following REST API's could be used as described above.(vCGNAPT/vACL/vFW) :: + /vnf/dbg e.g curl http://10.223.166.240/vnf/config/dbg @@ -258,10 +265,12 @@ samplevnf directory). 8. For stats we can use the following method (vCGNAPT/vACL/vFW) :: + /vnf/stats e.g curl <IP>/vnf/stats 9. For quittiong the application (vCGNAPT/vACL/vFW) :: + /vnf/quit e.g curl <IP>/vnf/quit diff --git a/docs/testing/user/userguide/07-Config_files.rst b/docs/testing/user/userguide/07-Config_files.rst index d5564e8d..f96462e1 100644 --- a/docs/testing/user/userguide/07-Config_files.rst +++ b/docs/testing/user/userguide/07-Config_files.rst @@ -380,7 +380,7 @@ This configuration doesn't require LOADB and TXRX pipelines vACL Config files ----------------- +----------------- The reference configuration files explained here are for Software and Hardware loadbalancing with IPv4 traffic type and single port pair. diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst index 64b01b88..5cc2c5e1 100644 --- a/docs/testing/user/userguide/index.rst +++ b/docs/testing/user/userguide/index.rst @@ -10,7 +10,6 @@ SampleVNF User Guide .. toctree:: :maxdepth: 4 - :numbered: 01-introduction.rst 02-methodology.rst diff --git a/docs/testing/user/userguide/references.rst b/docs/testing/user/userguide/references.rst index 30f6e604..f00a872c 100644 --- a/docs/testing/user/userguide/references.rst +++ b/docs/testing/user/userguide/references.rst @@ -11,8 +11,8 @@ References OPNFV ===== -* Yardstick wiki: https://wiki.opnfv.org/yardstick -* SampleVNF wiki: https://wiki.opnfv.org/samplevnf +* Yardstick wiki: https://wiki-old.opnfv.org/display/yardstick +* SampleVNF wiki: https://wiki-old.opnfv.org/display/SAM References used in Test Cases ============================= @@ -22,7 +22,7 @@ References used in Test Cases * DPDK: http://dpdk.org * DPDK supported NICs: http://dpdk.org/doc/nics * fdisk: http://www.tldp.org/HOWTO/Partition/fdisk_partitioning.html -* fio: http://www.bluestop.org/fio/HOWTO.txt +* fio: https://github.com/axboe/fio * free: http://manpages.ubuntu.com/manpages/trusty/en/man1/free.1.html * iperf3: https://iperf.fr/ * Lmbench man-pages: http://manpages.ubuntu.com/manpages/trusty/lat_mem_rd.8.html |