diff options
| -rw-r--r-- | docs/userguide/apexlake_framework/apexlake_api.rst | 110 | ||||
| -rw-r--r-- | docs/userguide/apexlake_framework/apexlake_installation.rst | 210 | ||||
| -rw-r--r-- | docs/userguide/apexlake_framework/index.rst | 11 | ||||
| -rw-r--r-- | yardstick/vTC/apexlake/README.md | 122 | ||||
| -rw-r--r-- | yardstick/vTC/apexlake/README.rst | 210 | ||||
| -rw-r--r-- | yardstick/vTC/apexlake/experimental_framework/common.py | 1 | ||||
| -rw-r--r-- | yardstick/vTC/apexlake/setup.py | 2 |
7 files changed, 541 insertions, 125 deletions
diff --git a/docs/userguide/apexlake_framework/apexlake_api.rst b/docs/userguide/apexlake_framework/apexlake_api.rst new file mode 100644 index 000000000..94f07dc06 --- /dev/null +++ b/docs/userguide/apexlake_framework/apexlake_api.rst @@ -0,0 +1,110 @@ +================================= +Apexlake API interface definition +================================= + +The API interface provided by the framework in order to execute the test cases is defined in the following. + + +init +---- + +**static init()** + + Initializes the Framework + + **Returns** None + + +get_available_test_cases +------------------------ + +**static get_available_test_cases()** + + Returns a list of available test cases. This list include eventual modules developed by the user, if any. + Each test case is returned as a string that represents the full name of the test case and that + can be used to get more information calling get_test_case_features(test_case_name) + + **Returns** list of strings + + +get_test_case_features +---------------------- + +**static get_test_case_features(test_case)** + + Returns a list of features (description, requested parameters, allowed values, etc.) + for a specified test case. + + **Parameters** + + - **test_case** + + Name of the test case (string). The string represents the test + case and can be obtained calling “get_available_test_cases()” method. + + **Returns** + dict() containing the features of the test case + + +execute_framework +----------------- + +**static execute_framework** (test_cases, + + iterations, + + heat_template, + + heat_template_parameters, + + deployment_configuration, + + openstack_credentials) + + Executes the framework according the inputs + + **Parameters** + + - **test_cases** + + Test cases to be ran on the workload (dict() of dict()) + + Example: + test_case = dict() + + test_case[’name’] = ‘module.Class’ + + test_case[’params’] = dict() + + test_case[’params’][’throughput’] = ‘1’ + + test_case[’params’][’vlan_sender’] = ‘1000’ + + test_case[’params’][’vlan_receiver’] = ‘1001’ + + test_cases = [test_case] + + - **iterations** + Number of cycles to be executed (int) + + - **heat_template** + (string) File name of the heat template of the workload to be deployed. + It contains the parameters to be evaluated in the form of #parameter_name. + (See heat_templates/vTC.yaml as example). + + - **heat_template_parameters** + (dict) Parameters to be provided as input to the + heat template. See http://docs.openstack.org/developer/heat/ template_guide/hot_guide.html + section “Template input parameters” for further info. + + - **deployment_configuration** + ( dict[string] = list(strings) ) ) Dictionary of parameters + representing the deployment configuration of the workload + + The key is a string corresponding to the name of the parameter, + the value is a list of strings representing the value to be + assumed by a specific param. The parameters are user defined: + they have to correspond to the place holders (#parameter_name) + specified in the heat template. + + **Returns** dict() containing results diff --git a/docs/userguide/apexlake_framework/apexlake_installation.rst b/docs/userguide/apexlake_framework/apexlake_installation.rst new file mode 100644 index 000000000..38c838494 --- /dev/null +++ b/docs/userguide/apexlake_framework/apexlake_installation.rst @@ -0,0 +1,210 @@ +.. _DPDK: http://dpdk.org/doc/nics +.. _DPDK-pktgen: https://github.com/Pktgen/Pktgen-DPDK/ +.. _SRIOV: https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking + +=========================== +Apexlake installation guide +=========================== +ApexLake is a framework that provides automatic execution of experiments and related data collection to help +the user validating the infrastructure from the perspective of a Virtual Network Function. +To do so in the context of Yardstick, the virtual Traffic Classifier network function is utilized. + + +Hardware dependencies to run the framework +========================================== +In order to run the framework some hardware dependencies are required to run ApexLake. + +The framework needs to be installed on a physical node where the DPDK packet DPDK-pktgen_ +can be correctly installed and executed. +That requires for the packet generator to have 2 NICs DPDK_ Compatible. + +The 2 NICs will be connected to the switch where the Openstack VM network is managed. + +The switch is required to support multicast traffic and snooping protocol. + +The corresponding ports to which the cables are connected will be configured as VLAN trunks +using two of the VLAN IDs available for Neutron. +The mentioned VLAN IDs will be required in further configuration steps. + + +Software dependencies to run the framework +========================================== +Before to start the framework, a set of dependencies are required to be installed. +In the following a set of instructions to be executed on the Linux shell to install dependencies +and configure the environment is presented. + +1. Install dependencies. + +To install the dependencies required by the framework it is necessary install the following packages. +The following example is provided for Ubuntu and need to be executed as root. +:: + + apt-get install python-dev + apt-get install python-pip + apt-get install python-mock + apt-get install tcpreplay + apt-get install libpcap-dev + +2. Install the framework on the system. + +The installation of the framework on the system requires the setup of the project. +After entering into the apexlake directory, it is sufficient to run the following command. +:: + + python setup.py install + +3. Source OpenStack openrc file. + +:: + + source openrc + +4. Create 2 Networks based on VLANs in Neutron. + +In order for the network communication between the packet generator and the Compute node to +work fine, it is required to create through Neutron two networks and map those on the VLAN IDs +that have been previously used for the configuration on the physical switch. +The underlying switch needs to be configured accordingly. +:: + + VLAN_1=2025 + VLAN_2=2021 + neutron net-create apexlake_inbound_network \ + --provider:network_type vlan \ + --provider:segmentation_id $VLAN_1 \ + --provider:physical_network physnet1 + + neutron subnet-create apexlake_inbound_network \ + 192.168.0.0/24 --name apexlake_inbound_subnet + + neutron net-create apexlake_outbound_network \ + --provider:network_type vlan \ + --provider:physical_network physnet1 + + neutron net-create apexlake_inbound_network \ + --provider:network_type vlan \ + --provider:segmentation_id $VLAN_2 \ + --provider:physical_network physnet1 + + neutron subnet-create apexlake_outbound_network 192.168.1.0/24 \ + --name apexlake_outbound_subnet + +5. Configure the Test Cases. + +The VLAN tags are also required into the test case Yardstick yaml file as parameters the following test cases: + - TC 006 + - TC 007 + - TC 020 + - TC 021 + + +Install and configure DPDK Pktgen ++++++++++++++++++++++++++++++++++ +The execution of the framework is based on DPDK Pktgen. +If DPDK Pktgen has not been installed on the system by the user, it is necessary to download, compile and configure it. +The user can create a directory and download the dpdk packet generator source code: +:: + + cd experimental_framework/libraries + mkdir dpdk_pktgen + git clone https://github.com/pktgen/Pktgen-DPDK.git + +For the installation and configuration of DPDK and DPDK Pktgen please follow the official DPDK Pktgen README file. +Once the installation is completed, it is necessary to load the DPDK kernel driver, as follow: +:: + + insmod uio + insmod DPDK_DIR/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko + +It is required to properly set the configuration file according to the system on Pktgen runs on. +A description of the required configuration parameters and examples is provided in the following: +:: + + [PacketGen] + packet_generator = dpdk_pktgen + + # This is the directory where the packet generator is installed + # (if the user previously installed dpdk-pktgen, + # it is required to provide the director where it is installed). + pktgen_directory = /home/user/software/dpdk_pktgen/dpdk/examples/pktgen/ + + # This is the directory where DPDK is installed + dpdk_directory = /home/user/apexlake/experimental_framework/libraries/Pktgen-DPDK/dpdk/ + + # Name of the dpdk-pktgen program that starts the packet generator + program_name = app/app/x86_64-native-linuxapp-gcc/pktgen + + # DPDK coremask (see DPDK-Pktgen readme) + coremask = 1f + + # DPDK memory channels (see DPDK-Pktgen readme) + memory_channels = 3 + + # Name of the interface of the pktgen to be used to send traffic (vlan_sender) + name_if_1 = p1p1 + + # Name of the interface of the pktgen to be used to receive traffic (vlan_receiver) + name_if_2 = p1p2 + + # PCI bus address correspondent to if_1 + bus_slot_nic_1 = 01:00.0 + + # PCI bus address correspondent to if_2 + bus_slot_nic_2 = 01:00.1 + + +To find the parameters related to names of the NICs and addresses of the PCI buses +the user may find useful to run the DPDK tool nic_bind as follows: +:: + + DPDK_DIR/tools/dpdk_nic_bind.py --status + +which lists the NICs available on the system, show the available drivers and bus addresses for each interface. +Please make sure to select NICs which are DPDK compatible. + +Installation and configuration of smcroute +++++++++++++++++++++++++++++++++++++++++++ +The user is required to install smcroute which is used by the framework to support multicast communications. +In the following a list of commands to be ran to download and install smroute is provided. +:: + + cd ~ + git clone https://github.com/troglobit/smcroute.git + cd smcroute + sed -i 's/aclocal-1.11/aclocal/g' ./autogen.sh + sed -i 's/automake-1.11/automake/g' ./autogen.sh + ./autogen.sh + ./configure + make + sudo make install + cd .. + +It is also required to create a configuration file using the following command: + + SMCROUTE_NIC=(name of the nic) + +where name of the nic is the name used previously for the variable "name_if_2". +In the example it would be: +:: + + SMCROUTE_NIC=p1p2 + +Then create the smcroute configuration file /etc/smcroute.conf +:: + + echo mgroup from $SMCROUTE_NIC group 224.192.16.1 > /etc/smcroute.conf + + +At the end of this procedure it will be necessary to perform the following actions to add the user to the sudoers: +:: + + adduser USERNAME sudo + echo "user ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers + + +Experiment using SR-IOV configuration on the compute node ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +In order to enable SR-IOV interfaces on the physical NIC of the compute node, a compatible NIC is required. +NIC configuration depends on model and vendor. After proper configuration to support SR-IOV, +a proper configuration of openstack is required. +For further information, please look at the _SRIOV configuration guide diff --git a/docs/userguide/apexlake_framework/index.rst b/docs/userguide/apexlake_framework/index.rst new file mode 100644 index 000000000..47ebfcdf4 --- /dev/null +++ b/docs/userguide/apexlake_framework/index.rst @@ -0,0 +1,11 @@ +******************************** +Apexlake Framework Documentation +******************************** + +.. toctree:: + :numbered: + :maxdepth: 2 + + apexlake_installation + apexlake_api + diff --git a/yardstick/vTC/apexlake/README.md b/yardstick/vTC/apexlake/README.md deleted file mode 100644 index b89df0725..000000000 --- a/yardstick/vTC/apexlake/README.md +++ /dev/null @@ -1,122 +0,0 @@ -Welcome to ApexLake's documentation! -==================================== -ApexLake is a framework that provides automatic execution of experiment and related data collection to help -the user validating the infrastructure from a Virtual Network Function perspective. - -Install framework and dependencies ----------------------------------- -Before to start the framework, a set of dependencies are required. -In the following a set of instructions to be executed on the Linux shell to install dependencies and configure the environment. - -1. Install dependencies - - # apt-get install python-dev - - # apt-get install python-pip - - # apt-get install python-mock - - # apt-get install tcpreplay - - # apt-get install libpcap-dev - -2. Install the framework on the system - - # python setup.py install - -3. Source OpenStack openrc file - - $ source openrc - -4. Create 2 Networks (and subnets) based on VLANs (provider:network_type = vlan) in Neutron - - $ neutron net-create apexlake_inbound_network --provider:network_type vlan --provider:physical_network physnet1 - - $ neutron subnet-create apexlake_inbound_network 192.168.0.0/24 --name apexlake_inbound_subnet - - $ neutron net-create apexlake_outbound_network --provider:network_type vlan --provider:physical_network physnet1 - - $ neutron subnet-create apexlake_outbound_network 192.168.1.0/24 --name apexlake_outbound_subnet - -5. Insert VLAN tags related to the networks have to ApexLake, either: - - into the "conf.cfg" configuration file, or - - through the Python API. - - -Install and configure DPDK Pktgen -+++++++++++++++++++++++++++++++++ -The execution of the framework is based on DPDK Pktgen. -If DPDK Pktgen has not been installed on the system by the user, it is necessary to download, compile and configure it. -The user can create a directory and download the dpdk packet generator source code: - - $ cd experimental_framework/libraries - - $ mkdir dpdk_pktgen - - $ git clone https://github.com/pktgen/Pktgen-DPDK.git - -For the installation and configuration of DPDK and DPDK Pktgen please follow the official DPDK Pktgen README file. -Once the installation is completed, it is necessary to load the DPDK kernel driver, as follow: - - # insmod uio - - # insmod DPDK_DIR/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko - -It is required to properly set the configuration file according to the system on Pktgen runs on. -An example is provided in the following: - - - [PacketGen] - - packet_generator = dpdk_pktgen - - pktgen_directory = /home/user/apexlake/experimental_framework/libraries/dpdk_pktgen/dpdk/examples/pktgen/ - -- This is the directory where the packet generator is installed (if the user previously installed dpdk-pktgen, it is required to provide the director where it is installed). - - dpdk_directory = /home/user/apexlake/experimental_framework/libraries/Pktgen-DPDK/dpdk/ - -- This is the directory where DPDK is installed - - program_name = app/app/x86_64-native-linuxapp-gcc/pktgen - -- This is the name of the dpdk-pktgen program that starts the packet generator - - coremask = 1f - -- DPDK coremask (see DPDK-Pktgen readme) - - memory_channels = 3 - -- DPDK memory channels (see DPDK-Pktgen readme) - - name_if_1 = p1p1 - -- Name of the interface of the pktgen to be used to send traffic - - name_if_2 = p1p2 - -- Name of the interface of the pktgen to be used to receive traffic - - bus_slot_nic_1 = 01:00.0 - -- PCI bus address correspondent to the if_1 - - bus_slot_nic_2 = 01:00.1 - -- PCI bus address correspondent to the if_2 - - -To find the parameters related to names of the NICs and addresses of the PCI buses the user may find useful to run the DPDK tool nic_bind as follows: - - - $ DPDK_DIR/tools/dpdk_nic_bind.py --status - -which lists the NICs available on the system, show the available drivers and bus addresses for each interface. -Please make sure to select NICs which are DPDK compatible. - -Installation and configuration of smcroute -++++++++++++++++++++++++++++++++++++++++++ -The user is required to install smcroute which is used by the framework to support multicast communications. -In the following a list of commands to be ran to download and install smroute is provided. - - - $ cd ~ - - $ git clone https://github.com/troglobit/smcroute.git - - $ cd smcroute - - $ sed -i 's/aclocal-1.11/aclocal/g' ./autogen.sh - - $ sed -i 's/automake-1.11/automake/g' ./autogen.sh - - $ ./autogen.sh - - $ ./configure - - $ make - - $ sudo make install - - $ cd .. - -It is also required to create a configuration file using the following command: - - - $ SMCROUTE_NIC=(name of the nic) - -where name of the nic is the name used previously for the variable "name_if_2". -In the example it would be: - - - $ SMCROUTE_NIC=p1p2 - -Then create the smcroute configuration file /etc/smcroute.conf - - - $echo mgroup from $SMCROUTE_NIC group 224.192.16.1 > /etc/smcroute.conf - - -At the end of this procedure it will be necessary to perform the following actions to add the user to the sudoers: - - - adduser USERNAME sudo - - echo "user ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers - - -Experiment using SR-IOV configuration on the compute node -+++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -In order to enable SR-IOV interfaces on the physical NIC of the compute node, a compatible NIC is required. -NIC configuration depends on model and vendor. After proper configuration to support SR-IOV, a proper configuration of openstack is required. -For further information, please look at the following link: -https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking diff --git a/yardstick/vTC/apexlake/README.rst b/yardstick/vTC/apexlake/README.rst new file mode 100644 index 000000000..38c838494 --- /dev/null +++ b/yardstick/vTC/apexlake/README.rst @@ -0,0 +1,210 @@ +.. _DPDK: http://dpdk.org/doc/nics +.. _DPDK-pktgen: https://github.com/Pktgen/Pktgen-DPDK/ +.. _SRIOV: https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking + +=========================== +Apexlake installation guide +=========================== +ApexLake is a framework that provides automatic execution of experiments and related data collection to help +the user validating the infrastructure from the perspective of a Virtual Network Function. +To do so in the context of Yardstick, the virtual Traffic Classifier network function is utilized. + + +Hardware dependencies to run the framework +========================================== +In order to run the framework some hardware dependencies are required to run ApexLake. + +The framework needs to be installed on a physical node where the DPDK packet DPDK-pktgen_ +can be correctly installed and executed. +That requires for the packet generator to have 2 NICs DPDK_ Compatible. + +The 2 NICs will be connected to the switch where the Openstack VM network is managed. + +The switch is required to support multicast traffic and snooping protocol. + +The corresponding ports to which the cables are connected will be configured as VLAN trunks +using two of the VLAN IDs available for Neutron. +The mentioned VLAN IDs will be required in further configuration steps. + + +Software dependencies to run the framework +========================================== +Before to start the framework, a set of dependencies are required to be installed. +In the following a set of instructions to be executed on the Linux shell to install dependencies +and configure the environment is presented. + +1. Install dependencies. + +To install the dependencies required by the framework it is necessary install the following packages. +The following example is provided for Ubuntu and need to be executed as root. +:: + + apt-get install python-dev + apt-get install python-pip + apt-get install python-mock + apt-get install tcpreplay + apt-get install libpcap-dev + +2. Install the framework on the system. + +The installation of the framework on the system requires the setup of the project. +After entering into the apexlake directory, it is sufficient to run the following command. +:: + + python setup.py install + +3. Source OpenStack openrc file. + +:: + + source openrc + +4. Create 2 Networks based on VLANs in Neutron. + +In order for the network communication between the packet generator and the Compute node to +work fine, it is required to create through Neutron two networks and map those on the VLAN IDs +that have been previously used for the configuration on the physical switch. +The underlying switch needs to be configured accordingly. +:: + + VLAN_1=2025 + VLAN_2=2021 + neutron net-create apexlake_inbound_network \ + --provider:network_type vlan \ + --provider:segmentation_id $VLAN_1 \ + --provider:physical_network physnet1 + + neutron subnet-create apexlake_inbound_network \ + 192.168.0.0/24 --name apexlake_inbound_subnet + + neutron net-create apexlake_outbound_network \ + --provider:network_type vlan \ + --provider:physical_network physnet1 + + neutron net-create apexlake_inbound_network \ + --provider:network_type vlan \ + --provider:segmentation_id $VLAN_2 \ + --provider:physical_network physnet1 + + neutron subnet-create apexlake_outbound_network 192.168.1.0/24 \ + --name apexlake_outbound_subnet + +5. Configure the Test Cases. + +The VLAN tags are also required into the test case Yardstick yaml file as parameters the following test cases: + - TC 006 + - TC 007 + - TC 020 + - TC 021 + + +Install and configure DPDK Pktgen ++++++++++++++++++++++++++++++++++ +The execution of the framework is based on DPDK Pktgen. +If DPDK Pktgen has not been installed on the system by the user, it is necessary to download, compile and configure it. +The user can create a directory and download the dpdk packet generator source code: +:: + + cd experimental_framework/libraries + mkdir dpdk_pktgen + git clone https://github.com/pktgen/Pktgen-DPDK.git + +For the installation and configuration of DPDK and DPDK Pktgen please follow the official DPDK Pktgen README file. +Once the installation is completed, it is necessary to load the DPDK kernel driver, as follow: +:: + + insmod uio + insmod DPDK_DIR/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko + +It is required to properly set the configuration file according to the system on Pktgen runs on. +A description of the required configuration parameters and examples is provided in the following: +:: + + [PacketGen] + packet_generator = dpdk_pktgen + + # This is the directory where the packet generator is installed + # (if the user previously installed dpdk-pktgen, + # it is required to provide the director where it is installed). + pktgen_directory = /home/user/software/dpdk_pktgen/dpdk/examples/pktgen/ + + # This is the directory where DPDK is installed + dpdk_directory = /home/user/apexlake/experimental_framework/libraries/Pktgen-DPDK/dpdk/ + + # Name of the dpdk-pktgen program that starts the packet generator + program_name = app/app/x86_64-native-linuxapp-gcc/pktgen + + # DPDK coremask (see DPDK-Pktgen readme) + coremask = 1f + + # DPDK memory channels (see DPDK-Pktgen readme) + memory_channels = 3 + + # Name of the interface of the pktgen to be used to send traffic (vlan_sender) + name_if_1 = p1p1 + + # Name of the interface of the pktgen to be used to receive traffic (vlan_receiver) + name_if_2 = p1p2 + + # PCI bus address correspondent to if_1 + bus_slot_nic_1 = 01:00.0 + + # PCI bus address correspondent to if_2 + bus_slot_nic_2 = 01:00.1 + + +To find the parameters related to names of the NICs and addresses of the PCI buses +the user may find useful to run the DPDK tool nic_bind as follows: +:: + + DPDK_DIR/tools/dpdk_nic_bind.py --status + +which lists the NICs available on the system, show the available drivers and bus addresses for each interface. +Please make sure to select NICs which are DPDK compatible. + +Installation and configuration of smcroute +++++++++++++++++++++++++++++++++++++++++++ +The user is required to install smcroute which is used by the framework to support multicast communications. +In the following a list of commands to be ran to download and install smroute is provided. +:: + + cd ~ + git clone https://github.com/troglobit/smcroute.git + cd smcroute + sed -i 's/aclocal-1.11/aclocal/g' ./autogen.sh + sed -i 's/automake-1.11/automake/g' ./autogen.sh + ./autogen.sh + ./configure + make + sudo make install + cd .. + +It is also required to create a configuration file using the following command: + + SMCROUTE_NIC=(name of the nic) + +where name of the nic is the name used previously for the variable "name_if_2". +In the example it would be: +:: + + SMCROUTE_NIC=p1p2 + +Then create the smcroute configuration file /etc/smcroute.conf +:: + + echo mgroup from $SMCROUTE_NIC group 224.192.16.1 > /etc/smcroute.conf + + +At the end of this procedure it will be necessary to perform the following actions to add the user to the sudoers: +:: + + adduser USERNAME sudo + echo "user ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers + + +Experiment using SR-IOV configuration on the compute node ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +In order to enable SR-IOV interfaces on the physical NIC of the compute node, a compatible NIC is required. +NIC configuration depends on model and vendor. After proper configuration to support SR-IOV, +a proper configuration of openstack is required. +For further information, please look at the _SRIOV configuration guide diff --git a/yardstick/vTC/apexlake/experimental_framework/common.py b/yardstick/vTC/apexlake/experimental_framework/common.py index f16a423fc..e86cc832c 100644 --- a/yardstick/vTC/apexlake/experimental_framework/common.py +++ b/yardstick/vTC/apexlake/experimental_framework/common.py @@ -47,7 +47,6 @@ PKTGEN_BUS_SLOT_NIC_2 = None PKTGEN_NAME_NIC_1 = None PKTGEN_NAME_NIC_2 = None -# TODO: remove Influx INFLUXDB_IP = None INFLUXDB_PORT = None INFLUXDB_DB_NAME = None diff --git a/yardstick/vTC/apexlake/setup.py b/yardstick/vTC/apexlake/setup.py index 8ab3f4845..dcd004698 100644 --- a/yardstick/vTC/apexlake/setup.py +++ b/yardstick/vTC/apexlake/setup.py @@ -5,8 +5,6 @@ Experimental Framework from distutils.core import setup -# TODO: Add instruction to compile the test_sniff - setup(name='apexlake', version='1.0', description='Framework to automatically run experiments/benchmarks ' |