aboutsummaryrefslogtreecommitdiffstats
path: root/tools/report
AgeCommit message (Collapse)AuthorFilesLines
2016-03-21integration: Support of integration testcasesMartin Klozik1-4/+14
Generic support for integration testcases with first set of tests for vswitch testing. New test option "TestSteps" has been introduced to define test step by step directly in configuration file. In case that this concept will be accepted, there are plenty of possibilities for future improvements. For example: * use it also for performance tests without explicit call of validation methods * introduce step macros for repetitive scenarios, so new tests can be easily written * further generalization, which would go beyond usage of controllers and will operate directly with vswitch, vnf and trafficgen objects Change-Id: Ifad166c8ef9cfbda6694682fe6b3421e0e97bbf2 JIRA: VSPERF-212 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-03-16tunneling: Vanilla OVS decapsulation testsDino Simeon Madarang2-1/+6
Add Vanilla OVS VXLAN/GRE/GENEVE decapsulation tests Recent changes: * Line wrap * Fix GENEVE IP settings * Add GRE and GENEVE steps to integration.rst * Add DUT_NIC1_MAC and DUT_NIC2_MAC * Remove unnecessary call to vswitch.set_tunnel_arp * Move optional VXLAN/GRE/GENEVE decap settings to a different section * Add tunnel type to report files md and rst * Fix typo error: l4 to L4 in conf/integration/03_traffic.conf Change-Id: I8a43521668d5870ec893aed1df49467f0a84a07d JIRA: VSPERF-180 Signed-off-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-02-23CI: Generate test report and push logs and report to the artifactoryMartin Klozik1-4/+0
Script for CI job execution automatically generates final test report, collects log files and pushes these results into artifactory. Support for branch specific configuration file has been added. Change-Id: Ifdf13b1c4c389f8d20dbc8e0ed99f43273e0820b JIRA: VSPERF-181 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com>
2016-02-11bugfix: Blank lines are required to join RST templatesMartin Klozik1-0/+4
Blank lines are required to correctly join multiple RST templates. Otherwise their text will be merged into one paragraph and formatting tags will be ignored. Change-Id: I6885019f7af2fd2201b9f30496d78200a0432e8e JIRA: VSPERF-181 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-02-05docs: fix link to the page with test resultsMartin Klozik2-2/+2
Test report publishes link to the page with test results. Link has been fixed to point to the correct page. Change-Id: I12f2959dffdc081dda9ef15333f9ea937b56eb35 JIRA: VSPERF-181 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-02-05docs: adding license to all rst files.Maryam Tahhan1-0/+4
Add Creative Commons Creative Commons license to all documentation in vsperf. JIRA: VSPERF-239 Change-Id: I88721205a36078f6e0d4a4c5dd842907729d2b6b Signed-off-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-02-03Add simple VxLAN decapsulation performance testDino Simeon Madarang1-5/+11
Measure OVS DPDK VXLAN decapsulation performance. The DUT is configured as a VTEP (VXLAN Tunnel Endpoint) which performs decapsulation of frames and sends traffic to the 2nd traffic generator port. This test is unidirectional. Recent changes: * Userguide update * Move src/dstport from L2 to L4 (conf, TCL) * Fix 'TestCase' object has no attribute '_tunnel_operation' error when running non overlay tests * Fix merge conflict * Move doc to docs/userguide/integration.rst * Add setting of DUT_NIC1_MAC and setting of other variables to docs/userguide/integration.rst Change-Id: Ia44f8888ef727831543c80f5c98c866686c5c92c JIRA: VSPERF-190 Signed-off-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com> Signed-off-by: Robert Wojciechowicz <robertx.wojciechowicz@intel.com> Signed-off-by: Timo Puha <timox.puha@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com
2016-01-28docs: Add a note on pidstat and unknown results.Maryam Tahhan1-2/+8
Add a Note on why %CPU and %USER > 100%, as well as a note regarding the "unkown" results. Change-Id: I7e173c91a549932301645c9344648c9b73da4a70 Signed-off-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com>
2016-01-28docs: Final report conclusions and recommendationsMaryam Tahhan2-4/+9
Remove TODO sections from the Release B report. Change-Id: I4520c23ad9cbb980c9f1c1ee14cc7e91060933a4 Signed-off-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Martin Klozik <martinx.klozik@intel.com>
2016-01-28bugfix: do not report DPDK version for vanilla OVS vswitchMartin Klozik3-1/+7
DPDK version check has been removed in case that OVS does not use DPDK. Change-Id: I78560a72e4fde3d7113165d88eb81d80f6399af8 JIRA: VSPERF-209 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2016-01-25report: create test report for all executed TCsMartin Klozik3-109/+98
Every testcase generates report with its results into separate file. It is required to merge all partial reports into final overall report for whole set of TCs executed by VSPERF. Hugepages are mounted also for TestPMD packet forwarder. Change-Id: Iaa1ab4e08d1637106da804e06b7f30100d609cd6 JIRA: VSPERF-181 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com>
2016-01-22report: add rst template for results reportingMartin Klozik3-19/+222
Add a template that can be used for reporting vsperf release B results. This patch also fixes the line length issues in the markdown template. Change-Id: Id03f5c7eb84536697deca73d3fb397629b647a85 JIRA: VSPERF-181 Signed-off-by: Maryam Tahhan <maryam.tahhan@intel.com> Signed-off-by: Martin Klozik <martinx.klozik@intel.com>
2016-01-19reporting: add vswitch, vnf and trafficgen version into the reportMartin Klozik2-16/+34
Final test report MD file should contain information about version of vswitch, vnf, VM loopback forwarding application and traffic generator used during the test. In case that component is cloned from GIT repository, then hash of its recent commit should be part of the report too. Change-Id: I4eb398bc95bc5030d0852d08bcf9febbf17640d4 JIRA: VSPERF-172 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Radek Zetik <radekx.zetik@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com> Reviewed-by: Tv Rao <tv.rao@freescale.com>
2015-12-14bugfix: Fix failure caused by unset localeMartin Klozik1-1/+1
In case, that VSPERF detects missing locale settings, it will set language and encodding settings to default values specified by configuration parameter DEFAULT_LOCALE. Localized external commands with output parsed by VSPERF are executed with modified locale to ensure correct VSPERF function. Locale settings for such commands is specified by configuration parameter DEFAULT_CMD_LOCALE. Change-Id: If5c15115b778ce90046e390f10438b780f82695b JIRA: VSPERF-132 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com> Reviewed-by: Gurpreet Singh <gurpreet.singh@spirent.com> Reviewed-by: Tv Rao <tv.rao@freescale.com>
2015-10-21Add Pylint to VSPERF commit gateMartin Klozik1-1/+1
Initial pylint config file has been created to follow current code structure and design. A few issues have been fixed, so currently 56 of 67 files have rating 10/10. There are still issues to be fixed, mainly TODOs/FIXMEs and missing impelentation of abstract class members. Change-Id: Icf5b12c2150258f8ac657f8d50b47a697d1660a0 JIRA: VSPERF-91 Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Fatih Degirmenci <fatih.degirmenci@ericsson.com> Reviewed-by: Gene Snider <eugene.snider@huawei.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Tv Rao <tv.rao@freescale.com> Reviewed-by: Brian Castelli <brian.castelli@spirent.com>
2015-09-29Sysmetrics implementation updateMartin Klozik2-28/+23
New sysmetrics implementation is based on pidstat command line tool from sysstat package. Old non-functional implementation was removed. Reporting was refactored to generate report after each TC from values already available in memory. Following files were affected: modified: conf/01_testcases.conf modified: conf/02_vswitch.conf modified: conf/05_collector.conf deleted: core/collector_controller.py modified: core/component_factory.py modified: docs/NEWS.rst modified: packages.txt modified: requirements.txt modified: testcases/testcase.py modified: tools/collectors/collector/collector.py modified: tools/collectors/sysmetrics/__init__.py deleted: tools/collectors/sysmetrics/linuxmetrics.py new file: tools/collectors/sysmetrics/pidstat.py modified: tools/report/report.jinja modified: tools/report/report.py modified: tools/systeminfo.py modified: vsperf JIRA: VSPERF-67 Change-Id: I25a79f2afef405b9ac46ae85c18044af167a62a4 Signed-off-by: Martin Klozik (martinx.klozik@intel.com) Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Gurpreet Singh <gurpreet.singh@spirent.com> Reviewed-by: Tv Rao <tv.rao@freescale.com>
2015-08-19Initial reporting implemenationMartin Klozik3-0/+274
Reporting from TOIT was merged and improved. Default template was modified to support any testcase and to show more details about system environment. Affected files: * docs/NEWS.md * testcases/testcase.py * tools/report/__init__.py * tools/report/report.jinja * tools/report/report.py * tools/systeminfo.py * vsperf JIRA: VSPERF-71 Change-Id: I4dc84ca69e5c292eae1f8dede1411c06ae3ef8af Signed-off-by: Martin Klozik (martinx.klozik@intel.com) Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Billy O Mahony <billy.o.mahony@intel.com>