summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2017-02-15Normalize data seriesmbeierl2-13/+43
Changes the data series from using seconds to samples so that slope is not artificially flattened. Change-Id: Idf87926a47c2ba67e66e2254d3572adad7a81b44 JIRA: STORPERF-106 Signed-off-by: mbeierl <mark.beierl@dell.com>
2017-02-01Add URL for results reportmbeierl6-22/+42
Records the URL returned from testresults db. Changes the URL ref from localhost to external and reports in daily job. Change-Id: I0068ea963671fb183779ac20592ba6933647eea0 JIRA: STORPERF-104 Signed-off-by: mbeierl <mark.beierl@dell.com>
2017-01-31Fixing double JSON encoding of results payloadmbeierl2-5/+5
Change-Id: I5e5a8180d5851630b1200ade84f6beb5dfa18466 Signed-off-by: mbeierl <mark.beierl@dell.com>
2017-01-30Make it easier to see what's happening in Jenkinsmbeierl4-11/+32
Change-Id: I3ac727faa98c1e87dcd9522c0063ca6b251c62b4 Signed-off-by: mbeierl <mark.beierl@dell.com>
2017-01-27Testresults DB Alignmentmbeierl5-51/+110
Changes the key names sent to the testresults db to align with other projects. Defines a clear PASS/FAIL based on the ability to find steady state for all runs. If one run failed to achieve steady state, run gets labelled as FAIL. Change-Id: I73143779453689eea93829079388ed8035134aa3 JIRA: STORPERF-100 Signed-off-by: mbeierl <mark.beierl@dell.com>
2017-01-27Dump env so I can see what variables are visible once the job is run under ↵mbeierl1-1/+7
Jenkins Change-Id: I576a75864fa0d7c6d12990c91984e12b1489258f Signed-off-by: mbeierl <mark.beierl@dell.com>
2017-01-27Fixed line to long errormbeierl4-3/+4
Fixes the version of requests to match what docker was already pulling in. Truncate log message to 512 characters. Change-Id: Idba9374c0f339c8b4cb6902c921c8b26fef573cc JIRA: STORPERF-103 Signed-off-by: mbeierl <mark.beierl@dell.com>
2017-01-26Pin version of setuptoolsmbeierl2-1/+2
Forces explicit version of setuptools to be used because of an upstream change: https://github.com/pypa/setuptools/issues/937 Change-Id: I4124dedee38713ca1bfa445944d8d79cf1be6c15 JIRA: STORPERF-102 Signed-off-by: mbeierl <mark.beierl@dell.com>
2017-01-26Hardening FIO interactionmbeierl11-72/+136
Fixes a problem where FIO does not terminate by scheduling a second killall if we get a specific message back from FIO stderr. Introduces a new flavor for StorPerf that has a little more memory as larger memory maps for duplicate blocks sometimes caused out of memory killer to be invoked. Change-Id: I06856561ad73fef582a81d4136a36a1bea47654a JIRA: STORPERF-99 Signed-off-by: mbeierl <mark.beierl@dell.com>
2017-01-20Fix multiple workload runsmbeierl5-59/+73
Change reporting so that multiple workloads in one job execution can be reported instead of overwriting the previous value. Change the daily job to use a single, multiple workload run. Change-Id: I8e350350ae13d2272b584af7a60ad269de160587 JIRA: STORPERF-98 Signed-off-by: mbeierl <mark.beierl@dell.com>
2017-01-20Correct duplicate event notificationmbeierl1-0/+5
Clears the listener list on start and remove listener after run is completed. Change-Id: I76e1f13379d60bdb72fcf2faf57d9762a62f17d3 JIRA: STORPERF-96 Signed-off-by: mbeierl <mark.beierl@dell.com>
2017-01-20Change to Ubuntu 16.04mbeierl2-9/+9
Change the base image used for the agent VM to Ubuntu 16.04 as the 14.04 image has kernel panic when the Cinder Volume is detached Change-Id: I37e41939826fbc7c1efcc4360b99b1a2b34b8d11 JIRA: STORPERF-97 Signed-off-by: mbeierl <mark.beierl@dell.com>
2017-01-18Actually check out releng locallymbeierl1-0/+4
Change-Id: I50358e0685be45abd9330639fcdfa9c8a39ff5ba Signed-off-by: mbeierl <mark.beierl@dell.com>
2017-01-18Fixing up paths and checking out releng locallymbeierl2-2/+2
Change-Id: I62d877acc37a10391e59db1611c0a0af3d57fca3 Signed-off-by: mbeierl <mark.beierl@dell.com>
2017-01-18Fixed typombeierl1-1/+0
Change-Id: Iedc8f023636325dee664bfafab45381997d6403a Signed-off-by: mbeierl <mark.beierl@dell.com>
2017-01-18StorPerf Daily Jobmbeierl6-37/+97
Helper script that determines what installer was used. Script to generate RC file for container. Script to determine number of Cinder nodes. Daily job to pull new container, create stack based on number of Cinder nodes, issue a warm up, and then run a series of performance tests against the stack. Deletes stack once complete. JIRA: STORPERF-87 Signed-off-by: mbeierl <mark.beierl@dell.com> Change-Id: I551f21ffc85f11b0c0724205e952551022a94f5b
2017-01-17Expose TENANT_IDmbeierl1-8/+9
Change-Id: I2b07bb9724b19f8b25e35160fba3bf11621a30d7 JIRA: STORPERF-93 Signed-off-by: mbeierl <mark.beierl@dell.com>
2016-12-16Use v3 Authmbeierl4-34/+45
Change to detect environment variable and use v3 authentication instead if present. Change-Id: Iada49741b3297f20b740e232b2b35bec92fda91a JIRA: STORPERF-88 Signed-off-by: mbeierl <mark.beierl@dell.com>
2016-12-06Allow 0 to match steady state and don't terminate the warmupmbeierl2-10/+11
Change-Id: I61f8ce4e719e2c2ef7597c0e41aff8c8d79edc2d Signed-off-by: mbeierl <mark.beierl@dell.com>
2016-12-06Fixed typombeierl1-1/+1
Change-Id: I82cf026017690436a1112daabfa0b084275d8fe7 Signed-off-by: mbeierl <mark.beierl@dell.com>
2016-12-05Steady state detectionMark Beierl17-63/+447
Detection of steady state after 10+ samples of data Change-Id: I29368b06188c6370d17b3d567fece6486d171235 JIRA: STORPERF-72 STORPERF-73 Signed-off-by: Mark Beierl <mark.beierl@dell.com>
2016-12-03Merge "Change to Paramiko"Mark Beierl7-100/+82
2016-12-02Change to ParamikoMark Beierl7-100/+82
Use Paramiko ssh client instead of invoking ssh and scp from the command line Change-Id: Ibc8395b98842fd7f40b49c4dafa2688d8e64abc7 JIRA: STORPERF-91 Signed-off-by: Mark Beierl <mark.beierl@dell.com>
2016-11-30bugfix: Add BRANCH Argument in the DockerfileMatthewLi1-2/+2
Change-Id: If5ccbe0633ed5331b38d8231e0dd6eadd05ecbe8 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2016-11-30Data Handling RefactoringMark Beierl4-56/+185
Break out test db interaction into new module and make the push event driven instead of the sleep that was there before. Change-Id: I9485aba1405f6c3b4ee5000168fbc037efa87c81 JIRA: STORPERF-90 Signed-off-by: Mark Beierl <mark.beierl@dell.com>
2016-11-30Add BRANCH Argument in the Dockerfilejose.lausuch1-2/+4
This patch allows: - clone the respective repo branch while building the image - slightly reduce the size of the image by cloning only the latest Change-Id: I48994d7d9cf230493fa22f39cf1ed7ef4fd72d9e Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2016-11-25Data Reporting GateMark Beierl3-0/+118
Module that allows passing of a gate only once all other peers have also reported in within a specified time period Change-Id: If4baf1d4377026c7833a6f30bfc2e36698f675e8 JIRA: STORPERF-71 Signed-off-by: Mark Beierl <mark.beierl@dell.com>
2016-11-25Daily JobMark Beierl8-18/+271
Creation of a daily job script that runs under Jenkins and kicks off a series of tests using the freshly cloned workspace. Change-Id: Ibc63c1df954578ad78604321ea410c8fd8c63c41 JIRA: STORPERF-87 Signed-off-by: Mark Beierl <mark.beierl@dell.com>
2016-11-23Ability to query job statusMark Beierl3-8/+26
Provide a method to query the status of a job by id Change-Id: I4d3567995e7c7c3359596346009358276e79d65c JIRA: STORPERF-89 Signed-off-by: Mark Beierl <mark.beierl@dell.com>
2016-11-22Wait for agent statusMark Beierl1-0/+21
Test the agent's ssh port prior to proceeding with test Change-Id: Iff65f2c12a50063aa9f2ee4b75638ec0666ded1b JIRA: STORPERF-34 Signed-off-by: Mark Beierl <mark.beierl@dell.com>
2016-11-11Additional CoverageMark Beierl2-0/+3
Adds HTML output to coverage so we can see it in Jenkins Change-Id: I89595921bf86b8963890051a25e45f7bd08b0636 Signed-off-by: Mark Beierl <mark.beierl@dell.com>
2016-11-10Create Stack FixesMark Beierl3-24/+32
Removes check stack as that can timeout when a large number of agents are requested Waits for stack to be created before returning JIRA: STORPERF-70 Change-Id: I1cbe45b1f60f0ecab0691f0822ed5643778d218b Signed-off-by: Mark Beierl <mark.beierl@dell.com>
2016-09-16Installation Guidembeierl5-0/+241
Populate the installation guide Change-Id: I271ee6d5a38195aaf95a9c821bee8049470d674f JIRA: DOCS-124 Signed-off-by: mbeierl <mark.beierl@emc.com>
2016-08-24Slope errorMark Beierl2-2/+8
Test and fix for error in slope STORPERF-74 Change-Id: I17a8e0e1e588deb875ecace85290f92167df1d5e Signed-off-by: Mark Beierl <mark.beierl@emc.com>
2016-07-15Add Steady State Detection moduleTim Rault8-38/+337
Added a Steady State Detection module containing a steady_state(data_series) function that is able to return a boolean indicating wether or not steady state is reached with the data_series being passed. This module requires a data_treatment(data_series) and an average(data_series) modules that have been added in this commit as well. The data treatment function aims at formatting the data series that is passed to the high level steady_state function to reach the requirement of each sub-module (slope, average and range). Modified the Slope and Range functions so they return None when passed an empty data series instead of 0 which was wrong. Modified the corresponding test cases. Modified the math_range_test.py file to fix a bug in the 2 last tests. Change-Id: I9c3854cb0a21cc37b0787b8afca0821eefaa203d JIRA: STORPERF-60 JIRA: STORPERF-59 JIRA: STORPERF-61 JIRA: STORPERF-62 Signed-off-by: Tim Rault <tim.rault@cengn.ca>
2016-07-14Separation of test and sourceMark Beierl16-1/+1
Moving the test files into their own top-level directory to keep things clean Change-Id: Ic50b881045bc59b003807923424345b335dd5c95 Signed-off-by: Mark Beierl <mark.beierl@emc.com>
2016-07-14Enforce Style GuidelinesMark Beierl3-8/+14
Update to fail verify if pep8 compliance not met. Revert back to Coverage 4.1 to show code coverage report in Jenkins again Change-Id: Icb467451f101942255c7944ad796497609dcb07e Signed-off-by: Mark Beierl <mark.beierl@emc.com>
2016-07-14Merge "Fix missing test"Mark Beierl1-0/+0
2016-07-14Add Range function for Steady State detectionTim Rault3-43/+200
Added a range_value function in utilities/math.py able to compute the range of a series of y values : [y1, y2, ..., yn]. Implemented a test harness for this range_value function in the tests/utilities section. Renamed the math_slope.py and math_range.py test files to add _test.py for Jenkins. Cleaned up the code so it is compliant to the pep8 rules. Renamed the previous 'math' modules (storperf/utilities/math.py and storperf/test/utilities/math.py) as 'math_slope' to be coherent with the new notation. Change-Id: I02ccd2b87f0b72e7a28c416b593aae4d8ad97961 JIRA: STORPERF-57 JIRA: STORPERF-58 Signed-off-by: Tim Rault <tim.rault@cengn.ca>
2016-07-14Fix missing testMark Beierl1-0/+0
All test files must end in _test.py or they don't get executed Change-Id: If3a46a66570f7d07d0e93de72438a8a46b3e0a22 Signed-off-by: Mark Beierl <mark.beierl@emc.com>
2016-07-06Add Least Squares slope functionTim Rault2-0/+118
Added a math module in utilities that contains a slope function able to compute the slope of the best Least Squares curve fit given a series of [x,y] values. Implemented a test harness for this math module in the tests/utilities section. Change-Id: If4d63af092d0904b2269c5ee0991e18ab84533c0 JIRA: STORPERF-54 JIRA: STORPERF-55 JIRA: STORPERF-51 Signed-off-by: Tim Rault <tim.rault@cengn.ca>
2016-06-16Force a compatible version of CoverageMark Beierl1-1/+1
LiClipse / PyDev only works with 3.4 Change-Id: I351df5f8120889af07b10ebb7a740d1e318cb1c2 Signed-off-by: Mark Beierl <mark.beierl@emc.com>
2016-05-26Autocreate the key pairMark Beierl1-5/+9
Change-Id: Ib4b8cb92b4cbc7dd778363335b6d0e22ae1993fb Signed-off-by: Mark Beierl <mark.beierl@emc.com>
2016-05-20Standardize the ReST APIMark Beierl3-14/+14
Change-Id: I004824711abf17f06781302cf48a0b0e3d095ecb Signed-off-by: Mark Beierl <mark.beierl@emc.com>
2016-05-13Fixed logic indendation bugMark Beierl1-1/+2
Change-Id: I6cb17c699ae3b88a591278da23d60dfc0da91e40 Signed-off-by: Mark Beierl <mark.beierl@emc.com>
2016-05-12Fixed spacing for swagger docsMark Beierl1-9/+9
Change-Id: I994153d03981f9b706834ab2ab09196e235cc7ce Signed-off-by: Mark Beierl <mark.beierl@emc.com>
2016-05-12Results DB ReportingMark Beierl11-15/+217
Add the ability to push results to the test result db. If the environment variable TEST_DB_URL is defined, results will be pushed there. If not, no push is attempted. Change-Id: Ib833530d7379c5f37f0d2904a83d31a4ee559ae6 JIRA: STORPERF-13 Signed-off-by: Mark Beierl <mark.beierl@emc.com>
2016-05-11Max test durationMark Beierl3-7/+48
Add a maximum time limit for any one given test so that silly mistakes like block size 512, queue depth 1 does not end up running for 2 days JIRA: STORPERF-53 Change-Id: I1f3e5046e1dd346fc5b28a7b935df2ee9aa6ba3d Signed-off-by: Mark Beierl <mark.beierl@emc.com>
2016-05-11Job Run MetadataMark Beierl3-12/+43
Add the ability to store arbitrary metadata about the job for later reporting Change-Id: If7f1d0a69111567d69db865baec6c20f9a1f494f Signed-off-by: Mark Beierl <mark.beierl@emc.com>
2016-05-10Fix double loop bugMark Beierl2-6/+8
Change-Id: I47f20dec309d7ab8402f40f6cf09828aa134e902 Signed-off-by: Mark Beierl <mark.beierl@emc.com>