summaryrefslogtreecommitdiffstats
path: root/dovetail
AgeCommit message (Collapse)AuthorFilesLines
2016-12-13[dovetail tool] support shell scripts for testcase validationLeo Wang47-150/+380
JIRA: DOVETAIL-46 1. for now a testcase has two kinds of validation types(functest, yardstick), and it is not enough to check the complete funcionality 2. add new validation type(shell) for extra validation of the test case to make result more accurate and more convincing. Change-Id: I45dca6b8dbd888757da163189d261f6e4dba5034 Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
2016-12-13bugfix for flake8 formatMatthewLi1-2/+2
error log see https://build.opnfv.org/ci/view/dovetail/job/dovetail-merge-master/58/console Change-Id: I7f5a7f290d82e79249561322259aa5e72186e126 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2016-12-13Merge "[WIP] Migrate to tox based environment"hongbo tian4-17/+21
2016-12-13[WIP] Migrate to tox based environmentLeif Madsen4-17/+21
Migrate the custom setup.py to a more formal setup environment using tox, in the same way that OpenStack configures their projects. Adds metadata to tox.ini from the setup.py file, and breaks out the dependencies from setup.py into requirements.txt and test-requirements.txt. Executes flake8 testing for pep8 coverage. Defaults to using Python 2 for the environment. Change-Id: Ieaea7d9f3dc227febd8fb1dc9142d478ba197a92 Signed-off-by: Leif Madsen <lmadsen@redhat.com> Signed-off-by: zshi <zshi@redhat.com>
2016-12-13Merge "dovetail tool: add debug option to control the log level showing in ↵hongbo tian3-16/+23
screen"
2016-12-09[dovetail tool]: move config info from .py to config .yml fileMatthewLi9-39/+48
JIRA: DOVETAIL-145 Change-Id: Ia9b3c678f1a491e5daf2d280f05904ead3b0a296 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2016-12-07Merge "Copy the jumphost ssh key into the container"hongbo tian1-2/+1
2016-12-06Copy the jumphost ssh key into the containerDan Radez1-2/+1
When functest runs against Apex it needs to ssh from the container to the undercloud to get the keystonerc file so it can authenticate to the overcloud. Change-Id: Iea3273d4d1611733f167ffab0b2a75330c413edc Signed-off-by: Dan Radez <dradez@redhat.com>
2016-12-05dovetail tool: add unit test into script unittest.shxudan1-2/+3
1. Since JIRA: DOVETAIL-55 has been merged, unit tests can't pass without load_config_files(). 2. Fix the bug by adding load_config_files(). 3. Add run_tests() into script unittest.sh to include all unit tests in tests/unit. JIRA: DOVETAIL-149 Change-Id: I7bdb0732fefde3a3f3b82441b4cc7e46ece66a1e Signed-off-by: xudan <xudan16@huawei.com>
2016-12-05dovetail tool: add debug option to control the log level showing in screenxudan3-16/+23
1. add debug option in cmd_config.yml for controling log showing in screen. 2. rename existent DEBUG as CON_DEBUG because it is used for functest/yardstick container to control the log level. 3. fix the bug in function update_config_envs(). If one key is not in system's envs, os.environ[key] get an error with ErrorKey. JIRA: DOVETAIL-148 Change-Id: If3d75109f51e2e91743b8f1134a76da42e57383a Signed-off-by: xudan <xudan16@huawei.com>
2016-12-02Merge "dovetail tool: reorganize statements in class DovetailConfig"hongbo tian2-10/+16
2016-12-02Merge "dovetail tool: add brackets to print message for consistency"hongbo tian1-1/+1
2016-12-02Merge "dovetail tool: fix unittest bugs"hongbo tian1-1/+2
2016-12-02[WIP] dovetail report improvementMatthewLi5-47/+114
JIRA: DOVETAIL-49 1)details please refer to https://wiki.opnfv.org/display/dovetail/Dovetail+Reports 2)results please refer to one example https://etherpad.opnfv.org/p/dovetail_report Change-Id: I63a1ba3b6aa1ffe5fa043a11cd3777dd0e8d52d9 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2016-11-30dovetail tool: fix unittest bugszshi1-1/+2
JIRA: DOVETAIL-143 1) fix unittest import module path 2) add create_log in setup Change-Id: Ic894fa971b4a5091affb36e8e6aac05e0e8623fb Signed-off-by: zshi <zshi@redhat.com>
2016-11-30Merge "dovetail tool: run.py execution always wipes of results files"hongbo tian2-14/+16
2016-11-30dovetail: modify process flow in function update_envs()wu.zhihui1-3/+4
bugfix task DOVETAIL-144 1. If run command "python run.py" without options, e.g. --SUT_TYPE, --SUT_IP, env values will be updated by os enviromental parameter. 2. If run command "python run.py" with options, no update from os enviromental parameter. JIRA: DOVETAIL-144 Change-Id: Id3bde7546722a5f4defe52d4f00609d7049a288b Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
2016-11-29dovetail tool: run.py execution always wipes of results filesxudan2-14/+16
1. move function clean_results_dir() into run.py 2. call this function at the beginning of function main() 3. this can only work correctly after all log initializations packaged, see JIRA: DOVETAIL-56 and gerrit https://gerrit.opnfv.org/gerrit/#/c/24059/ JIRA: DOVETAIL-80 Change-Id: I1cdda8f6d43f21c45c0537868cb7ed10d14ea9df Signed-off-by: xudan <xudan16@huawei.com>
2016-11-29dovetail tool: reorganize statements in class DovetailConfigxudan2-10/+16
1. move some statements in class DovetailConfig into function load_config_files() 2. call load_config_files() in run.py 3. this can only work correctly after moving clean_results_dir() into run.py, see JIRA:DOVETAIL-80 and gerrit https://gerrit.opnfv.org/gerrit/#/c/24929/ JIRA: DOVETAIL-55 Change-Id: I217c1c21dc6ba8849da7c00564f62d3c22cdc391 Signed-off-by: xudan <xudan16@huawei.com>
2016-11-29dovetail tool: To exec testsuite loading after command line arguments is TrueMatthewLi1-2/+2
JIRA: DOVETAIL-142 Change-Id: I1d8a444580755b4f4d9a0f7098b4b17ea66d6f58 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2016-11-29Merge "dovetail tool: change CI_DEBUG to DEBUG"hongbo tian3-5/+7
2016-11-29dovetail tool: package all logger initializationxudan5-57/+122
1. For container.py, parser.py, report.py and testcase.py, they use functions in classes to create loggers. So each class will have its own logger. 2. run.py will create all the loggers by create_logs() function before the use of them. So logs will not be initialized until the call of create_logs(). 3. Move logger in run.py into function main(). JIRA: DOVETAIL-56 Change-Id: I53de090df270cc7a34762ff99cbc9115cfb09465 Signed-off-by: xudan <xudan16@huawei.com>
2016-11-28dovetail tool: add brackets to print message for consistencyzshi1-1/+1
Change-Id: I74c0aefb7ada960aac55aed15aab6b7b9984154e Signed-off-by: zshi <zshi@redhat.com>
2016-11-28dovetail tool: change CI_DEBUG to DEBUGzshi3-5/+7
JIRA: DOVETAIL-42 1) change 'CI_DEBUG' to 'DEBUG' in cmd_config.yml and dovetail_logger.py 2) assign 'DEBUG' env value to 'CI_DEBUG' when updating functest and yardstick parameter 'envs' Change-Id: I6d6130e394fddf11d83be619f2c9684bf8c859f9 Signed-off-by: zshi <zshi@redhat.com>
2016-11-26dovetail tool: testarea argument addedMatthewLi7-43/+72
JIRA: DOVETAIL-71 1) argument testarea is added 2) terminology "scenario" change to "testsuite" 3) related documents amended 4) all word "certification" changed to "compliance" Change-Id: I038a9e04bf83cfdac20f0c59adb1841bf5470584 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2016-11-25Merge "dovetail tool: reorganize dovetail_config.py and modify related files"hongbo tian7-87/+86
2016-11-24[Dovetail Tool] Prepare env fail due to dependency on pyyamlLeo Wang1-4/+2
JIRA:DOVETAIL-92 root cause: 1. import dovetail_logger while it depends on dovetail_config 2. dovetail_config import yaml 3. yaml is not installed yet action: 1. remove dovetail_logger Change-Id: I51f5e8bab2451639a4a9ebf1978fe8a292df1704 Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
2016-11-17dovetail tool: upstream functest env preparation and CLI adaptionxudan1-2/+2
1. Functest has changed it's directory structure, so cmds in functest_config.yml cannot run successfully. 2. Use functest CLI to prepare and run test cases instead of running prepare_env.py or run_tests.py. JIRA: DOVETAIL-40 Change-Id: I7907dd70fd671efbed5f4132c0d76d6eb90b63eb Signed-off-by: xudan <xudan16@huawei.com>
2016-11-17dovetail tool: reorganize dovetail_config.py and modify related filesxudan7-87/+86
1. move all sentencs and functions into class DovetailConfig 2. delete dict container_config for it has been replaced by dovetail_config 2. modify files that import dovetail_config, including container.py, parser.py, report.py, run.py, dovetail_logger.py and testcase.py JIRA: DOVETAIL-55 Change-Id: Ib5ca3cadd751075c879f39e8cb2eb2e0adcef6aa Signed-off-by: xudan <xudan16@huawei.com>
2016-11-16Merge "Fix option handling in case of no '--tag' option."hongbo tian1-2/+2
2016-11-16Merge "dovetail tool: cleanup the result directory before running"hongbo tian1-2/+2
2016-11-16dovetail tool: cleanup the result directory before runningxudan1-2/+2
1. Jenkins will fail with error rm: cannot remove '/home/opnfv/dovetail/results': Device or resource busy 2. The reason is that Jenkins will mount this directory before running 3. The effective solution is removing all contents in this directory rather than removing this mounted directory JIRA: DOVETAIL-38 Change-Id: Ic32ea19341ad2048322ec11a4abc48e13019a7bb Signed-off-by: xudan <xudan16@huawei.com>
2016-11-15dovetail tool: replace the hard-coded "Tescase" and modify some classmethodLinghui Zeng4-34/+36
JIRA: DOVETAIL-45 1. In the testcase.py file, we replace the hard-coded "Tesecase" with "cls" or "self" based on the specific contexts. 2. For all the four committed files, we modify some classmethod into staticmethod because each method is basically just a function, called syntactically like a method, but without access to the object and its' internals (attributes and other methods). Change-Id: Ieb452f476a2d33ae9aca4c904ae7d2c92b68689e Signed-off-by: Linghui Zeng <linghui.zeng@huawei.com>
2016-11-15Fix option handling in case of no '--tag' option.Tomofumi Hayashi1-2/+2
In case of default, 'None' is in kwargs['tag'] so need to check it in option handling. The code is to fix it. Change-Id: Iaadf7e6f0f064dda819970346a810a6a8b47c74c Signed-off-by: Tomofumi Hayashi <tohayash@redhat.com>
2016-11-15Merge "dovetail tool: reorganize prepare_env.py"hongbo tian1-7/+15
2016-11-15Merge "generate report data for reuse"hongbo tian1-24/+48
2016-11-15Merge "Add "docker tag" option for each testcase containers."hongbo tian2-0/+27
2016-11-15generate report data for reuseLeo Wang1-24/+48
JIRA: DOVETAIL-22 1. report text is string, can not be reused by other module, 2. generate report data first, then this data can be uploaded to database or be translated to report text Change-Id: Ieecd87c827537c825c87a0db4ff170ae490d75ae Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>
2016-11-14dovetail tool: reorganize prepare_env.pyxudan1-7/+15
1. add main function and move statements into it JIRA: DOVETAIL-57 Change-Id: I69158fa64ffc10668fc39835de434f0b44b8754a Signed-off-by: xudan <xudan16@huawei.com>
2016-11-10Add "docker tag" option for each testcase containers.Tomofumi Hayashi2-0/+27
JIRA: DOVETAIL-69 This commit is to add new option to specify docker tags for each test containers, such as '--TAG "functest:stable"'. Change-Id: Ica11e16abfb21649a3eea9c7d7d5cd31502d2e21 Signed-off-by: Tomofumi Hayashi <tohayash@redhat.com>
2016-11-10dovetail tool: 2 config files, compliance_set and proposed_testsMatthewLi39-8/+332
JIRA: DOVETAIL-50 1)'proposed_tests.yml' is added, which runs as the total tests run. The 'compliance_set.yml' will run as dovetail certification tests; 2)ipv6 test cases added accroding to 'docs/testsuites/ipv6/ipv6_all_testcases', tc026 missed temporarily,yardstick ipv6tc026 only supports 2 installers by now according to https://wiki.opnfv.org/display/dovetail/Dovetail+Test+Cases 3)vimops001-006 added for VIM images operation test cases 4)nfvi001-002 added for vping ssh/userdata in fucntest 5)more test cases can be added in future in proposed_tests.yml, besides, the test cases can be adjusted in future in proposed_tests.yml Change-Id: I8a8634583549a802f6f64fbea651e7be4efa361c Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2016-11-09Merge "dovetail tool: ensure sub_testcase_list and testcase are not None"hongbo tian2-9/+22
2016-11-09Merge "dovetail tool: os name bugfix"hongbo tian1-3/+4
2016-11-09Merge "dovetail tool: cleanup result directory before running"hongbo tian1-3/+16
2016-11-07dovetail tool: os name bugfixMatthewLi1-3/+4
JIRA: DOVETAIL-54 bug: in some envs, return value platform_os is Ubuntu rather than ubuntu. Change-Id: I6a2aa964d00eff9f336f8204071a05a6b5f11134 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2016-11-08dovetail tool: ensure sub_testcase_list and testcase are not Nonezshi2-9/+22
JIRA: DOVETAIL-52 1) save return value of testcase.sub_testcase() in sub_testcase_list for multiple calls later 2) ensure sub_testcase_list is not called with None value 3) ensure testcase is not called with None value 4) identify None testcase with 'Undefined' in dovetail_report.txt Change-Id: Ifcf0c0846dc788575310f29932a4757752f1e3a3 Signed-off-by: zshi <zshi@redhat.com>
2016-11-03Merge "dovetail tool: Add cli mechanism for changing envs at run-time"hongbo tian5-13/+123
2016-11-03Merge "dovetail tool: rename report_file_path variable"hongbo tian1-4/+4
2016-11-03Merge "dovetail tool: Cleanup functest_conf.yml"hongbo tian1-4/+0
2016-11-03Merge "Add more distro support when preparing environment"hongbo tian1-1/+39