Age | Commit message (Collapse) | Author | Files | Lines |
|
JIRA: DOVETAIL-634
The OVP web port needs to read from the test results if the strict API
response validation exemption was used or not and show it
correspondingly.
Detailed description:
As discussed on the Dovetail call on 2018-04-11, the web portal should
show if API response validation was enabled or disabled. The information
if API response validation was enabled or disabled is to be determined
based on the dovetail.log file. If API response validation was
disabled,** the log file contains the line "Strict API response
validation DISABLED.". Upon uploading a result tar-package, the web
portal should extract this information from the log file, store it in
the database, and display it as follows:
This indication shall be displayed along with the test summary on the
Test Run Results page as either:
"API response validation enabled" or
"API response validation disabled"
For clarification: API response validation is enabled by default. API
response validation is disabled only if the --no-api-validation command
line option is given.
Change-Id: Ifacb1eb2127cc8dc6364c98dead1217aeedfd359
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
The original refstack results file is refstack/refstack.log
For Fraser, it has been changed into refstack/tempest.log
Change-Id: I7d40abe0c5a366b5fbb1912ec58d15109f22c0d0
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Use Functest Fraser docker images whose tag is fraser
2. Use Yardstick stable docker images
3. Functest fraser has moved testcases.yaml file from functest/ci to xtesting/ci
4. Functest fraser has removed 'functest env prepare' command
5. Functest uses rally to execute osinterop test cases
instead of refstack_client
Change-Id: I81ebefe6ea959ca77c2c87f1d75ef56ac18c165f
Signed-off-by: xudan <xudan16@huawei.com>
|
|
This patch adds the ability to Dovetail to disable strict API
response validation in Tempest-based test cases run by Functest.
Corresponding updates of the web portal and backporting of this patch to
OVP 1.0.0 (Danube) will follow.
JIRA: DOVETAIL-633
Change-Id: Iace99ea1b6224ea907a2c3af8676e9285e6ad3ee
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
- Add Functest healthcheck tier to Dovetail as new test suite
- Intention is that these test cases are not part of mandatory
test cases in compliance program
- Use for debug and SUT pre-flight check
- May consider running tc001 (connection_check) at other
test execution stages to determine if SUT is still functioning, as
this test takes < 10 sec (more important as test complexity increases
such as in vnf test tier)
- Replacement for patch:
https://gerrit.opnfv.org/gerrit/#/c/52461/
- Removed CI filters, as suggested
Change-Id: I2e72931b27e43359d6250f6959a17af8526c5094
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
|
|
There are some special Docker setting items when creating
Yardstick Docker Containers.
1. Remove the extra settings which are required by Yardstick Danube.
2. Remove the settings for pushing results to DB because Dovetail
plan to disable this function.
3. Move the other special Functest settings to config files.
JIRA: DOVETAIL-640
Change-Id: Ic2d90393e227d7886b14d805eaa307b7aef56e4a
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Currently each type of test cases will use one docker image.
2. For example,
Functest test cases use opnfv/functest-restapi:euphrates
Yardstick uses opnfv/yardstick:opnfv-5.1.0
3. It needs to support to specify the docker image for each test case.
4. For example,
vping test cases use opnfv/functest-smoke:euphrates
sdnvpn uses opnfv/functest-features:euphrates
5. The benefit of this is that these docker images are well verified
by Functest CI and from Functest plan, they may not support
functest-restapi docker image in its future release.
JIRA: DOVETAIL-638
Change-Id: I8a30eea2233aeff809af2e241b6c939194397832
Signed-off-by: xudan <xudan16@huawei.com>
|
|
This patch was needed to modify the timeout values of the HA test cases
in Yardstick for the Danube release. In Euphrates, the timeout values
have been fixed upstream, so the patch is not needed anymore. The code
for applying the patch was already removed in
https://gerrit.opnfv.org/gerrit/#/c/51997/
Change-Id: I67cceb57c063378d77f6cbc720bc6464d2be4aed
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
Change-Id: Ib2aae63dd7576a8458f068250e9b6dda8678a1fd
Signed-off-by: rpaik <rpaik@linuxfoundation.org>
|
|
There are some special Docker setting item when creating Functest Containers.
1. Remove the extra settings which are required by Functest Danube.
2. Remove the settings for pushing results to DB because
Dovetail plans to disable this function.
3. Move the other special Functest settings to config files.
JIRA: DOVETAIL-639
Change-Id: Ibc4090b5c0aafdf9ed64e65a9a92e34a12a16df5
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change-Id: I6a1fa5f27df2f5127eb00156b3135dc79caf5e77
Issue-ID: DOVETAIL-629
Signed-off-by: Moshe <moshehoa@amdocs.com>
|
|
This document describes an exemption process for disabling strict API
response validation.
JIRA: DOVETAIL-617
Change-Id: I690efdcae7dd15a6c0781a837a62677904836fca
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
1. The original testcases.yaml file of Functest restricts sdnvpn test cases
to run on Fuel, Apex or Netvirt on bgpvpn scenario.
https://github.com/opnfv/functest/blob/master/functest/ci/testcases.yaml#L278
2. Dovetail has a testcases.yaml file to allow sdnvpn test cases run on
any commercial SUTs.
https://github.com/opnfv/dovetail/blob/master/etc/userconfig/testcases.yaml#L113
Change-Id: I0d5ed5b22e55dc3dc0fe584dc57b28239334b7de
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change-Id: Ie470d03bd168c909fd02c316523049c67a6bb095
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
|
|
The directory of userconfig has changed from
/home/opnfv/dovetail/dovetail/userconfig to /etc/dovetail/userconfig.
Change-Id: Id3e8b1f3e9299e9263a4adf3456b6733fdb62e11
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Many HA test cases try to kill special processes with their name.
2. Then they check corresponding openstack services and
the status of processes' recovery.
3. For different SUTs, the names of the processes and the hosts' names
may be different.
4. So there is a requirement for Dovetail tool to provide one way to
allow users to specify all the infos about process name and its host name.
5. All process infos can be added into file DOVETAIL_HOME/user_config/pod.yaml
6. The infos include 'attack_process' and 'attack_host' for each HA test cases.
7. If not given in this file, will use Yardtsick default values.
JIRA: DOVETAIL-627
Change-Id: I83cee991f72a8685080ed562597c70d73002623a
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Specifying separate valid docker tags for functest and yardstick for
input validation.
DOVETAIL-592
Change-Id: I3365cdbd9ab521bf1368750c3f34b6863b3f7ec8
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
The function 'check_tc_result' may return 'None'.
In this case it will throw an exception because of
https://github.com/opnfv/dovetail/blob/master/dovetail/run.py#L65
The exception is like
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/dovetail/run.py", line 339, in <module>
main()
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 716, in _call_
return self.main(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 696, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 889, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 534, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/dovetail/run.py", line 304, in main
duration = run_test(testsuite_yaml, testarea, logger, kwargs)
File "/usr/local/lib/python2.7/dist-packages/dovetail/run.py", line 65, in run_test
if stop_on_fail['criteria'] == "FAIL" and kwargs['stop']:
TypeError: 'NoneType' object has no attribute '_getitem_'
JIRA: DOVETAIL-622
Change-Id: I9e612b380c3dfb148692074a3bf8eb31875213ac
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change-Id: I89d15e18c588c27ab41596119d5288cba6e4e02d
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
|
|
1. mv conf/ userconfig/ compliance/ patch to etc/
2. after installation etc/ will be put under /etc/dovetail
3. leveraging /etc/dovetail in coding
Change-Id: Icf9cb00ff7fc7cf35755f60d3e258b23e2c70839
Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
|
|
Adding the SDNVPN Tempest API test cases of the SDNVPN project to the
proposed_tests suite to allow for running these tests in the CI pipeline for
validating their suitability for inclusion in the OVP Euphrates release.
JIRA: DOVETAIL-608
Change-Id: Ic12db1584a2540a14979094c65fc128c6020ecea
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
Change-Id: Ic880258ef8ffbcbdcf3452419dd385227567a5a3
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
|
|
If the hosts.yaml file is like,
—
hosts_info:
192.168.141.101:
it will raise an exception,
File "/usr/local/lib/python2.7/dist-packages/dovetail/utils/dovetail_utils.py",
line 227, in add_hosts_info
names=hostnames)
File "/usr/local/lib/python2.7/dist-packages/python_hosts/hosts.py",
line 54, in _init_
raise Exception('Address and Name(s) must be specified.')
Exception: Address and Name(s) must be specified.
If the hosts.yaml file is empty, it will crash with,
File "/usr/local/lib/python2.7/dist-packages/dovetail/container.py",
line 174, in create
if hosts_yaml['hosts_info']:
TypeError: 'NoneType' object has no attribute '_getitem_'
JIRA: DOVETAIL-621
Change-Id: I547928e514b3294c048379459c07df89879fbb03
Signed-off-by: xudan <xudan16@huawei.com>
|
|
The INFO file has been replaced by the INFO.yaml file.
Change-Id: I818697134cbb380a318237fdf593d7b6bf95ec04
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
https://lists.opnfv.org/pipermail/opnfv-tech-discuss/2018-February/020497.html
Change-Id: Ic2b536551d919e9b66f41ba00dd6dc8b71d93f7b
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
- Uses extra files config.env, vhost.env to hold environment variables
for docker configuration.
- Provides samples for these config that can be copied and modified to
fit deployment requirements.
- Updates the docker images to the newest releases.
Change-Id: Ibe9b8597edfb38940297433708497c244200d0af
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
|
|
Bottlenecks stress test needs some adaptions about Bottlenecks
Euphrates release.
JIRA: DOVETAIL-587
Change-Id: Ia447143c8f08924ac1b190ee0173f39e462c5dfa
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change-Id: I06b1e53bca44f5486a64977edef8b0e53072c625
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
Add cloudify_ims test case into proposed_tests test suite.
Then this test case can be run by Dovetail on CI.
JIRA: DOVETAIL-603
Change-Id: I5ab832002388f0916e04beef9d630de2cb37df59
Signed-off-by: xudan <xudan16@huawei.com>
|
|
- Update the OPNFV Verified logo for OVP portal home page
- Update branding guide link in OVP portal left side navigation
Change-Id: I77ead3e68b34d30d55fd07f0e4a176c5ae71852e
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
|
|
Contributions Dan Xu:
Dan Xu has been almost exclusively doing the development for the
Dovetail tool, CI integration, and the practical validation of test case
candidates.
https://gerrit.opnfv.org/gerrit/#/q/owner:%22Dan+Xu+%253Cxudan16%2540huawei.com%253E%22
Leo Wang has announced that he is stepping down as committer from the
project. Thank you for your contributions to the project.
https://lists.opnfv.org/pipermail/opnfv-tsc/2018-February/004130.html
Change-Id: I14032657b057bdd3da1f9c3514b3c27e34dbafc0
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
Add 3 new HA test cases into proposed_tests test suite and running them on CI.
For the current 8 HA test cases and 3 other HA test cases plans to add,
do some adaptions for the Yardstick Euphrates release.
1. Don't need the bugfix patch anymore.
2. The data format has changed, and has done some adaptions.
3. For Yardstick Euphrates, if setting OS_INSECURE=True, it needs to set
OS_CACERT=False
4. Test case tc011 needs a image and flavor to create the stack.
JIRA: DOVETAIL-614
JIRA: DOVETAIL-586
Change-Id: I9da8c8316246b9edee0d81ad18422ae6018becaf
Signed-off-by: xudan <xudan16@huawei.com>
|
|
1. Currently it won't pass the cacert file to other containers if the
OS_AUTH_URL is http not https.
2. However, even though the OS_AUTH_URL is http, it still needs cacert file
somewhere for Functest and Rally.
3. This problem is found on Fuel Euphrates.
4. Add a function to check whether the cacert file exists.
5. If exists, pass it to testing project containers.
JIRA: DOVETAIL-616
Change-Id: Ied7bcc72e8f1c738bbce32c18096ca13641d3cd7
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Add snaps_smoke into proposed_tests test suite.
Then this test case can be run by Dovetail on CI.
JIRA: DOVETAIL-596
Change-Id: I8fc4bb656bcac551ae4b693dbc4d817b665c6111
Signed-off-by: xudan <xudan16@huawei.com>
|
|
- Improve test execution control by adding stop on failure flag
- Modify test check function to return value regardless of whether
report or local file option is employed
- Modify run test loop to check for failure after every test case
if stop flag set
Change-Id: I27a4f3aaaa8b8cb8c52eb3f092f52048980abb44
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
|
|
INFO.yaml meant to be machine readable.
This file was auto generated from an ldap lookup
and your projects INFO file.
Meeting times are not currently filled out.
please ammend this patchset to complete
Going forward comitter promotions should be done
against this file.
PTLs:
Must be merged for you to gain delete access on
testresults.opnfv.org
If project_lead: is missing (my script could not parse)
please amend with same from commiters section eg:
project_lead: &opnfv_dovetail_ptl
name: 'Lead Name'
email: 'lead email'
id: 'lead lfid'
company: 'lead company'
Change-Id: Ie151ba96dd6b6c7d0d8e74e1462e35133a7c235f
Signed-off-by: agardner <agardner@linuxfoundation.org>
|
|
- EXTERNAL_NETWORK env variable can be explicitly set
- If set, the value will be used in Yardstack configuration
- Otherwise, previous method of using output from
'openstack network list' will still be employed
Change-Id: I9d8b4a69ff3f5093c034823cc95e92e151bfade6
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
|
|
- Try new license header to deal with anteater failure
- Ability to specify 'default' as argument to --report, which
fetches host IP automatically and forms URL with 8000 as testapi
port
- This should not break the previous behavior of report option,
if default option not specified
Change-Id: I1e05bb52cf4fc211c59796d4e763e2e04177b406
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
|
|
Add all tempest smoke test cases into debug test suite.
Then these test cases can be run by Dovetail on CI.
The results of CI will help to choose sub test cases from tempest smoke.
Here use tempest_custom to run all tempest smoke test cases because it can
support to run a custom defined sub test cases list.
for Functest tempest_smoke_serial, it always runs whole tempest smoke
test cases and can't be a subset.
JIRA: DOVETAIL-595
Change-Id: Id6358691d2bdef149f16775ce84dcfbf12f5e568
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Use opnfv/functest-restapi:euphrates.
Adapt Functest E framework for all Functest test cases.
1. All path of functest and sdnvpn has been changed.
2. Sdnvpn can clean resources by itself with E release, no need to clean again.
3. Add a file testcases.yaml to overwrite the file in functest container,
then it can support to run tempest_custom i.e. ipv6 and tempest in OVP.
JIRA: DOVETAIL-584
Change-Id: I3ba9aa8e80becaa99cc89c8db4b39acbb89dfda4
Signed-off-by: xudan <xudan16@huawei.com>
|
|
Change-Id: Ieb32dbda8fd16942312900cb99a27f734e57d91b
Signed-off-by: xudan <xudan16@huawei.com>
|
|
- Fix typos and word changes
- Remove editor's note
- Adopt rewording suggestions
- Add back toc that was inadvertently deleted
- Modify to reflect process used during beta and current OVP web portal
usage
- Remove reference to application form and use online participation form
instead. Add new fields required in participation form
Change-Id: Ia51b7674315467e01081015b3aa780ec03652ebd
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
|
|
If we upload a new test, we can't modify the SUT version.
The reason is we must predefine the model.
Change-Id: I8822b078c25a015dd4cc4becfdaae141aed66359
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
Change default container tag to be:
1. Functest ovp.1.0.0
2. testapi ovp.1.0.0
Add test suite ovp.1.0.0 and set it to be the default test suite.
Comment out bottlenecks CLI for initial release.
JIRA: DOVETAIL-580
Change-Id: I956e49e43fc7ba0176c14079b7228089db5dc108
Signed-off-by: xudan <xudan16@huawei.com>
|
|
- Fix test case listing and another bullet
Change-Id: I46b0cff945176f464322f06737dc5c10f977598e
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
|
|
- Second pass
- Incorporated most comments
- Added missing links for participation form, governance guidelines,
terms & conditions
- Added links for test collection framework and testscope
- Initial draft of release notes
Change-Id: Ib805513011d917e4ecd0096e51f2d63c898d7142
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
|
|
- Add 2018.01 to title to match reviewer guide, release notes, etc. as
this document may change per release
- Modify link for Governance Guidelines to reflect ratified doc on
opnfv.org domain
Change-Id: Ifdaa287b2919c44d408aa0ddd0b164b8ca4f406d
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
|
|
- Fixed vping, ha, sdnvpn test case names to match test specs
- Removed aliases from VIM operations and matched test case
name changes in patch https://gerrit.opnfv.org/gerrit/#/c/47691/4
- CVP to OVP name change
- A test case was removed from tempest tc004 because of the Tempest
version from 15 to 14. A reference to it was still in this doc.
Change-Id: If2e07e15b48ae6bca5ac0e262aead784cd67cd0f
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
|
|
JIRA: DOVETAIL-579
Under the 'Governance & Workflow' left-side navigation category:
Modify 'Overview' link href from:
https://www.opnfv.org/compliance-verification-program-cvp
to:
https://www.opnfv.org/verified
Modify link text for 'Application Form' to 'Participation Form'
Modify link href for above from:
http://docs.opnfv.org/en/latest/submodules/dovetail/docs/testing/user/certificationworkflow/ApplicationForm.html
to:
https://na3.docusign.net/Member/PowerFormSigning.aspx?PowerFormId=e03e78d7-c32e-47d3-8292-350b747a0105
Add another link right below the 'Governance' link named 'Terms & Conditions'. The href for this links is:
https://www.opnfv.org/wp-content/uploads/sites/12/2018/01/OVP-Terms-and-Conditions-011918.pdf
Under the 'Training Resources' left-side navigation category:
Modify link text for 'Dovetail Test Tool User Guide' to 'OVP / Dovetail User Guide'
Modify link href for above from:
http://docs.opnfv.org/en/latest/submodules/dovetail/docs/testing/user/userguide/testing_guide.html
to:
http://docs.opnfv.org/en/stable-danube/submodules/dovetail/docs/testing/user/userguide/testing_guide.html
Modify 'Test Specifications' link href from:
http://docs.opnfv.org/en/latest/submodules/dovetail/docs/testing/user/testspecification/
to:
http://docs.opnfv.org/en/stable-danube/submodules/dovetail/docs/testing/user/testspecification/
Modify 'Dovetail CLI' link href from:
http://docs.opnfv.org/en/latest/submodules/dovetail/docs/testing/user/userguide/cli_reference.html
to:
http://docs.opnfv.org/en/stable-danube/submodules/dovetail/docs/testing/user/userguide/cli_reference.html
Modify 'OVP Guidelines Addendum' link href from:
http://docs.opnfv.org/en/latest/submodules/dovetail/docs/testing/user/cvpaddendum/index.html
to:
http://docs.opnfv.org/en/stable-danube/submodules/dovetail/docs/testing/user/ovpaddendum/index.html
Under the 'Release Information' left-side navigation category:
For the '2018.01 Release Notes' link, specify the href as
http://docs.opnfv.org/en/stable-danube/submodules/dovetail/docs/release/release-notes/index.html
Change-Id: Ia4d2cc9f4c3c9874b3c328658bb4bce643aa4c6c
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|
|
We don't allow SUT version to be None when submitting test.
So we also not allow user to change it to Nonw after submitting.
In this patch, I also make the result table self-adaption.
Change-Id: I5c391592dcbdbd9eb8697e44d7bb5dda41431bd6
Signed-off-by: chenjiankun <chenjiankun1@huawei.com>
|