diff options
Diffstat (limited to 'docs/com/pres/euphrates_functest_evolution/euphrates.md')
| -rw-r--r-- | docs/com/pres/euphrates_functest_evolution/euphrates.md | 281 |
1 files changed, 281 insertions, 0 deletions
diff --git a/docs/com/pres/euphrates_functest_evolution/euphrates.md b/docs/com/pres/euphrates_functest_evolution/euphrates.md new file mode 100644 index 000000000..49aab16cd --- /dev/null +++ b/docs/com/pres/euphrates_functest_evolution/euphrates.md @@ -0,0 +1,281 @@ +# Functest +### Feedback on Euphrates evolutions + +2017/10/09 + + +### Main Framework evolutions + * Functest Framework refactoring + * Docker slicing with Alpine + * Building Functest dockers + * Requirement management...1st time...in OPNFV + * Functest API + + +## Functest Framework refactoring + +### The framework + * Initiated in Danube + * Finalized in euphrates + * Goal: ease the integration of feature/vnf tests + +### See complete presentation on the framework +http://testresults.opnfv.org/functest/framework/ + + +## Docker slicing + + +#### Short story of Functest Docker + + +### Many rivers to cross +| Version | Comment | +|-------------|-----------------------------------------------------| +| Arno | No docker, all tests initiated from the jumphost | +| Brahmaputra | Introduction of Docker | +| Colorado | Growth of Historical Docker | +| Danube | Growth of Historical Docker | +| Euphrates | Introduction of Alpine | + + +### Focus on the Historical Docker + * Based on ubuntu 14.04 + +| Version | Compressed Size | +|--------------|-----------------| +| Brahmaputra | 354 MB | +| Colorado.1.0 | 627 MB | +| Danube.1.0 | 713 MB | +| cvp.0.2.0 | 710 MB | + +* src: https://hub.docker.com/r/opnfv/functest/tags/ + + +### Introduction to Alpine +* Alpine Linux is a GNU/Linux distribution based on musl and Busybox +* Hardened kernel, compiles all user space binaries as position-independent executables with stack-smashing protection. +* Docker Alpine leverage Alpine Linux: https://docs.docker.com/samples/library/alpine/ + + +### Our goals +* Lighten docker / save bandwidth / save CI resources +* Slice testing +* Simplify Requirement management +* Isolate "exotic" test cases + + +### From 1 to many dockers +| Docker | Size | Role | +|-------------|-------|-------------------------------------------------| +| core | 122MB | baseline (tools, env) | +| healthcheck | 122MB | OS connectivity, API, DHCP testing | +| smoke | 131MB | vpings, Tempest/Refstack, Rally, odl, Snaps | +| features | 214MB | doctor, domino, sdnvpn, sfc, promise, barometer | + + +### From 1 to many dockers +| Docker | Size | Role | +|-------------|-------|-------------------------------------------------| +| components | 122MB | Full Tempest, Full Rally | +| vnf | 155MB | vIMS, vRouter | +| parser | 127MB | parser (feature needs pike clients) | + + +### Easy way to run +src: https://wiki.opnfv.org/display/functest/Run+Alpine+Functest+containers + * env: OPNFV env variables + * openstack.creds: OpenStack rc file + +``` +sudo docker run --env-file env \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \ + -v $(pwd)/images:/home/opnfv/functest/images \ + opnfv/functest-smoke:euphrates +``` + + +#### Env +``` +cat env +INSTALLER_TYPE=Compass +INSTALLER_IP=XXX +EXTERNAL_NETWORK=ext-net +DEPLOY_SCENARIO=os-nosdn-nofeature-ha +``` + + +#### openstack.creds +``` +cat openstack.creds +export OS_AUTH_URL=XXX +export OS_USER_DOMAIN_NAME=XXX +export OS_PROJECT_DOMAIN_NAME=XXX +export OS_USERNAME=XXX +export OS_PROJECT_NAME=XXX +export OS_PASSWORD=XXX +export OS_IDENTITY_API_VERSION=3 +``` + + +#### Retrieve all the images +``` +mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | bash -s -- images && ls -1 images/* +images/CentOS-7-aarch64-GenericCloud.qcow2 +images/CentOS-7-aarch64-GenericCloud.qcow2.xz +images/CentOS-7-x86_64-GenericCloud.qcow2 +images/cirros-0.3.5-x86_64-disk.img +images/cirros-0.3.5-x86_64-lxc.tar.gz +images/cirros-d161201-aarch64-disk.img +images/cirros-d161201-aarch64-initramfs +images/cirros-d161201-aarch64-kernel +images/cloudify-manager-premium-4.0.1.qcow2 +images/img +images/trusty-server-cloudimg-amd64-disk1.img +images/ubuntu-14.04-server-cloudimg-amd64-disk1.img +images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img +images/ubuntu-16.04-server-cloudimg-amd64-disk1.img +images/vyos-1.1.7.img +``` +* could be improved (retrieve only needed images Tier/config) + + +#### Results +``` ++----------------------+--------- + ------+----------+--------+ +| TEST CASE | PROJECT | TIER | DURATION | RESULT | ++----------------------+--------- + ------+----------+--------+ +| vping_ssh | functest | smoke | 01:19 | PASS | +| vping_userdata | functest | smoke | 01:56 | PASS | +| tempest_smoke_serial | functest | smoke | 26:30 | PASS | +| rally_sanity | functest | smoke | 19:42 | PASS | +| refstack_defcore | functest | smoke | 22:00 | PASS | +| snaps_smoke | functest | smoke | 41:14 | PASS | +| odl | functest | smoke | 00:16 | PASS | +| odl_netvirt | functest | smoke | 00:00 | SKIP | +| fds | functest | smoke | 00:00 | SKIP | ++----------------------+--------- + ------+----------+--------+ + +``` + + +#### Easy way to customize + * Change list of testcase -v your_config.yaml:/usr/lib/python2.7/site-packages/functest/ci/testcases.yaml + * Change logger param -v your_logger.ini:/usr/lib/python2.7/site-packages/functest/ci/logging.ini + + +#### Docker slicing technical presentation +http://testresults.opnfv.org/functest/dockerslicing/ + + + +## Building Functest dockers + + +#### Until Danube + * the unique Functest docker was "produced" from Releng + + +#### For Euphrates, build was done on Docker hub + * Releng adaptations not ready in time + * more capabilities using Docker Hub + * Use of private ollivier then official opnfv Docker Hub + * Euphrates 5.1: come back to releng with at least same level of features than Docker Hub/Travis CI + + + +## Requirement management +Until now, requirements were managed as follow: + + +# ? + + +### Requirement management + * Nothing was done + * Danube, Colorado, .. dockers may run but no garantee on mid/long term because + * No control of upstream or internal project dependencies + * the dependencies of the last feature project overwrite the previous ones + + +### Requirement management + * Nothing done in any OPNFV project + * No dependency list, no reco (e.g. support Python 3) + * Only high level wiki declarative intentions for OpenStack (e.g. Euphrates => Ocata) + * Nothing equivalent to OpenStack https://releases.openstack.org/ocata/ + + +### Functest evolutions + * creation of requirements.txt, upper-constraints.text + https://git.opnfv.org/functest/tree/requirements.txt + https://git.opnfv.org/functest/tree/upper-constraints.txt + ``` + pbr>=1.8 # Apache-2.0 + PyYAML>=3.10.0 # MIT + GitPython>=1.0.1 # BSD License (3 clause) + keystoneauth1>=2.18.0 # Apache-2.0 + python-cinderclient!=1.7.0,!=1.7.1,>=1.6.0 # Apache-2.0 + python-glanceclient>=2.5.0 # Apache-2.0 + python-heatclient>=1.6.1 # Apache-2.0 + ... + ``` + * sync with ocata done manually by Cédric + + +### Functest evolutions + * Code of the feature projects under project responsibility + * code moved to their own repo + * no more mix between Functest and Feature project code + * All python OPNFV projects imported properly as python modules + * all dependencies a priori under control... + + + +## Functest Rest API + + +### Introduction +* A Rest API has been introduced in Euphrates (Linda) +* Goal: allow third party to invoke Functest resources + * pseudo micro services approach + * avoid overlap (e.g. deployement of vIMS from other project to run perfo tests) +src: https://wiki.opnfv.org/display/functest/Functest+REST+API + + +### Functest rest API +| resource | Methods | Description | +|-------------|----------|-------------------------------------------------| +| environment | GET,POST | show, prepare environment | +| openstack | GET,POST | show, check, clean, update credentials | +| testcases | GET,POST | list, show, run | +| tiers | GET,POST | list, show | +| tasks | GET | Get the result of the task id | + + +### Examples +``` +curl -X POST --header "Content-Type: application/json" \ + --data '{"action":"run_test_case", "args": {"opts": {}, "testcase": "vping_ssh"}}' \ + http://127.0.0.1:5000/api/v1/functest/testcases/action +{ + "task_id": "1a9f3c5d-ce0b-4354-862e-dd08b26fc484", + "testcase":"vping_ssh" +} +``` + + + +## Conclusions + * Framework heavily refactored for Euphrates + * Better code quality and rules (coverage, pylint, pep8, ..) + * Much more "trustable", light and evolutive + + +## Next steps + * Still lots of work for Fraser + * Adaptation to use it for XCI gating (xTesting) + * Generic dockerfile Functest customized docker on demand + * split framework and testcases in order to be able to reuse Functest for k8 or even beyond OPNFV + * integrate k8 tests + * better management of the images + * ... |