path: root/docs/com/pres/euphrates_functest_evolution/euphrates.md

# Functest
### Feedback on Euphrates evolutions

2017/10/09


### Main Framework evolutions
  * Functest Framework refactoring
  * Docker slicing with Alpine
  * Building Functest dockers
  * Requirement management...1st time...in OPNFV
  * Functest API


## Functest Framework refactoring

### The framework
  * Initiated in Danube
  * Finalized in euphrates
  * Goal: ease the integration of feature/vnf tests

### See complete presentation on the framework
http://testresults.opnfv.org/functest/framework/


## Docker slicing


#### Short story of Functest Docker


### Many rivers to cross
| Version     | Comment                                             |
|-------------|-----------------------------------------------------|
| Arno        | No docker, all tests initiated from the jumphost    |
| Brahmaputra | Introduction of Docker                              |
| Colorado    | Growth of Historical Docker                         |
| Danube      | Growth of Historical Docker                         |
| Euphrates   | Introduction of Alpine                              |


### Focus on the Historical Docker
  * Based on ubuntu 14.04

| Version      | Compressed Size |
|--------------|-----------------|
| Brahmaputra  |  354 MB         |
| Colorado.1.0 |  627 MB         |
| Danube.1.0   |  713 MB         |
| cvp.0.2.0    |  710 MB         |

* src: https://hub.docker.com/r/opnfv/functest/tags/


### Introduction to Alpine
* Alpine Linux is a GNU/Linux distribution based on musl and Busybox
* Hardened kernel, compiles all user space binaries as position-independent executables with stack-smashing protection.
* Docker Alpine leverage Alpine Linux: https://docs.docker.com/samples/library/alpine/


### Our goals
* Lighten docker / save bandwidth / save CI resources
* Slice testing
* Simplify Requirement management
* Isolate "exotic" test cases


### From 1 to many dockers
| Docker      | Size  |  Role                                           |
|-------------|-------|-------------------------------------------------|
| core        | 122MB | baseline (tools, env)                           |
| healthcheck | 122MB | OS connectivity, API, DHCP testing              |
| smoke       | 131MB | vpings, Tempest/Refstack, Rally, odl, Snaps     |
| features    | 214MB | doctor, domino, sdnvpn, sfc, promise, barometer |


### From 1 to many dockers
| Docker      | Size  |  Role                                           |
|-------------|-------|-------------------------------------------------|
| components  | 122MB | Full Tempest, Full Rally                        |
| vnf         | 155MB | vIMS, vRouter                                   |
| parser      | 127MB | parser (feature needs pike clients)             |


### Easy way to run
src: https://wiki.opnfv.org/display/functest/Run+Alpine+Functest+containers
  * env: OPNFV env variables
  * openstack.creds: OpenStack rc file

```
sudo docker run --env-file env \
    -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds  \
    -v $(pwd)/images:/home/opnfv/functest/images  \
    opnfv/functest-smoke:euphrates
```


#### Env
```
cat env
INSTALLER_TYPE=Compass
INSTALLER_IP=XXX
EXTERNAL_NETWORK=ext-net
DEPLOY_SCENARIO=os-nosdn-nofeature-ha
```


#### openstack.creds
```
cat openstack.creds
export OS_AUTH_URL=XXX
export OS_USER_DOMAIN_NAME=XXX
export OS_PROJECT_DOMAIN_NAME=XXX
export OS_USERNAME=XXX
export OS_PROJECT_NAME=XXX
export OS_PASSWORD=XXX
export OS_IDENTITY_API_VERSION=3
```


#### Retrieve all the images
```
mkdir -p images && wget -q -O- https://git.opnfv.org/functest/plain/functest/ci/download_images.sh | bash -s -- images && ls -1 images/*
images/CentOS-7-aarch64-GenericCloud.qcow2
images/CentOS-7-aarch64-GenericCloud.qcow2.xz
images/CentOS-7-x86_64-GenericCloud.qcow2
images/cirros-0.3.5-x86_64-disk.img
images/cirros-0.3.5-x86_64-lxc.tar.gz
images/cirros-d161201-aarch64-disk.img
images/cirros-d161201-aarch64-initramfs
images/cirros-d161201-aarch64-kernel
images/cloudify-manager-premium-4.0.1.qcow2
images/img
images/trusty-server-cloudimg-amd64-disk1.img
images/ubuntu-14.04-server-cloudimg-amd64-disk1.img
images/ubuntu-14.04-server-cloudimg-arm64-uefi1.img
images/ubuntu-16.04-server-cloudimg-amd64-disk1.img
images/vyos-1.1.7.img
```
* could be improved (retrieve only needed images Tier/config)


#### Results
```
+----------------------+--------- + ------+----------+--------+
|      TEST CASE       | PROJECT  |  TIER | DURATION | RESULT |
+----------------------+--------- + ------+----------+--------+
|      vping_ssh       | functest | smoke |  01:19   |  PASS  |
|    vping_userdata    | functest | smoke |  01:56   |  PASS  |
| tempest_smoke_serial | functest | smoke |  26:30   |  PASS  |
|     rally_sanity     | functest | smoke |  19:42   |  PASS  |
|   refstack_defcore   | functest | smoke |  22:00   |  PASS  |
|     snaps_smoke      | functest | smoke |  41:14   |  PASS  |
|         odl          | functest | smoke |  00:16   |  PASS  |
|     odl_netvirt      | functest | smoke |  00:00   |  SKIP  |
|         fds          | functest | smoke |  00:00   |  SKIP  |
+----------------------+--------- + ------+----------+--------+

```


#### Easy way to customize
  * Change list of testcase -v your_config.yaml:/usr/lib/python2.7/site-packages/functest/ci/testcases.yaml
  * Change logger param -v your_logger.ini:/usr/lib/python2.7/site-packages/functest/ci/logging.ini


#### Docker slicing technical presentation
http://testresults.opnfv.org/functest/dockerslicing/



## Building Functest dockers


#### Until Danube
  * the unique Functest docker was "produced" from Releng


#### For Euphrates, build was done on Docker hub
  * Releng adaptations not ready in time
  * more capabilities using Docker Hub
  * Use of private ollivier then official opnfv Docker Hub
  * Euphrates 5.1: come back to releng with at least same level of features than Docker Hub/Travis CI



## Requirement management
Until now, requirements were managed as follow:


# ?


### Requirement management
  * Nothing was done
  * Danube, Colorado, .. dockers may run but no garantee on mid/long term because
    * No control of upstream or internal project dependencies
    * the dependencies of the last feature project overwrite the previous ones


###  Requirement management
  * Nothing done in any OPNFV project
    * No dependency list, no reco (e.g. support Python 3)
    * Only high level wiki declarative intentions for OpenStack (e.g. Euphrates => Ocata)
  * Nothing equivalent to OpenStack https://releases.openstack.org/ocata/


###  Functest evolutions
  * creation of requirements.txt, upper-constraints.text
  https://git.opnfv.org/functest/tree/requirements.txt
  https://git.opnfv.org/functest/tree/upper-constraints.txt
  ```
  pbr>=1.8 # Apache-2.0
  PyYAML>=3.10.0 # MIT
  GitPython>=1.0.1 # BSD License (3 clause)
  keystoneauth1>=2.18.0 # Apache-2.0
  python-cinderclient!=1.7.0,!=1.7.1,>=1.6.0 # Apache-2.0
  python-glanceclient>=2.5.0 # Apache-2.0
  python-heatclient>=1.6.1 # Apache-2.0
  ...
  ```
  * sync with ocata done manually by Cédric


###  Functest evolutions
  * Code of the feature projects under project responsibility
    * code moved to their own repo
    * no more mix between Functest and Feature project code
  * All python OPNFV projects imported properly as python modules
  * all dependencies a priori under control...



## Functest Rest API


### Introduction
* A Rest API has been introduced in Euphrates (Linda)
* Goal: allow third party to invoke Functest resources
  * pseudo micro services approach
  * avoid overlap (e.g. deployement of vIMS from other project to run perfo tests)
src: https://wiki.opnfv.org/display/functest/Functest+REST+API


### Functest rest API
| resource    | Methods  |  Description                                    |
|-------------|----------|-------------------------------------------------|
| environment | GET,POST | show, prepare environment                       |
| openstack   | GET,POST | show, check, clean, update credentials          |
| testcases   | GET,POST | list, show, run                                 |
| tiers       | GET,POST | list, show                                      |
| tasks       | GET      | Get the result of the task id                   |


### Examples
```
curl -X POST --header "Content-Type: application/json" \
  --data '{"action":"run_test_case", "args": {"opts": {}, "testcase": "vping_ssh"}}' \
  http://127.0.0.1:5000/api/v1/functest/testcases/action
{
    "task_id": "1a9f3c5d-ce0b-4354-862e-dd08b26fc484",
    "testcase":"vping_ssh"
}
```



## Conclusions
  * Framework heavily refactored for Euphrates
  * Better code quality and rules (coverage, pylint, pep8, ..)
  * Much more "trustable", light and evolutive


## Next steps
  * Still lots of work for Fraser
    * Adaptation to use it for XCI gating (xTesting)
    * Generic dockerfile Functest customized docker on demand
    * split framework and testcases in order to be able to reuse Functest for k8 or even beyond OPNFV
    * integrate k8 tests
    * better management of the images
    * ...