Merge "auto update of testAPI using ansible-playbook"

9 files changed, 160 insertions, 58 deletions

diff --git a/utils/test/result_collection_api/update/README.md b/utils/test/result_collection_api/update/README.md
index 41b7fffa3..d3aef7efe 100644
--- a/utils/test/result_collection_api/update/README.md
+++ b/utils/test/result_collection_api/update/README.md
@@ -1,59 +1,105 @@
-# opnfv-testapi update
-
-## How to use:
-
-# backup mongodb,
-# arguments:
-# -u/--url: Mongo DB URL, default = mongodb://127.0.0.1:27017/
-# -o/--output_dir: Output directory for the backup, default = ./
-# the backup output will be put under dir/db__XXXX_XX_XX_XXXXXX/db
-# -d/--db: database for the backup, default = test_results_collection
-```
-python backup.py
-```
-
-# restore mongodb
-# arguments:
-# -u/--url: Mongo DB URL, default = mongodb://127.0.0.1:27017/
-# -i/--input_dir: Input directory for the Restore, must be specified
-# the restore input must be specified to dir/db__XXXX_XX_XX_XXXXXX/db
-# -d/--db: database name after the restore, default = basename of input_dir
-```
-python restore.py
-```
-
-# update mongodb
-# arguments:
-# -u/--url: Mongo DB URL, default = mongodb://127.0.0.1:27017/
-# -d/--db: database name to be updated, default = test_results_collection
-# changes need to be done:
-# change collection name, modify changes.collections_old2New
-# collections_old2New = {
-#     'old_collection': 'new_collection',
-# }
-# change field name, modify changes.fields_old2New
-# fields_old2New = {
-#     'collection': [(query, {'old_field': 'new_field'})]
-# }
-# change the doc, modify changes.docs_old2New
-# docs_old2New = {
-#     'test_results': [
-#         ({'field': 'old_value'}, {'field': 'new_value'}),
-#         (query, {'field': 'new_value'}),
-#     ]
-# }
-```
-python update.py
-```
-
-# update opnfv-testapi process
-# this script must be run right in this directory
-# and remember to change ../etc/config.ini before running this script
-# operations includes:
-# kill running test_collection_api & opnfv-testapi
-# install or update dependencies according to ../requirements.txt
-# install opnfv-testapi
-# run opnfv-testapi
+Welcome to TESTAPI Update!
+========================
+
+
+This file is used to describe how testapi update works
+
+----------
+How to use
+---------------
+
+#### <i class="icon-file"></i> backup mongodb
+
+arguments:
+: -u/--url: Mongo DB URL, default = mongodb://127.0.0.1:27017/
+the backup output will be put under dir/db__XXXX_XX_XX_XXXXXX/db
+-d/--db: database for the backup, default = test_results_collection
+
+usage:
+```
+python backup_mongodb.py
+```
+
+#### <i class="icon-file"></i> restore mongodb
+
+arguments:
+: -u/--url: Mongo DB URL, default = mongodb://127.0.0.1:27017/
+  -i/--input_dir: Input directory for the Restore, must be specified,
+  the restore input must be specified to dir/db__XXXX_XX_XX_XXXXXX/db
+  -d/--db: database name after the restore, default = basename of input_dir
+
+usage:
+```
+python restore_mongodb.py
+```
+#### <i class="icon-file"></i> update mongodb
+
+ arguments:
+: -u/--url: Mongo DB URL, default = mongodb://127.0.0.1:27017/
+ -d/--db: database name to be updated, default = test_results_collection
+
+changes need to be done:
+change collection name, modify changes.collections_old2New
+ > collections_old2New = {
+     'old_collection': 'new_collection',
+ }
+
+ change field name, modify changes.fields_old2New
+ > fields_old2New = {
+     'collection': [(query, {'old_field': 'new_field'})]
+ }
+
+ change the doc, modify changes.docs_old2New
+ > docs_old2New = {
+     'test_results': [
+         ({'field': 'old_value'}, {'field': 'new_value'}),
+         (query, {'field': 'new_value'}),
+     ]
+ }
+
+#### <i class="icon-file"></i> update opnfv-testapi process
+This script must be run right in this directory and remember to
+change ../etc/config.ini before running this script.
+
+operations includes:
+: kill running test_collection_api & opnfv-testapi
+install or update dependencies according to ../requirements.txt
+install opnfv-testapi
+run opnfv-testapi
+
+usage:
 ```
 python update_api.py
 ```
+#### <i class="icon-file"></i> update opnfv/testapi container
+Here ansible-playbook is used to implement auto update.
+Please make sure that the remote server is accessible via ssh.
+
+install ansible, please refer:
+```
+http://docs.ansible.com/ansible/intro_installation.html
+```
+run update.yml
+arguments:
+: host: remote server, must provide
+user: user used to access to remote server, default to root
+port: exposed port used to access to testapi, default to 8000
+image: testapi's docker image, default to opnfv/testapi:latest
+update_path: templates directory in remote server, default to /tmp/testapi
+mongodb_url: url of mongodb, default to 172.17.0.1, docker0 ip
+swagger_url: swagger access url, default to http://host:port
+
+usage:
+```
+ansible-playbook update.yml --extra-vars "
+host=10.63.243.17
+user=zte
+port=8000
+image=opnfv/testapi
+update_path=/tmp/testapi
+mongodb_url=mongodb://172.17.0.1:27017
+swagger_url=http://10.63.243.17:8000"```
+> **Note:**
+
+> - If documents need to be changed, please modify file
+templates/changes_in_mongodb.py, and refer section **update mongodb**
diff --git a/utils/test/result_collection_api/update/templates/__init__.py b/utils/test/result_collection_api/update/templates/__init__.py
new file mode 100644
index 000000000..e69de29bb
--- /dev/null
+++ b/utils/test/result_collection_api/update/templates/__init__.py
diff --git a/utils/test/result_collection_api/update/backup.py b/utils/test/result_collection_api/update/templates/backup_mongodb.py
index faa4fd410..7e0dd5545 100644
--- a/utils/test/result_collection_api/update/backup.py
+++ b/utils/test/result_collection_api/update/templates/backup_mongodb.py
@@ -6,9 +6,9 @@
 # which accompanies this distribution, and is available at
 # http://www.apache.org/licenses/LICENSE-2.0
 ##############################################################################
-import os
 import argparse
 import datetime
+import os
 
 from utils import execute, main, get_abspath
 
diff --git a/utils/test/result_collection_api/update/changes.py b/utils/test/result_collection_api/update/templates/changes_in_mongodb.py
index b3e23803d..b3e23803d 100644
--- a/utils/test/result_collection_api/update/changes.py
+++ b/utils/test/result_collection_api/update/templates/changes_in_mongodb.py
diff --git a/utils/test/result_collection_api/update/restore.py b/utils/test/result_collection_api/update/templates/restore_mongodb.py
index c45a0e621..c45a0e621 100644
--- a/utils/test/result_collection_api/update/restore.py
+++ b/utils/test/result_collection_api/update/templates/restore_mongodb.py
diff --git a/utils/test/result_collection_api/update/templates/rm_olds.sh b/utils/test/result_collection_api/update/templates/rm_olds.sh
new file mode 100644
index 000000000..c6bca1867
--- /dev/null
+++ b/utils/test/result_collection_api/update/templates/rm_olds.sh
@@ -0,0 +1,15 @@
+#!/bin/bash
+
+proc_number=`ps -ef | grep opnfv-testapi | grep -v grep | wc -l`
+if [ $proc_number -gt 0 ]; then
+    procs=`ps -ef | grep opnfv-testapi | grep -v grep`
+    echo "begin to kill opnfv-testapi $procs"
+    ps -ef | grep opnfv-testapi | grep -v grep | awk '{print $2}' | xargs kill -kill &>/dev/null
+fi
+
+number=`docker ps -a | awk 'NR != 1' | grep testapi | wc -l`
+if [ $number -gt 0 ]; then
+    containers=number=`docker ps -a | awk 'NR != 1' | grep testapi`
+    echo "begin to rm containers $containers"
+    docker ps -a | awk 'NR != 1' | grep testapi | awk '{print $1}' | xargs docker rm -f &>/dev/null
+fi
diff --git a/utils/test/result_collection_api/update/update.py b/utils/test/result_collection_api/update/templates/update_mongodb.py
index 8b385a0b3..b1e378dd7 100644
--- a/utils/test/result_collection_api/update/update.py
+++ b/utils/test/result_collection_api/update/templates/update_mongodb.py
@@ -10,8 +10,8 @@ import argparse
 
 from pymongo import MongoClient
 
+from changes_in_mongodb import collections_old2New, fields_old2New, docs_old2New
 from utils import main, parse_mongodb_url
-from changes import collections_old2New, fields_old2New, docs_old2New
 
 parser = argparse.ArgumentParser(description='Update MongoDBs')
 
diff --git a/utils/test/result_collection_api/update/utils.py b/utils/test/result_collection_api/update/templates/utils.py
index a18ff0389..a18ff0389 100644
--- a/utils/test/result_collection_api/update/utils.py
+++ b/utils/test/result_collection_api/update/templates/utils.py
diff --git a/utils/test/result_collection_api/update/update.yml b/utils/test/result_collection_api/update/update.yml
new file mode 100644
index 000000000..08839564a
--- /dev/null
+++ b/utils/test/result_collection_api/update/update.yml
@@ -0,0 +1,41 @@
+---
+- hosts: "{{ host }}"
+  remote_user: "{{ user }}"
+  become: yes
+  become_method: sudo
+  vars:
+    user: "root"
+    port: "8000"
+    update_path: "/tmp/testapi"
+    image: "opnfv/testapi"
+    mongodb_url: "mongodb://172.17.0.1:27017"
+    swagger_url: "http://{{ host }}:{{ port }}"
+  tasks:
+    - name: create temporary update directory
+      file:
+        path: "{{ update_path }}"
+        state: directory
+    - name: transfer files in templates
+      copy:
+        src: templates/
+        dest: "{{ update_path }}"
+    - name: backup mongodb database
+      command: "python {{ update_path }}/backup_mongodb.py -u {{ mongodb_url }} -o {{ update_path }}"
+    - name: stop and remove old versions
+      command: bash "{{ update_path }}/rm_olds.sh"
+      register: rm_result
+    - debug: msg="{{ rm_result.stderr }}"
+    - name: delete old docker images
+      command: docker rmi "{{ image }}"
+      ignore_errors: true
+    - name: update mongodb
+      command: "python {{ update_path }}/update_mongodb.py -u {{ mongodb_url }}"
+    - name: docker start testapi server
+      command: docker run -dti -p "{{ port }}:8000"
+               -e "mongodb_url={{ mongodb_url }}"
+               -e "swagger_url={{ swagger_url }}"
+               "{{ image }}"
+    - name: remove temporary update directory
+      file:
+        path: "{{ update_path }}"
+        state: absent
\ No newline at end of file