Tempest and SNAPS not pushing results to DB in case of FAIL

this problem was due to the fact that we add exit condition
EX_ERROR when the result was FAIL

EX_ERROR shall be used as exit condition when there is an
execution error, i.e the test case cannot be run

if it can be run, there is no execution error
the result can be PASS or FAIL, but from the CI, execution is OK

JIRA: FUNCTEST-714

Change-Id: Id434bb9aa88b50277c40c717259304f26f7b3966
Signed-off-by: Morgan Richomme <morgan.richomme@orange.com>

1 files changed, 6 insertions, 2 deletions

diff --git a/functest/core/pytest_suite_runner.py b/functest/core/pytest_suite_runner.py
index 1eed92b5..c168d7d9 100644
--- a/functest/core/pytest_suite_runner.py
+++ b/functest/core/pytest_suite_runner.py
@@ -41,14 +41,18 @@ class PyTestSuiteRunner(base.TestcaseBase):
             for test, message in result.failures:
                 self.logger.error(str(test) + " FAILED with " + message)
 
+        # a result can be PASS or FAIL
+        # But in this case it means that the Execution was OK
+        # we shall distinguish Execution Error from FAIL results
+        # TestcaseBase.EX_RUN_ERROR means that the test case was not run
+        # not that it was run but the result was FAIL
+        exit_code = base.TestcaseBase.EX_OK
         if ((result.errors and len(result.errors) > 0)
                 or (result.failures and len(result.failures) > 0)):
             self.logger.info("%s FAILED" % self.case_name)
             self.criteria = 'FAIL'
-            exit_code = base.TestcaseBase.EX_RUN_ERROR
         else:
             self.logger.info("%s OK" % self.case_name)
-            exit_code = base.TestcaseBase.EX_OK
             self.criteria = 'PASS'
 
         self.details = {}