Age | Commit message (Collapse) | Author | Files | Lines |
|
We want to generate pod.yaml from Heat contexts so we can
re-use the context without destroying it.
But we don't have node role information and it doesn't
make sense in this case, so make the role optional.
Since we changed Heat to use pkey instead of key_filename,
we can embed the pkey into the pod.yaml, but we have
to make sure to convert the pkey to string, in case
it is a RSAKey object
Change-Id: Ibefcfbd8236e68013a704c39964cb870da825da8
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
Change-Id: I1c529eeb0ef47752ed15e3e7941f57f7793ebfd4
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
We can either cat remote files, or we can just sftp get
them.
use sftp get for /proc/cpuinfo since it can be so very large
on systems with 88+ cores.
Change-Id: I420b8c5eefdce8bb3e3b13dcc8257583dee537c1
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
Refactored main NSB VNF classes accroding to class diagram
https://wiki.opnfv.org/display/yardstick/NSB+class+diagram
All the SampleVNFs have been separated and placed under
the SampleVNF class.
Added AutoConnectSSH to automatically create SSH conneciton on demand.
Added VnfdHelper class to wrap the VNFD dictionary in prepartion for
class-based modeling.
Extracted DpdkVnfSetupEnvHelper for DPDK based VNF setup.
Extracted Stats and other client config to ResourceHelper
Had to replace dict_key_flatten with deepgetitem due to Python 2.7
Jinja2 infinite recursion.
Change-Id: Ia8840e9c44cdbdf39aab6b02e6d2176b31937dc9
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
Signed-off-by: Edward MacGillivray <edward.s.macgillivray@intel.com>
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
we need to be following defautl paramiko rules,
first use pkey, then key_filenames (autodetecting ~/.ssh/ keys),
then password
We have too much boilerplate redudant code everywhere, we need
to standardize on a factory function that takes a node dict.
Using Python3 ChainMap we can layer overrides and defaults.
VNF descriptors have to default key_filename, password to Python None.
The only way to do this is to omit key values if the variable is not
defined, this way the dict will not have the value and it will
default to Python None
Add python2 chainmap backport
Updated unittest mocking to use ssh.SSH.from_node
Change-Id: I80b0cb606e593b33e317c9e5e8ed0b74da591514
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
Process running via ssh can return "None" or emtpy data from the
application. To avoid encodutils raise NoneType issue. Check the data before
encode.
JIRA: YARDSTICK-539
Change-Id: I7e86e6a17c0adc95d41714f6fec463dfadc2b81b
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
|
|
Porting to Python3 using Openstack guidelines:
https://wiki.openstack.org/wiki/Python3
This passes unittests on Python 3.5 and passes opnfv_smoke suite
Updates:
use six for urlparse and urlopen
fix exception.message attribute removal
run unittests on python3
use unitest.mock on python 3
fix open mock for vsperf
fix float division by using delta/eplison comparison
use unicode in StringIO
use plugin/sample_config.yaml relative path from test case
fixed apexlake unittests
upgraded to mock 2.0.0 to match python3 unittest.mock features
fixed flake8 issues
implement safe JSON decode with oslo_serialization.jsonutils.dump_as_bytes()
implement safe unicode encode/decode with oslo_utils.encodeutils
heat: convert pub key file from bytes to unicode
pkg_resources returns raw bytes, in python3
we have to decode this to utf-8 unicode
so JSON can encode it for heat template
JIRA: YARDSTICK-452
Change-Id: Ib80dd1d0c0eb0592acd832b82f6a7f8f7c20bfda
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
~ is not expanded in double quotes, so we have a dilemma.
We need to quote in order to preserve filenames with spaces,
but we have to make sure we don't quote the ~ so it can be expanded.
To resolve this we use a regex to search for tidle-prefixes
and excluded them from quotes.
Added unittests for the cases:
path with tilde
path with space
path with tilde and space
see bash man page for details of tidle expansion
Tilde Expansion
If a word begins with an unquoted tilde character (`~'), all of the
characters preceding the first unquoted slash (or all characters, if there is
no unquoted slash) are considered a tilde-prefix. If none of the characters in
the tilde-prefix are quoted, the characters in the tilde-prefix following the
tilde are treated as a possible login name. If this login name is the null
string, the tilde is replaced with the value of the shell parameter HOME. If
HOME is unset, the home directory of the user executing the shell is
substituted instead. Otherwise, the tilde-prefix is replaced with the
home directory associated with the specified login name.
JIRA: YARDSTICK-501
Change-Id: I324be20aba0dbd50434fbd8081685c598ebd8a84
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-501
An redundant pair of quotation in ssh.py causes remotepath cannot be found.
Change-Id: I2df8ab59830fd28d8ad8882a93a8efbd4d1f7cb7
Signed-off-by: JingLu5 <lvjing5@huawei.com>
|
|
For some VNFs we may want to send periodic commands, for example to
print statistics etc.
When you open a SSH connection, request a pseudo terminal (pty) which
allows passing of control characters to the connection.
JIRA: YARDSTICK-453
Change-Id: Ibfd4164e745f005d0e29f6efdc63076e1e220b60
Signed-off-by: Deepak S <deepak.s@linux.intel.com>
|
|
|
|
|
|
|
|
The logging methods do string interpolation themselves
From the reference:
https://docs.python.org/2/library/logging.html#logging.Logger.debug
Logger.debug(msg, *args, **kwargs)
Logs a message with level DEBUG on this logger. The msg is the message format string, and the args are the arguments which are merged into msg using the string formatting operator. (Note that this means that you can use keywords in the format string, together with a single dictionary argument.)
There are two keyword arguments in kwargs which are inspected: exc_info which, if it does not evaluate as false, causes exception information to be added to the logging message. If an exception tuple (in the format returned by sys.exc_info()) is provided, it is used; otherwise, sys.exc_info() is called to get the exception informatio
The reason logging does string interpolation itselfs is to implement deferred interpolation.
String interpolation involves evaluating arguments, so it can introduce significant computation. The logging module tries to be smart about deferring interpolation until the last possible moment.
The logging methods check isEnabledFor for the log level and won't interpolate if the level is not enabled.
https://github.com/python/cpython/blob/2.7/Lib/logging/__init__.py#L1178
def warning(self, msg, *args, **kwargs):
if self.isEnabledFor(WARNING):
self._log(WARNING, msg, args, **kwargs)
logging actually waits to interpolate the string in LogRecord.getMessage()
https://github.com/python/cpython/blob/2.7/Lib/logging/__init__.py#L328
if self.args:
msg = msg % self.args
Change-Id: Ie09efe0a66881e19bd8119caa376075e605627a2
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
upstream openstack rally added new _put_file_* methods
we should use these
https://github.com/openstack/rally/blob/0.7.0/rally/common/sshutils.py#L270
Updates:
imported rally test__put_file_shell unittests
quote to prevent word split
use -- guard
only chmod on cat success
Change-Id: I357d1a66b5beddad8042958f4e55d67fc68929f6
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
If we setup root logger correctly and have each
module logger propogate we shouldn't need individual
logger configuration
updates:
lower paramiko to WARN level
dispatcher/file.py was missing logging.handlers import
purge all existing handlers and add our own handlers
move everything back into yardstick/__init__.py so API can use it
make _LOG_STREAM_HDLR global, so we can set loglevel on it whenever
added api/server.py call to _init_logging
removed old LOG_FORMATTER from cli.py
only setLevel on yardstick logger
Change-Id: If000799590379d3407655a7d54378481a96ea3d4
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
change PseudoFile example to use io.RawIOBase baseclass
Change-Id: Ib5e3c844a0514274e5098061beb0ee6f8af97977
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
For some VNFs we may want to send periodic commands, for example to print
statistics, but otherwise not write anything for long periods of time.
Currently when we can no longer read from stdin we close it.
A workaround is to constantly spam stdin with newlines to keep forcing
stdin open. We don't want to have to do this, so add an enable flag to
keep stdin open. If the caller wants to close stdin at some point it
can.
Change-Id: I9496022295dfd19804572e484fe4f170ca7d4ac3
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
not all enviroments have port 22 enabled for SSH.
In particular for network isolation NAT and port forwarding may be used.
example pod.yaml:
nodes:
- ip: 10.2.45.145
name: node1
password: ''
role: Controller
ssh_port: 5000
user: root
- ip: 10.2.45.145
name: node2
password: ''
role: Controller
ssh_port: 5001
user: root
- ip: 10.2.45.145
name: node3
password: ''
role: Controller
ssh_port: 5002
user: root
JIRA: YARDSTICK-407
Change-Id: I8f9d6e388f31d291dd15cb900d7f71f347e41ef6
Signed-off-by: Ross Brattain <ross.b.brattain@intel.com>
|
|
JIRA: YARDSTICK-319
Change-Id: I25ee9a2582c751024b30e91eabf77aebb14a02f7
Signed-off-by: rexlee8776 <limingjiang@huawei.com>
|
|
VM A runs pktgen-dpdk as a traffic generator and is connected
back-to-back with VM B running testpmd for forwarding packets.
1. use linux expect to fetch the latency statics data.
2. fetch 10 samples of latency and calculate avg_latency.
3. use screen to keep test scripts running in the background.
4. add a function send_command() for screen in ssh.py
JIRA:YARDSTICK-233
Change-Id: I90ae64b3d198c79f159275ae35715eb284f05080
Signed-off-by: wu.zhihui <wu.zhihui1@zte.com.cn>
|
|
JIRA: YARDSTICK-122
Change-Id: I8144215059a9abea08314a4c1e6a733dcdf0df53
Signed-off-by: QiLiang <liangqi1@huawei.com>
|
|
The ssh module is used when executing benchmark scenarios using
remote login to a VM.
ssh.py originates from rally but is stripped down and slightly
modified (some dependencies removed).
Change-Id: I0dc479fa1cb06e7469c20e24d3124f020b97a73c
JIRA: -
Signed-off-by: Hans Feldt <hans.feldt@ericsson.com>
|