From c22fed53362c7ee31cd9396a8e57cae0c2cbc5a4 Mon Sep 17 00:00:00 2001 From: "Mytnyk, Volodymyr" Date: Tue, 21 Nov 2017 14:30:17 +0100 Subject: VES: Updated Ubuntu/CentOS install steps - Updated Kafka URL Change-Id: I3ffe96d5cf4a8c2f4e5c14d81c0ee52a25103caf Signed-off-by: Mytnyk, Volodymyr --- docs/release/userguide/collectd.ves.userguide.rst | 109 +++++++++++----------- docs/release/userguide/ves-app-guest-mode.png | Bin 0 -> 6066 bytes docs/release/userguide/ves-app-host-mode.png | Bin 0 -> 8057 bytes 3 files changed, 52 insertions(+), 57 deletions(-) create mode 100644 docs/release/userguide/ves-app-guest-mode.png create mode 100644 docs/release/userguide/ves-app-host-mode.png (limited to 'docs') diff --git a/docs/release/userguide/collectd.ves.userguide.rst b/docs/release/userguide/collectd.ves.userguide.rst index 1f69b6ad..9cb2db8f 100644 --- a/docs/release/userguide/collectd.ves.userguide.rst +++ b/docs/release/userguide/collectd.ves.userguide.rst @@ -24,28 +24,20 @@ Install Kafka Broker 1. Dependencies: install JAVA & Zookeeper. - JAVA: + Ubuntu 16.04: .. code:: bash - $ sudo apt install default-jre + $ sudo apt-get install default-jre + $ sudo apt-get install zookeeperd + $ sudo apt-get install python-pip - CentOS 7.x use: + CentOS: .. code:: bash $ sudo yum install java-1.6.0-openjdk - - Zookeeper: - - .. code:: bash - - $ sudo apt install zookeeperd - - CentOS 7.x use: - - .. code:: bash - + $ sudo yum install python-pip $ sudo yum install zookeeper .. note:: You may need to add the repository that contains zookeeper. @@ -57,7 +49,7 @@ Install Kafka Broker $ sudo yum install https://archive.cloudera.com/cdh5/one-click-install/redhat/7/x86_64/cloudera-cdh-5-0.x86_64.rpm - CentOS 7.x start zookeeper: + Start zookeeper: .. code:: bash @@ -69,62 +61,41 @@ Install Kafka Broker .. code:: bash - $ sudo /usr/lib/zookeeper/bin/zkServer-initialize.sh + $ sudo /usr/lib/zookeeper/bin/zkServer-initialize.sh No myid provided, be sure to specify it in /var/lib/zookeeper/myid if using non-standalone - To test if Zookeeper is running as a daemon. +2. Test if Zookeeper is running as a daemon. .. code:: bash $ telnet localhost 2181 Type 'ruok' & hit enter. - Expected response is 'imok'. Zookeeper is running fine. + Expected response is 'imok' which means that Zookeeper is up running. - .. note:: +3. Install Kafka - VES doesn't work with version 0.9.4 of kafka-python. - The recommended/tested version is 1.3.3. + .. note:: VES doesn't work with version 0.9.4 of kafka-python. The + recommended/tested version is 1.3.3. .. code:: bash - $ sudo yum install python-pip $ sudo pip install kafka-python - -2. Download Kafka: - - .. code:: bash - - $ wget "http://www-eu.apache.org/dist/kafka/0.11.0.0/kafka_2.11-0.11.0.0.tgz" - -3. Extract the archive: - - .. code:: bash - + $ wget "https://archive.apache.org/dist/kafka/0.11.0.0/kafka_2.11-0.11.0.0.tgz" $ tar -xvzf kafka_2.11-0.11.0.0.tgz + $ sed -i -- 's/#delete.topic.enable=true/delete.topic.enable=true/' kafka_2.11-0.11.0.0/config/server.properties + $ sudo nohup kafka_2.11-0.11.0.0/bin/kafka-server-start.sh \ + kafka_2.11-0.11.0.0/config/server.properties > kafka_2.11-0.11.0.0/kafka.log 2>&1 & -4. Configure Kafka Server: - - .. code:: bash - - $ vi kafka_2.11-0.11.0.0/config/server.properties - - By default Kafka does not allow you to delete topics. Please uncomment: - - .. code:: bash - - delete.topic.enable=true - -5. Start the Kafka Server. - - Run 'kafka-server-start.sh' with nohup as a background process: + .. note:: If Kafka server fails to start, please check if the system IP + address is associated with the hostname in the static host lookup + table. If it doesn't exist, use the following command to add it. .. code:: bash - $ sudo nohup kafka_2.11-0.11.0.0/bin/kafka-server-start.sh \ - kafka_2.11-0.11.0.0/config/server.properties > kafka_2.11-0.11.0.0/kafka.log 2>&1 & + $ echo "$(ip route get 8.8.8.8 | awk '{print $NF; exit}') $HOSTNAME" | sudo tee -a /etc/hosts -6. Test Kafka Broker Installation +4. Test the Kafka Installation To test if the installation worked correctly there are two scripts, producer and consumer scripts. These will allow you to see messages pushed to broker and receive from broker. @@ -141,7 +112,7 @@ Install Kafka Broker .. code:: bash $ kafka_2.11-0.11.0.0/bin/kafka-console-consumer.sh --zookeeper \ - localhost:2181 --topic TopicTest --from-beginning + localhost:2181 --topic TopicTest --from-beginning --max-messages 1 --timeout-ms 3000 Install collectd @@ -149,13 +120,22 @@ Install collectd Install development tools: -.. code:: bash + Ubuntu 16.04: + + .. code:: bash + + $ sudo apt-get install build-essential bison autotools-dev autoconf + $ sudo apt-get install pkg-config flex libtool + + CentOS: - $ sudo yum group install 'Development Tools' + .. code:: bash + + $ sudo yum group install 'Development Tools' -.. The libkafka installed via yum pkg manager is 0.11.0 which doesn't work with - collectd (compilation issue). Thus, we have to use the library installed - from sources using latest stable version which works with collectd. +.. The libkafka installed via the package manager may not work with collectd + (due to compilation issue). Thus, it's recommented to use the library installed + from sources using latest stable version of libkafka. Install Apache Kafka C/C++ client library: @@ -228,6 +208,13 @@ Start VES Test Collector: Setup VES application (guest mode) ---------------------------------- +This mode is used to collect guest VM statistics provided by collectd +and send those metrics into the VES collector. + +.. figure:: ves-app-guest-mode.png + + VES guest mode setup + Install dependencies: .. code:: bash @@ -253,6 +240,14 @@ Clone Barometer repo and start the VES application: Configure VES in host mode -------------------------- +This mode is used to collect hypervisor statistics about guest VMs and to send +those metrics into the VES collector. Also, this mode collects host statistics +and send them as part of the guest VES message. + +.. figure:: ves-app-host-mode.png + + VES host mode setup + Running the VES in host mode looks like steps described in `Setup VES application (guest mode)`_ but with the following exceptions: diff --git a/docs/release/userguide/ves-app-guest-mode.png b/docs/release/userguide/ves-app-guest-mode.png new file mode 100644 index 00000000..4d05dae2 Binary files /dev/null and b/docs/release/userguide/ves-app-guest-mode.png differ diff --git a/docs/release/userguide/ves-app-host-mode.png b/docs/release/userguide/ves-app-host-mode.png new file mode 100644 index 00000000..5a58787c Binary files /dev/null and b/docs/release/userguide/ves-app-host-mode.png differ -- cgit 1.2.3-korg