Merge "add document for openwrt demo deployment"

1 files changed, 318 insertions, 0 deletions

diff --git a/docs/arm/container4nfv_openwrt_demo_deployment.rst b/docs/arm/container4nfv_openwrt_demo_deployment.rst
new file mode 100644
index 0000000..3e56a84
--- /dev/null
+++ b/docs/arm/container4nfv_openwrt_demo_deployment.rst
@@ -0,0 +1,318 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Arm Limited.
+
+
+
+===============================================
+Container4NFV Openwrt Demo Deployment on Arm Server
+===============================================
+
+Abstract
+========
+
+This document gives a brief introduction on how to deploy openwrt services with multiple networking interfaces on Arm platform.
+
+Introduction
+============
+.. _sriov_cni: https://github.com/hustcat/sriov-cni
+.. _Flannel: https://github.com/coreos/flannel
+.. _Multus:  https://github.com/Intel-Corp/multus-cni
+.. _cni:     https://github.com/containernetworking/cni
+.. _kubeadm: https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/
+.. _openwrt: https://github.com/openwrt/openwrt
+
+The OpenWrt Project is a Linux operating system targeting embedded devices.
+Also it is a famouse open source router project.
+
+We use it as a demo to show how to deploy an open source vCPE in Kubernetes.
+For Lan port, we configured flannel cni for it. And for Wan port, we configured sriov cni for it.
+
+For demo purpose, I suggest that we use Kubeadm to deploy a Kubernetes cluster firstly.
+
+Cluster
+=======
+
+Cluster Info
+
+In this case, we deploy master and slave as one node.
+Suppose it to be: 192.168.1.2
+
+In 192.168.1.2, 2 NIC as required.
+Suppose it to be: eth0, eth1. eth0 is used to be controle plane, and eth1 is used to be data plane.
+
+Deploy Kubernetes
+-----------------
+Please see link(https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/) as reference.
+
+Creat CRD
+---------
+Please make sure that CRD was added for Kubernetes cluster.
+Here we name it as crdnetwork.yaml:
+
+::
+ apiVersion: apiextensions.k8s.io/v1beta1
+ kind: CustomResourceDefinition
+ metadata:
+   # name must match the spec fields below, and be in the form: <plural>.<group>
+   name: networks.kubernetes.com
+ spec:
+   # group name to use for REST API: /apis/<group>/<version>
+   group: kubernetes.com
+   # version name to use for REST API: /apis/<group>/<version>
+   version: v1
+   # either Namespaced or Cluster
+   scope: Namespaced
+   names:
+     # plural name to be used in the URL: /apis/<group>/<version>/<plural>
+     plural: networks
+     # singular name to be used as an alias on the CLI and for display
+     singular: network
+     # kind is normally the CamelCased singular type. Your resource manifests use this.
+     kind: Network
+     # shortNames allow shorter string to match your resource on the CLI
+     shortNames:
+     - net
+
+command:
+
+::
+ kubectl create -f crdnetwork.yaml
+
+Create Flannel-network for Control Plane
+----------------------------------------
+Create flannel network as control plane.
+Here we name it as flannel-network.yaml:
+
+::
+ apiVersion: "kubernetes.com/v1"
+ kind: Network
+ metadata:
+   name: flannel-conf
+ plugin: flannel
+ args: '[
+         {
+                 "masterplugin": true,
+                 "delegate": {
+                         "isDefaultGateway": true
+                 }
+         }
+ ]'
+
+command:
+
+::
+ kubectl create -f flannel-network.yaml
+
+Create Sriov-network for Data Plane
+-----------------------------------
+Create sriov network with PF mode as data plane.
+Here we name it as sriov-network.yaml:
+
+::
+ apiVersion: "kubernetes.com/v1"
+ kind: Network
+ metadata:
+   name: sriov-conf
+ plugin: sriov
+ args: '[
+        {
+                 "master": "eth1",
+                 "pfOnly": true,
+                 "ipam": {
+                         "type": "dhcp",
+                 }
+         }
+ ]'
+
+command:
+
+::
+ kubectl create -f sriov-network.yaml
+
+CNI Installation
+================
+.. _CNI: https://github.com/containernetworking/plugins
+Firstly, we should deploy all CNI plugins. The build process is following:
+
+
+::
+   git clone https://github.com/containernetworking/plugins.git
+   cd plugins
+   ./build.sh
+   cp bin/* /opt/cni/bin
+
+.. _Multus: https://github.com/Intel-Corp/multus-cni
+
+To deploy control plane and data plane interfaces, besides the Flannel CNI and SRIOV CNI,
+we need to deploy the Multus_. The build process of it is as:
+
+::
+   git clone https://github.com/Intel-Corp/multus-cni.git
+   cd multus-cni
+   ./build
+   cp bin/multus /opt/cni/bin
+
+To use the Multus_ CNI,
+we should put the Multus CNI binary to /opt/cni/bin/ where the Flannel CNI and SRIOV
+CNIs are put.
+
+.. _SRIOV: https://github.com/hustcat/sriov-cni
+The build process of it is as:
+
+::
+  git clone https://github.com/hustcat/sriov-cni.git
+  cd sriov-cni
+  ./build
+  cp bin/* /opt/cni/bin
+
+We also need to enable DHCP client for Wan port.
+So we should enable dhcp cni for it.
+
+::
+  /opt/cni/bin/dhcp daemon &
+
+CNI Configuration
+=================
+The following multus CNI configuration is located in /etc/cni/net.d/, here we name it
+as multus-cni.conf:
+
+::
+ {
+   "name": "minion-cni-network",
+   "type": "multus",
+   "kubeconfig": "/etc/kubernetes/admin.conf",
+   "delegates": [{
+     "type": "flannel",
+     "masterplugin": true,
+     "delegate": {
+       "isDefaultGateway": true
+     }
+   }]
+ }
+
+command:
+
+::
+  step1, remove all files in /etc/cni/net.d/
+    rm /etc/cni/net.d/* -rf
+
+  step2, copy /etc/kubernetes/admin.conf into each nodes.
+
+  step3, copy multus-cni.conf into /etc/cni/net.d/
+
+  step4, restart kubelet
+    systemctl restart kubelet
+
+
+Configuring Pod with Control Plane and Data Plane
+=================================================
+
+1, Save the below following YAML to openwrt-vpn-multus.yaml.
+In this case flannle-conf network object act as the primary network.
+
+::
+ apiVersion: v1
+ kind: ReplicationController
+ metadata:
+  name: openwrtvpn1
+ spec:
+  replicas: 1
+  template:
+    metadata:
+      name: openwrtvpn1
+      labels:
+        app: openwrtvpn1
+      annotations:
+        networks: '[
+          { "name": "flannel-conf" },
+          { "name": "sriov-conf" }
+        ]'
+    spec:
+      containers:
+      - name: openwrtvpn1
+        image: "younglook/openwrt-demo:arm64"
+        imagePullPolicy: "IfNotPresent"
+        command: ["/sbin/init"]
+        securityContext:
+          capabilities:
+            add:
+              - NET_ADMIN
+        stdin: true
+        tty: true
+        ports:
+        - containerPort: 80
+        - containerPort: 4500
+        - containerPort: 500
+ ---
+ apiVersion: v1
+ kind: Service
+ metadata:
+  name: openwrtvpn1
+ spec:  # specification of the pod's contents
+  type: NodePort
+  selector:
+    app: openwrtvpn1
+  ports: [
+    {
+      "name": "floatingu",
+      "protocol": "UDP",
+      "port": 4500,
+      "targetPort": 4500
+    },
+    {
+      "name": "actualu",
+      "protocol": "UDP",
+      "port": 500,
+      "targetPort": 500
+    },
+    {
+      "name": "web",
+      "protocol": "TCP",
+      "port": 80,
+      "targetPort": 80
+    },
+  ]
+
+2, Create Pod
+
+::
+ command:
+  kubectl create -f openwrt-vpn-multus.yaml
+
+3, Get the details of the running pod from the master
+
+::
+ # kubectl get pods
+ NAME                   READY     STATUS    RESTARTS   AGE
+ openwrtvpn1            1/1       Running   0          30s
+
+Verifying Pod Network
+=====================
+
+::
+ # kubectl exec openwrtvpn1 -- ip a
+ 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
+    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
+    inet 127.0.0.1/8 scope host lo
+       valid_lft forever preferred_lft forever
+    inet6 ::1/128 scope host
+       valid_lft forever preferred_lft forever
+ 3: eth0@if124: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue
+    link/ether 0a:58:0a:e9:40:2a brd ff:ff:ff:ff:ff:ff
+    inet 10.233.64.42/24 scope global eth0
+       valid_lft forever preferred_lft forever
+    inet6 fe80::8e6:32ff:fed3:7645/64 scope link
+       valid_lft forever preferred_lft forever
+ 4: net0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
+    link/ether 52:54:00:d4:d2:e5 brd ff:ff:ff:ff:ff:ff
+    inet 192.168.123.2/24 scope global net0
+       valid_lft forever preferred_lft forever
+    inet6 fe80::5054:ff:fed4:d2e5/64 scope link
+       valid_lft forever preferred_lft forever
+
+Contacts
+========
+
+Bin Lu:      bin.lu@arm.com