OPENVSWICH - Creating and Submitting Openvswitch Extension To Tinycore Repository

In May 2011, I read a request for installation Openvswitch on Qemu image. I started to play with Openvswitch and finally became fan of this project. I soon realized how powerful can be Openvswitch, offering many features so far available only in hardware switches.

After several days spent with playing with Core Linux and Openvswitch I was able to create Core Qemu image with 8021q support and install Openvswitch on the top of it. The reason why I chose Core Linux is that I am familiar with this minimal Linux distribution and the size of Core Qemu appliance is incredibly small.

The tutorial describes step-by-step installation of Openvswitch on Qemu image. It also explains how to create Openvswitch extension and make it ready for submitting to Tinycore public repository. Installation of Core Linux on Qemu image is not shown but you can find it here. Similarly Qemu compilation and installation on Fedora Linux is explained here.

I also created three labs using Openvswitch. I tested how can Openvswitch works with VLANs, 802.1q trunk ports and if it is capable of creating L3 VLAN interfaces. All labs are available here:

1. Start Qemu Core VM

Assuming your Core Qemu image with console support has been created and supports 8021q VLAN tagging start the image with the commad:

$ qemu /home/brezular/core.img -m 1024 --enable-kvm

2. Download Openvswitch

$ wget http://openvswitch.org/releases/openvswitch-x.y.z.tar.gz
$ tar zxvf ./openvswitch-x.y.x.tar.gz
$ cd ./openvswitch-x.y.z

3. Install Required Dependences for Openvswitch Compilation

$ tce-load -w -i compiletc.tcz python.tcz perl5.tcz openssl-1.0.0-dev.tcz linux-kernel-sources-env.tcz

To compile openvswitch with the ovsdmonitor tool, the machine must also have the following extensions installed. However it is very likely that the extensions are not available in recent tinycore repository.

$ tce-load -w -i python_twisted-2.7.tcz python-simplejson.tcz python_zope.interface-2.7 pyqt.tcz

If you are not sure with required applications needed for building Openvswitch, please read the installation manual.

Now we can run a script that prepares kernel sources for Core Linux. The script downloads patched Linux kernel and kernel config file from tinycore repository, installs extensions required for building kernel sources and build kernel sources.

$ /usr/local/bin/linux-kernel-sources-env.sh

4. Openvswitch Installation

First, we should read how manual describing process of creating extensions for Core Linux.  Then we can set compiler flags.

x86 Core Linux
$ export CFLAGS="-march=i486 -mtune=i686 -Os -pipe"
$ export CXXFLAGS="-march=i486 -mtune=i686 -Os -pipe"
$ export LDFLAGS="-Wl,-O1"

x86-64 Core Linux
$ export CFLAGS="-mtune=generic -Os -pipe"
$ export CXXFLAGS="-mtune=generic -Os -pipe"
$ export LDFLAGS="-Wl,-O1"

Compile Openvswitch.

$ ./configure --prefix=/usr/local --with-linux=/lib/modules/$(uname -r)/build

In case of that configuration finishes with a message as below, Openvswitch does not support your actual kernel version.

configure: error: Linux kernel in /lib/modules/3.16.6-tinycore64/build is version 3.16.6, but version newer than 3.14.x is not supported (please refer to the FAQ for advice)

You might solve it by downloading a higher Openvswitch version or you can try to generate and download source tarball directly from git tree.

$ wget https://github.com/openvswitch/ovs/archive/master.zip

$ unzip master.zip
$ cd ovs-master/

We have generated source tarball thus additional steps are needed before configure script is called. First download following extensions, then issue the commands below.

$ tce-load -wi autoconf automake libtool-dev
$ autoreconf --install --force

$ ./configure --prefix=/usr/local --with-linux=/lib/modules/$(uname -r)/build

$ make -j2

$ sudo su

# cd /home/tc/openvswitch-x.y.z/

or in case you have download Openvswitch from git:

# cd /home/tc/ovs-master/

# make  DESTDIR=/tmp/openvswitch-$(uname -r)  install

5. Backup and Load Module Openvswitch.ko

a) Backup module openvswitch.ko 

After building, a kernel module "openvswitch.ko" is saved in a ./datapath/linux/ directory. We will use this kernel module as performance is better when a kernel module is loaded. Create a directory in a structure of the openvswitch extension where the kernel module openvswitch.ko will be saved.

$ sudo mkdir -p /tmp/openvswitch-$(uname -r)/usr/local/lib/modules/$(uname -r)/kernel/openvswitch/

Copy the kernel module to the directory.

$ sudo cp /home/tc/openvswitch-x.y.z/datapath/linux/openvswitch.ko /tmp/openvswitch-$(uname -r)/usr/local/lib/modules/$(uname -r)/kernel/openvswitch/

or

$ sudo cp /home/tc/ovs-master/datapath/linux/openvswitch.ko /tmp/openvswitch-$(uname -r)/usr/local/lib/modules/$(uname -r)/kernel/openvswitch/

b) Check if  module is loaded

Check if we can load module into the kernel.

$ sudo mkdir -p /usr/local/lib/modules/$(uname -r)/kernel/openvswitch/

$ sudo cp home/tc/openvswitch-x.y.z/datapath/linux/openvswitch.ko /usr/local/lib/modules/$(uname -r)/kernel/openvswitch/

or

$ sudo cp /home/tc/ovs-master/datapath/linux/openvswitch.ko /usr/local/lib/modules/$(uname -r)/kernel/openvswitch/

$ sudo depmod -a &&  sudo modprobe openvswitch

Check if module openvswitch.ko is loaded to the kernel with lsmod command.

lsmod | grep openvswitch
openvswitch 49152 0

6. Backup Vswitch.ovsschema

In order to initialize the configuration database using ovsdb-tool, a file vswitch.ovsschema is needed. We must to copy it to /tmp/openvswitch-$(uname -r)/usr/local/etc/openvswitch/vswitchd/ to include it to the extension.

$ sudo mkdir -p /tmp/openvswitch-$(uname -r)/usr/local/etc/openvswitch/vswitchd/

$ sudo cp /home/tc/openvswitch-x.y.z/vswitchd/vswitch.ovsschema /tmp/openvswitch-$(uname -r)/usr/local/etc/openvswitch/vswitchd/

or

$ sudo cp /home/tc/ovs-master/vswitchd/vswitch.ovsschema /tmp/openvswitch-$(uname -r)/usr/local/etc/openvswitch/vswitchd/

7. Create Openvswitch Extension

a) Remove uneccessary files

$ sudo rm -rf /tmp/openvswitch-$(uname -r)/usr/local/share/man/

b) Install squashfs and create openvswitch-3.0.21-tinycore.tcz extension

$ tce-load -w -i squashfs-tools.tcz

$ sudo su
# cd /tmp/
# mksquashfs openvswitch-$(uname -r)/ openvswitch-$(uname -r).tcz

c) Create a list of files presented in extension

$ sudo su
# cd /tmp/openvswitch-$(uname -r)
# find usr -not -type d > ../openvswitch-$(uname -r).tcz.list
# cd ..

d) Create md5 check sum of openvswitch extension

# md5sum openvswitch-$(uname -r).tcz > openvswitch-$(uname -r).tcz.md5.txt

e) Create info file about openvswitch extension

An info file describing its contents (.tcz.info) - this content is standardized. Check repository for examples.

f) Create openvswitch build dependency file

Additional build instructions in a plain text file for future reference, mentioning such things as which extensions are required to build the package and what compile flags were used. We do not have to attach this file together with files sent with submitted openswitch extension.

g) Create the dependency list for openvswitch extension

List of the extensions that have to be presented to run openvswitch extension correctly.

h) Copy extension and the host

$ tce-load -wi openssh

$ sudo scp -rv openvswitch-$(uname -r).tcz* brezular@10.0.2.2:/home/brezular

8. Test and Submit Openvswitch Extension

a) Start Qemu image with clean install of Core Linux with no extensions installed

$ qemu /home/brezular/core.img -m 1024 --enable-kvm

b) Install submitqc.tcz and openssh.tcz

$ tce-load -wi submitqc5.tcz openssh.tcz

c) Copy Openvswitch extesion files

Copy Openvswitch files we have created in steps 1-7 (.tcz, .list, .md5.txt, .info, .dep) from Qemu image that we have used for Openvswitch installation to Qemu image with Core Linux with no extension installed to the directory  /tmp/All.

$ mkdir /tmp/All
$ cd /tmp/All

$ scp -rv brezular@10.0.2.2:/home/brezular/openvswitch/* .

Run a script for testing extensions:

sudo submitqc5

The script checks all openvswitch files in /tmp/All and creates the directory /tmp/submitqc/ containing the log files.

d) Send extension

$ sudo mv /tmp/submitqc /tmp/All
$ cd /tmp/All

$ sudo tar zcvf openvswitch-$(uname -r)-extension.tar.gz *

Send openvswitch-$(uname -r)-extension.tar.gz  to  tcesubmit@gmail.com

e) Copy new created files to host

$ sudo scp -rv submitqc/ brezular@10.0.2.2:/home/brezular/
$ sudo scp -rv openvswitch-$(uname -r).tcz.zsync brezular@10.0.2.2:/home/brezular/
$ sudo scp -rv openvswitch-$(uname -r)-extension.tar.gz brezular@10.0.2.2:/home/brezular/

9. Openvswitch After-Install Configuration

a) Make openvswitch, 8021q and ipv6 modules to be loaded to  kernel during boot of Core

$ echo "modprobe openvswitch" >> /opt/bootlocal.sh
$ echo "modprobe 8021q" >> /opt/bootlocal.sh
$ echo "modprobe ipv6" >> /opt/bootlocal.sh

$ sudo modprobe openvswitch
$ sudo modprobe 8021q
$ sudo modprobe ipv6

b) Initialize the configuration database using ovsdb-tool

Check if a directory /usr/local/etc/openvswitch/ exists, if not create it.

$ sudo mkdir -p /usr/local/etc/openvswitch/

Create conf.db configuration file.

$ sudo ovsdb-tool create /usr/local/etc/openvswitch/conf.db /usr/local/etc/openvswitch/vswitchd/vswitch.ovsschema

Add /usr/local/etc/openvswitch/ to the list of  persistent files .

$ echo "/usr/local/etc/openvswitch/" >> /opt/.filetool.lst

c) Start ovsdb-server dyring boot of Core

$ vim /opt/bootlocal.sh

/usr/local/sbin/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,Open_vSwitch,manager_options \
--private-key=db:Open_vSwitch,SSL,private_key \
--certificate=db:Open_vSwitch,SSL,certificate \
--bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert \
--pidfile --detach

:wq!

Note:  If you are not familiar with vi editor, use the reference below, please:

http://nemesis.lonestar.org/reference/docs/vi.html

d) Make the database initialialization using ovs-vsctl

This is only necessary the first time after you create the database with ovsdb-tool (but running it at any time is harmless).

$ echo "/usr/local/bin/ovs-vsctl --no-wait init" >> /opt/bootlocal.sh

e) Make Openvswitch daemon being started, telling it to connect to same Unix domain socket

$ echo "/usr/local/sbin/ovs-vswitchd --pidfile --detach" >> /opt/bootlocal.sh

f) Enable IPv4 and IPV6 packets forwarding between interfaces

Although not directly connected with Openvswitch configuration we must enable ipv4 and ipv6 packets forwarding between interfaces of Core.  The option  is disabled in kernel by default.

$ sudo sysctl -w net.ipv4.ip_forward=1
$ sudo sysctl -w net.ipv6.conf.all.forwarding=1

$ echo "sysctl -w net.ipv4.ip_forward=1" >> /opt/bootlocal.sh
$ echo "sysctl -w net.ipv6.conf.all.forwarding=1" >> /opt/bootlocal.sh

g) Run commands in /opt/bootlocal.sh and make them persistent after restart

$ sudo /opt/bootlocal.sh

Save files and directories listed in  /opt/.filetool.lst

$ /usr/bin/filetool.sh -b

h) Delete history

The steps are shown here.

10. Configuration Example

Now you may use ovs-vsctl to set up bridges and other Open vSwitch features.  For example, to create a bridge named br0 and add ports eth0, eth1  and eth2 to it:

$ sudo ovs-vsctl add-br br0

$ sudo ovs-vsctl add-port br0 eth0
$ sudo ovs-vsctl add-port br0 eth1
$ sudo ovs-vsctl add-port br0 eth2

Before shutdown you always force Core to save configuration changes in the Openvswitch database file -  /usr/local/etc/openvswitch/conf.db. Use the command:

$ /usr/bin/filetool.sh -b