Openswitch OPX Installation on Linux

We have recently covered installation of Openswitch OPS on Linux. Since the version 2.0, Openswitch OPS has transformed into to a completely new project, called Openswitch OPX Base. Similar to its predecessor, OpenSwitch OPX Base system also provides an abstraction of hardware devices of network switch platforms in a Linux OS environment. However, original Yocto OS has been replaced by an unmodified Linux kernel based on Debian Jessie distribution.

We can install OPX Base on a virtual machine, similar to installing OpenSwitch on hardware platforms. A virtual machine (VM) uses the same software binaries as those executed on S6000-ON devices. The main difference is that the low-level device drivers for the SAI and SDI libraries are replaced with the packages that support hardware simulation, and interact with the hardware simulation infrastructure.

A host machine running Openswitch OPX VM might be Windows, or Mac OS X with at least 8GB of RAM and 100GB available disk space, and Virtual Box installed. The virtual machine needs to have one network interface configured for the Management interface (eth0). The network adapter eth0 corresponds to the first adapter attached to the VM, e101-001-0 to the second adapter and so on, and e101-00N-1 to adapter N+1. All the virtual network adapters should be the same type (including adapter 1 for the management port). The Intel PRO/1000 MT Desktop (82540EM) is recommended for VirtualBox.

We will discuss installation steps for the host (Linux Ubuntu) using lvm utility. The lvm is OS10 VM tool for management of VM sessions running inside VirtualBox virtualizer. The tool can be used for create, delete, restart, delete all instances, or restart a session. However, I prefer Qemu as back-end virtualizer inside GNS3 to create and run Openswitch OPX instances inside GNS3. For this reason,  we are going to use lvm only for creating Openswitch VirtualBox VM. Once the Openswitch Vbox VM is created, we will extract the VBox HDD and copy it to GNS3 project directory.

Note: The lvm tool for Windows OS is called called vm.exe and nvm for Mac OS.

Note:  I'm sharing my OpenswitchOPX 2.3.2 VDI disk that is customized with my after install script (read the part 2). In order to make your  configuration persistent, copy your commands to /etc/rc.local between the lines starting with done and exit 0. Here is the example such a file.

1. Openswitch OPX Installation

The commands below downloads lvm and assign executable privileges to it.

$ wget http://archive.openswitch.net/vm-tools/lvm
$ chmod +x lvm

Now we create a new VirtualBox instance named openswitch with the lvm tool. You can choose the name according to your needs.

$ ./lvm create openswitch

Picture 1 - Openswitch OPX Installation

Once the command is executed, an installation process begins. Firstly, the command downloads the following files to the current directory.

• PKGS_OPX-stable-installer-x86_64.bin
• onie_kvm.iso

Note: The offline installation is also supported by lvm tool. You need to download the files manually and start the installation afterwards. See the example below.

$ wget http://archive.openswitch.net/vm-tools/lvm
$ wget https://dell-networking.bintray.com/opx-images/onie-kvm_x86_64-r0.iso
$ wget http://archive.openswitch.net/installers/stable/Dell-EMC/PKGS_OPX-stable-installer-x86_64.bin

Finally, we can start manual (offline) installation using lvm utility.

$ ./lvm create openswitch --iso onie_kvm.iso --bin PKGS_OPX-stable-installer-x86_64.bin

However, it is recommended to use online installation as it automatically  downloads the latest OPX files.

The file PKGS_OPX-stable-installer-x86_64.bin is a shell script executable that contains installation and configuration scripts along with Openswitch OPX VM file system. Once the command ./lvm create openswitch is started, lvm starts the script PKGS_OPX-stable-installer-x86_64.bin. Firstly, the script itself decodes base 64 encoded text located between the line starting with __INSTALLER__ and ending with __IMAGE__ inside the script PKGS_OPX-stable-installer-x86_64.bin. The decoded text is the tar package that is being extracted to the directory installer under the current directory by the script PKGS_OPX-stable-installer-x86_64.bin. The directory installer contains an installation script install_support.sh  along with other installation scripts and configuration files.

Afterwards, the script PKGS_OPX-stable-installer-x86_64.bin continues an installation  process starting the script install_support.sh. The script install_support.sh runs individual support module scripts located inside the ./lib directory. We do not need to explain a function of each individual script. However, it is worth to mention that the script os_core.sh extracts the tar package located after the line __IMAGE__ inside the script PKGS_OPX-stable-installer-x86_64.bin.  This tar package contains Openswitch OPX filesystem.

./lib/common.sh
./lib/grub.sh
./lib/nos_probe.sh
./lib/onie_support.sh
./lib/os_core.sh
./lib/partition.sh

When the installation process finishes, shutdown the VM openswitch using VM VirtualBox manager with CTRL+F  keys. The VirtualBox hard disk openswitch.vdi is located inside the home directory,  in the directory temp.

Openswitch VM settings are located inside the Vbox configuration file openswitch.vbox. We need all these parameters in order to configure GNS3 Openswitch Qemu VM correctly later. The file is stored in the directory ~/temp/openswitch/. Typically, each Openswitch OPX VM has assigned 2 CPUs, 2048MB RAM and 6x network adapters 82540EM. It has also mounted file onie-kvm_x86_64-r0.iso as the second IDE Master (CDROM). The first network adapter is used for management. It is configured in NAT mode by VirtualBox and forwarded to the 127.0.0.1 port 2222. If you prefer Virtualbox to Qemu,  you can use the command below to connect to the Openswitch OPX  VirtualBox instance from a host using ssh.

$ ssh -p 2222 opxUser@127.0.0.1

Now we can copy the file openswitch.vdi to the location where GNS3 files are stored.

$ cp ~/temp/openswitch/openswitch.vdi to ~/GNS3-files

2. After Install Steps

I have created the after install script install.sh that customizes our Openswitch OPX installation. First, the script installs and starts Quagga Routing Suite that we need for support of the advanced L3 routing. Afterwards,  it changes the default timeout for network interfaces configuration from 5 minutes to 15 seconds. It helps to decreases the booting time when DHCP server is not running in your network. In this case, the management interface eth0 is waiting for automatic IP address assignment via DHCP, however DHCP server is not available.

The script also flushes ebtables (Linux Ethernet bridge firewalling) and make changes persistent saving them to the file /usr/bin/dn_rules.sh.  You can check the default ebtables configuration with the command below.

# ebtables -L

Along with other rules, the default configuration contains the rules below. These rules drop forwarding ARP messages between interfaces.  It effectively prevents to establish connection between the hots connected to the OPX interfaces as they cannot learn MAC address of their counterparts.

ebtables -D FORWARD -d Broadcast -j mark --mark-set 0x2 --mark-target CONTINUE
ebtables -D FORWARD -d Multicast -j mark --mark-set 0x1 --mark-target CONTINUE
ebtables -D FORWARD -p ARP -j DROP

The script also contains modified /etc/rc.local file.  I use this file for storing configuration of all interfaces (e101-0xx-0, bridges). The script contains the while cycle that checks for presence of the interface e101-032-0 during the boot of  OpenswitchOPX instance. The interface e101-032-0 is loaded as the last one during the boot. When the interface is presented, my script loads the configuration stored in /etc/rec.local.  I have to insert the while cycle in the /etc/rec.local to postpone loading configuration stored in /etc/rc.local during boot. The interfaces e101-001-0 -  e101-032-0 are not presented in OpenSwitchOPX right after its boot. In case, we do not wait for the presence of the interfaces, their configuration would be ignored.

In order to make your  configuration persistent, copy your commands to /etc/rc.local between the lines starting with done and exit 0. Here is the example of rc.local containing such configuration.

2.1 Running after install script

Run the Openswitch OPX instance with Qemu virtualizer.

$ /usr/local/bin/qemu-system-x86_64 -m 2G -cpu host -smp cores=2 openswitch.vdi -enable-kvm -serial telnet:127.0.0.1:8888,server,nowait

Connect to vty lines with the telnet command.

$ telnet 127.0.0.1 8888

Login name is opxUser with no password set. Create a file install.sh using vi editor and copy the content of my script install.sh to the file install.sh.

$ vim install.sh

Close the file and assign execute privileges to the file. Start the script as a root.

$ sudo su
$ chmod +x install.sh
# ./install.sh

3. Testing

We will use GNS3 to create a simple testing topology that contains a single Openswitch OPX 2.3.2 instance and two computers connected to the ports Ethernet1/0 and Ethernet2/0. These interfaces are presented as the interfaces e101-001-0 and e101-002-0 in the CLI of Openswitch OPX. The first interface e0/0 has no computer connected. It is presented as the management interface eth0 in the Openswitch OPX CLI.

Picture 1 - Network Topology

We will create a bridge interface br0 and assign the both ports to the bridge.

$ sudo su

# brctl addbr br0

# brctl addif br0 e101-001-0
# brctl addif br0 e101-002-0

# ifconfig br0 up
# ifconfig e101-001-0 up
# ifconfig e101-002-0 up

The picture 2 depicts interfaces assigned to the bridge br0.

# brctl show

Picture 2 - Assigned Interfaces to Bridge br0

Now we can test connectivity between Server1 and Server2 with the ping command.

Picture 3 - Testing  Connectivity Between Hosts

4. Issues

When OpenswitchOPX fails to bring interfaces e101-0xx-0 to up, the while cycle in /etc/rc.local will loop infinitely as it waits for their presence. In this case, just restart your OpenswitchOPX instance.

Picture 4 - OpenswitchOPX Boot Fails

End.