Cisco TRex on Ubuntu Server 18.04

TRex is stateful and stateless traffic generator that is designed to benchmark platforms using realistic application traffic. It can generate L3-7 traffic and scale up to 20Gbps. TRex implements the both client and server side. The tutorial provides exact steps that you can follow to install TRex on Ubuntu Server 18.04. Hope you find it useful.

1. Preparing Qemu Ubuntu Server VM for TRex Installation

I assume that you have installed Ubuntu Server 18.0.4 on Qemu disk. Start Qemu Ubuntu Server VM.

$ /usr/local/bin/qemu-system-x86_64 -m 4G -enable-kvm Ubuntu18.04-server-TRex2.vmdk -serial telnet:localhost:2222,server,nowait

Assign IP address from the internal Qemu DHCP server to the guest NIC of Ubuntu Server VM.

$ sudo dhclient

IP address is 10.0.2.15/25 and the default gateway is 10.0.2.2. Now you can connect to the Ubuntu Server issuing telnet from the host.

$ telnet localhost 2222

Copy my script trex-vm.sh from the host to Ubuntu Server Qemu VM. The script installs packages required by TRex. It also redirects VM machine output to serial port and configures old-style Ethernet interfaces naming.

$ scp -rv brezular@10.0.2.2:/home/brezular/trx-vm.sh .

Assign execute privileges to the script and run the it. $ sudo su
# chmod +x trex-vm.sh
# ./trex-vm.sh

When the script finishes, reboot the VM with the reboot command. After boot assign the IP address from DHCP QEMU server to the Ubuntu server.

$ sudo dhclient eth0

2. TRex Installation

2.1 DPDK Installation Using TRex Core

DPDK is the Data Plane Development Kit that consists of libraries to accelerate packet processing workloads running on a wide variety of CPU architectures.  Build DPDK with the commands below.

$ wget https://github.com/cisco-system-traffic-generator/trex-core/archive/master.zip
$ unzip master.zip
$ cd trex-core-master/

$ cd linux_dpdk/
$ ./b configure
$ ./b build

Build DPDK kernel module igp_uio with the command below.

$ cd ..
$ cd scripts/ko/src/
$ make
$ sudo make install

$ sudo modprobe uio
$ sudo depmod -ae
$ sudo modprobe igb_uio

Add modules uio, igb_uio to the list of modules loaded after reboot.

$ sudo su
# echo "uio" >> /etc/modules
# echo "igb_uio" >> /etc/modules

Picture 1 - Modules uio, igb_uio Loaded in Linux Kernel

2.2 TRex Installation

Download and extract the latest TRex.

$ mkdir trex
$ cd trex
$ wget --no-cache http://trex-tgn.cisco.com/trex/release/latest
$ tar zxvf latest
$ cd ~/trex/v2.49/

We have finished TRex installation on Ubuntu 18.04 Server Qemu VM. We are going to deploy the TRex VM into GNS3 project.

3. Importing TRex Qemu VM into GNS3

Create TRex QEMU instance inside GNS3. Below are  GNS3 Qemu Guest settings for TRex Qemu VMs.

• 4x NIC Intel GigabitEthernet e1000
• 4x CPU
• 4096 MB RAM

TRex Qemu VM is connected to DUT (Device Under Test) with the interfaces eth0 and eth1 (Picture 2). TRex implements the role of the both - clients and servers, running on the same VM. Clients generate traffic that is sent from DUT back to the servers. The router R1 is DUT and it is running Cisco vIOS-L3. It is connected via the interfaces Gi0/0 and Gi0/1 to TRex VM (Ubuntu Server).

Picture 2 - Network Topology

TRex does not implement ARP so we need to configure  DUT's MAC addresses of the directly connected interfaces ((Picture 3). We will also configure  DUT to match the default MAC configured on TRex's interfaces.

R1# show interfaces gigabitEthernet 0/0 | include address
R1# show interfaces gigabitEthernet 0/1 | include address

Picture 3 - Checking MAC Addresses of Interfaces Gi0/0 and Gi0/1 on DUT (R1)

We will add the R1's MAC addresses of Gi0/0 (0cbb.1385.5100) and Gi0/1 (0cbb.1385.5101) into the TRex configuration file later. However, we also need to add MAC addresses of TRex's interfaces eth0 and eth1 into the TRex configuration file. We can find this information with ifconfig (Picture 4) in TRex CLI.

$ ifconfig eth0 | grep ether
$ ifconfig eth1 | grep ether

Picture 4 - Checking MAC Addresses of Interfaces eth0 and eth1 on TRex VM

The MAC address of the interface eth0 is 0c:bb:13:6c:88:00 and the MAC of the eth1 is 0c:bb:13:6c:88:00.

We have to create configuration file for TRex. The configuration file contains the MAC addresses of the DUT's interfaces, MAC addressed of TRex interfaces and their (Bus Device Functions) BDFs. Navigate to the directory v2.49 where the scripts are stored.

$ cd ~/trex/v2.49/

Identify BFD of the interfaces eth0 and eth1 that connect DUT to TRex. We will use the script dpdk_nic_bind.py for this purpose. The device eth0 has assigned the BDF 00:03.0 and the device eth1 BFD is 00:04.0 (Picture 5). Notice that we have omitted the first four leading zeroes (domain numbers).

$ sudo ./dpdk_nic_bind.py --status

Picture 5 - Identification of TRex BFD for NICs Eth0 and Eth1

Now we have all the pieces needed by TRex configuration file so we can create it (Picture 6).

$ sudo vi /etc/trex_cfg.yaml

Picture 6 - Content of TRex Configuration File

4. Configuring L3 Connectivity

In this part we are going to configure the router R1 for L3 connectivity. We will also add a static ARP entries.

R1(config)# interface gigabitEthernet 0/0
R1(config-if)# ip address 192.168.1.2 255.255.255.0
R1(config-if)# no shutdown

R1(config)# interface gigabitEthernet 0/1
R1(config-if)# ip address 192.168.2.2 255.255.255.0
R1(config-if)# no shutdown

Define static routes to clients (16.0.0.0/8) and servers (48.0.0.0/8) simulated on running on TRex.

R1(config)# ip route 16.0.0.0 255.0.0.0 192.168.1.1
R1(config)# ip route 48.0.0.0 255.0.0.0 192.168.2.1

Static ARP entries contains of combination of IP and MAC addresses of TRex interfaces. 

R1(config)# arp 192.168.1.1 0cbb.1351.3d00 ARPA
R1(config)# arp 192.168.2.1 0cbb.1351.3d01 ARPA

5. Testing

 TRex scripts are located in the directory v2.49.

$ cd ~/trex/v2.49/

Run TRex server in the background.

$ sudo ./t-rex-64 -i &

Ethernet ports eth0 and eth1 are now binded to DPDK and cannot be found with the ifconfig command. Check the status with the script below.

$ sudo ./dpdk_nic_bind.py --status

Picture 7 - Ethernet Ports eth0 and eth1 Binded to DPDK

Start sending DNS traffic for 100s (-d) using one CPU core (-c) and configuration file (-f) (Picture 8 and 9) from clients to the servers.

$ sudo ./t-rex-64 -f cap2/dns.yaml -d 100 -c 1

Picture 8 - Generating DNS Traffic

Picture 9 - Generating DNS Traffic

The configuration file depicted on the Picture 10 dns.yaml contains:

• the number of clients (512) (16.0.0.1 - 16.0.1.255) that will send traffic to 255 servers (48.0.0.1 - 48.0.0.255).
• path to pcap file: cap2/dns.pcap

$ cat ~/trex/v2.49/cap2/dns.yaml

Picture 10 - The Content of Configuration File dns.yaml

Picture 11 depicts the structure of the file dns.pcap that used as a traffic template for smart  replay of DNS traffic.

$ tcpdump -ttttnnr cap2/dns.pcap  

Picture 11 - Content of DNS Pcap File

The screenshot of Wireshark window showing the packets captured on the interface Gi0/1 of DUT (R1) is depicted on the Picture 12. You can also download the file dns_trex_traffic.pcap.zip that is DNS traffic traffic generated by TRex.

Picture 12 - Wireshark DNS Traffic Capture

End.