Decapsulation ERSPAN Traffic With Open Source Tools

Cisco Encapsulated Remote SPAN (ERSPAN) feature allows to monitor traffic on one or more ports and send the monitored traffic to one or more destination ports.  Traffic is encapsulated into GRE tunnel and routed via network to ERSPAN destination. Any device that supports ERSPAN can be used as ERSPAN destination. It might be another Cisco device or Linux with installed software that can decapsulate GRE traffic.

The goal of this article is to show methods and tools for decapsulation of  ERSPAN traffic. For this purpose I have built simple lab that consists of a Cisco CSR 1000v router and two Linux boxes. Core Linux represents a network host and generates network traffic (ICMP) that is going to be monitored. It is connected to the port GigabitEthernet1 of the Cisco router. The router is configured to monitor traffic on the port Gi1 and it sends traffic encapsulated in GRE tunneling protocol to IP address 10.230.10.1. It is the IP address of the ERSPAN destination configured on Linux Security Union. Security Onion is a unique Linux distro for intrusion detection, network security monitoring, and log management based on Ubuntu however any other Linux distro can be used.

Picture1_Toplogy

Picture 1 - ERSPAN Lab Topology

Below is an example of ERSPAN configuration on the CSR 1000v router. This is the source ERSPAN type and with configured rspan_id 1. The interface Gi1 is being monitored and the GRE traffic is sent to ERSPAN destination address IP 10.230.10.1.

CSR1000v# show running-config | b monitor
monitor session 1 type erspan-source
description ERSPAN to 10.230.10.1
source interface Gi1
destination
erspan-id 1
mtu 1464
ip address 10.230.10.1
origin ip address 10.230.10.2

You also must issue the command no shutdown after the command monitor session 1 type erspan-source in order to activate session.

1. Capturing ERSPAN Traffic with Wireshark

We are going to capture and analyze ERSPAN traffic with Wireshark packet sniffer. First configure IP address 10.230.10.1 on interface eth1 of the Linux Security Onion.

janosik@onion:~$ sudo su
root@onion# ip address add 10.230.10.1/24 dev eth1

Now use Wireshark to capture GRE traffic on Security Onion on its interface eth1 and ping the router IP address 192.168.1.2 from the Linux Core host (IP 192.168.1.1). If the source ERSPAN is properly configured on router, packets from the subnet 192.168.1.0/24 should appear in Wireshark output.

A closer look at the picture below reveals that the original packet ICMP packet (MAC header, IPv4 header and ICMP header) is now encapsulated as following.

MAC header + IPv4 header (10.230.10.2, 10.230.10.1) + GRE header (Protocol type ERSPAN) + ERPAN header + (original packet)

Picture1_Wireshark_Capture

Picture 2 - Encapsulated GRE Traffic Captured on Interface Eth1

An original ICMP packet is encapsulated into GRE tunnel and the new outer MAC and IPv4 + GRE + ERSPAN headers are added to original packets. It allows encapsulated traffic to be forwarded through network to ERSPAN destination. However if we want software application such as IPS/IDS to analyze encapsulated packets, the outer L2 and L3 headers must be striped from packet. This can be done with tools such as RCDCAP  which dissects packets from GRE tunnel.

2. Configuring GRE tunnel on ERSPAN Destination Device

If for some reason we do not want to install special software that dissects packets from GRE tunnel we can configure GRE tunnel on ERSPAN destination (Linux Security Onion) and let IDS to listen on a tunneled interface. Thanks to this configuration the outer MAC and IPv4 headers are stripped and do no appear in Wireshark output.

a) Load GRE module to kernel

janosik@onion:~$ sudo su
root@onion# modprobe ip_gre

You can check if GRE module is loaded into a kernel with command below.

picture3_gre_module_loaded

Picture 3 - GRE Module Loaded Into Kernel

b) Choose receiving interface and assign IPv4 to it

root@onion# ip addr add 10.230.10.1/24 dev eth1

Set the MTU of the network interface that receives GRE packets larger than 1500 e.g. to 1900.  Otherwise we are going to miss some bytes in larger packets.

root@onion# ip link set dev eth1 mtu 1900

c) Create virtual tunnel interface and associate it with IP previously configured on eth1 interface

root@onion# ip tunnel add mon0 mode gre local 10.230.10.1 ttl 8

d) Add IP address to interface mon0 which is not used for anything

root@onion# ip addr add 1.1.1.1/30 dev mon0

e) Change the state of mon0 device to up

root@onion# ip link set mon0 up

Check status of the interface mon0 with the command below. The interface must be in UP state and the Rx counter should increase.

picture4_status_of_mon0_interface

Picture 4 - Status of  Interface mon0

Again, generate network traffic in the subnet 192.168.1.0/24 and configure Wireshark to listen on interface mon0. Notice that the outer MAC and Ipv4 header are now stripped from the ICMP packet.

Picture2_Wireshark_Capture_GRE

Picture 5 - Decapsulated Traffic Captured on Interface mon0

3. Using RCDCAP for Decapsulating ERSPAN Traffic

RCDCAP is wrapper program that dissects the traffic and creates a virtual interface where the traffic is already decapsulated. The tool is great but an installation process can be somehow tricky. For this reason I have written a script install_rcdcap.sh for Ubuntu 16.04.x. The script downloads and installs RCDCap from Mercurial repository. The actual version of RCDCAP installed from repository and successfully tested in scenario is 0.8. The script cares about all dependencies so basically you should only run the script with the command:

$ bash install_rcdcap.sh.txt

Once RCDCAP is installed configure the interface eth1 for capturing.

user@onion:~$ sudo su
root@onion# ip addr add dev eth1 10.230.10.1/24
root@onion# ip link set dev eth1 mtu 1900
root@onion# ip link set dev eth1 up

Start RCDCAP with the command below and let Wireshark to listen on interface mon1.

root@onion# rcdcap -i eth1 --erspan --tap-persist --tap-device mon1 --expression "host 10.230.10.1"

We can see that RCDCAP have dissected monitored traffic from GRE and only original MAC + IPv4 + ICMP headers and pyaload are presented in Wireshark output.

Picture3_Wireshark_Capture_Dissected

Picture 6 - Decapsulated Traffic Captured on Interface Mon1 

Note: To test RCDCAP on Ubuntu Server OS (without GUI), issue the command below.

root@onion# rcdcap -i eth1 --erspan --tap-persist --tap-device mon1 --expression "host 10.230.10.1" &

Afterwards, start capturing a network traffic with tcpdump on the interface mon1 with the following command.

root@onion# tcpdump -i mon1 -v

End.

Reference:
http://packetpushers.net/erspan-new-favorite-packet-capturing-trick/
http://networkengineering.stackexchange.com/questions/3274/remote-sniffing-with-erspan-to-the-desktop

33 thoughts on “Decapsulation ERSPAN Traffic With Open Source Tools

  1. I am following your setup, but I am failed to decrypt the ERSPAN packet generated from VMware. any idea?

    1. Paul
      I am failed too with decrypt the ERSPAN packet generated from VMware
      Because VMware does not use ERSPAN. it use only GRE tunnel

      Did anyone ever got to Decapsulating traffic from the GRE tunnel (noe ERSPAN)?

  2. Is there a way to ENCAPSULATE traffic into an ERSPAN tunnel without using a cisco device? What if I wanted to source ERSPAN traffic from a Linux host instead of a switch?

    1. Did you figure it out? I need ERSPAN between EXtrahop and Vmware so not sure if I just need to connect my EXH port as an access port ???

  3. Hi I've tried following the steps above for the ip_gre tunnel and no packets are being received on the end of the tunnel.

    The ERSPAN from the Cisco router is arriving at interface eno1 so I have assigned IP address a.b.c.d/x to that interface and I have added an ip tunnel using 'ip tun add eno_gre mode gre local a.b.c.d/x ttl 255'. Then added ip addr 1.1.1.1/x to dev eno_gre and set it to up. When I do ifconfig on the eno_gre interface the Rx count is 0, despite packets arriving on eno1. Anybody any ideas what I am doing wrong?

          1. It's interesting. I copied the steps from the article to a clean installation of Ubuntu yesterday and it was working like a charm. Could you Post the exact commands you entered? I will try to reproduce them.

          2. I am using a clean install of security onion and i entered the commands exactly like you posted. Perhaps that is why is not working correctly. What version of ubuntu are you using?

      1. Hi, we have also followed the suggeset setup and are the same issue as Luis. When you say check iptables - forward chain what configuration do you expect?

        We have added iptables -A FORWARD -p gre -j ACCEPT

        Thanks

        1. We the same issue running centos7, packets arrive at Eth interface but never make it to the GRE tunnel. has anyone got any idea?

    1. how did you make it work?
      me and other people reply to this post never got it to work...
      erspan packets that sent to the host never tunnel to the gre... only the opposite way...

  4. Hi all, is there any way how to change RCDCap configuration for decapsulating erspan version 3 ????

    The problem is, that I'm still getting gre encapsulated dump and I think it's caused by erspan version header type 0x22eb. ERSPAN v1 header type is 0x88be

    My dump from RCDCap:
    (tos 0x28, ttl 25, id 64066, offset 0, flags [none], proto GRE (47), length 262)
    10.132.63.1 > localhost.localdomain: GREv0, Flags [sequence# present], seq 832122434, length 242
    gre-proto-0x22eb

  5. I am seeing no traffic on mon0 as well.
    Also, after installing rcd using the script, it seems to complete successfully but, when I run the command, I receive "command not found: rcdcap.
    Help appreciated.

      1. Thanks for your reply. Funny I had already searched and below are the results:
        ~/rcdcap-code# find / -name rcdcap
        /root/rcdcap-code/.hg/store/data/source/include/rcdcap
        /root/rcdcap-code/.hg/store/data/test/rcdcap
        /root/rcdcap-code/source/include/rcdcap
        /root/rcdcap-code/test/rcdcap

      2. I've installed it using the script twice. What could be the issue? Below is the version

        Linux version 3.16.0-4-amd64 (debian-kernel@lists.debian.org) (gcc version 4.8.4 (Debian 4.8.4-1) ) #1 SMP Debian 3.16.43-2+deb8u5 (2017-09-19)

          1. I ran a third time and now yes I am seeing errors. It seems the only way I will be able to resolve this erspan decapsulation issue is to pull the erspan logs into onion and then forward them somehow to the siem interface where I want to correlate them.

  6. I had the same issue that mon0 interface receives nothing.
    In my case, the ERSPAN traffic version == 0 (type = I) which not be supported by linux os (only version == 1 or 2). (I checked this by showing packets in Wireshark).
    Maybe you counter same problem.
    Hope it will help!

  7. I was able to get this to work on Ubuntu 16.04, same as others.

    There was an error "implicit declaration of function 'ACCESS_ONCE' [-Werror=implicit-function-declaration]"

    To get around that you can do the following.

    sed -i 's/ACCESS_ONCE/READ_ONCE/' /root/PF_RING-6.6.0/kernel/pf_ring.c

    One thing I am wondering about - not sure if anyone else has the same behavior and knows how to get around it.

    The mon1 interface is set to 10Mbit speed. Is there a way to increase this tap device to 1Gbit?

    $ ethtool mon1
    Settings for mon1:
    Supported ports: [ ]
    Supported link modes: Not reported
    Supported pause frame use: No
    Supports auto-negotiation: No
    Advertised link modes: Not reported
    Advertised pause frame use: No
    Advertised auto-negotiation: No
    Speed: 10Mb/s
    Duplex: Full
    Port: Twisted Pair
    PHYAD: 0
    Transceiver: internal
    Auto-negotiation: off
    MDI-X: Unknown
    Current message level: 0xffffffa1 (-95)
    drv ifup tx_err tx_queued intr tx_done rx_status pktdata hw wol 0xffff8000
    Link detected: yes

  8. Hi! I just want to thanks for the tutoria, is perfect and it was so useful for a project in my job! I installed under RedHAT and I did an small step by step for that. In case someone need it, you have to install these package:
    yum install mercurial.x86_64
    yum groupinstall "Development Tools"
    yum install boost-devel libpcap-devel libstdc++-static cmake
    yum install yum-utils
    yum install net-tools
    yum install "kernel-devel-uname-r == $(uname -r)"
    yum-config-manager --enable rhel-7-server-optional-rpms
    yum install libpcap-devel
    yum install libstdc++-static-4.8.5-36.el7.x86_64

    And the PFRING, just this repo and intructions from this link:
    http://packages.ntop.org/centos-stable/

    Thanks again, and oustanding tutorial!

  9. Hi Brezula.

    Thank you for the tutorial. I just want to know if I can set the RCDCap using below setup:

    Source --> ERSPAN Source --> ERSPAN Destination --> RCDCap --> Logger. My point is I want to strip the ERSPAN header before its send to the Logger. How I want to setup the RCDCap in this manner? I have a ubuntu (running on virtualbox) with 2 NIC. Thanks in advance.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.