Nprobe includes both a NetFlow v5/v9/IPFIX probe and collector. In a probe mode, nProbe captures packets on an interface and exports flows to a flow collector (Noction Flow Analyzer, Ntop etc.).
If configured in a collector mode, it collects flows from other device such as routers/switches and export them via TCP streaming to Syslog, Splunk etc. (parameter -tcp). They are other modes available such as proxy or ips mode. For example in a proxy mode, nProbe can collect sFlow or NetFlow v5/v9/IPFIX flows and send them towards the collector as NetFlow v5/v9/IPFIX flows.
In the first part of this tutorial, we will check the visibility of Layer 7 applications in NetFlow v9 flows and IPFIX messages exported using Nprobe. In the second part of the tutorial, we will enable the HTTP plugin and check that nProbe correctly decodes the HTTP network traffic parameters and exports them in IPFIX messages.
The network infrastructure is shown on the Figure 1.
Picture 1 - Network Topology with nProbe Pro
Used Software:
- Debian GNU/Linux 10 (buster) kernel 4.19.0-6-amd64 #1 SMP Debian 4.19.67-2 (2019-08-28) x86_64 GNU/Linux
- nProbe v.9.1.201023 (r6969)
- pfring 7.9.0-3252
1. nProbe Layer-7 Visibility Testing
The nProbe version 6.7 and later allows us to discover more than 250 applications including Skype, BitTorrent and Citrix and advertise the application name in exported flows. Application visibility at Layer 7 is achieved by adding the following information elements:
- [NFv9 57590][IPFIX 35632.118][Len 2] %L7_PROTO Layer 7 protocol (numeric)
- [NFv9 57591][IPFIX 35632.119][Len 16 varlen] %L7_PROTO_NAME Layer 7 protocol name
- [NFv9 57973][IPFIX 35632.501][Len 16 varlen] %L7_PROTO_CATEGORY Layer 7 protocol category
The L7_Protocol_Maping_nProbe.txt file contains a list of mapping L7 application protocol numbers to application names. However, it may be out of date; therefore, visit the ntop - nprobe website to get the latest information.
Note: The nProbe version 9.6 adds %L7_RISK_SCORE extension which associates flow risk score with a flow.
We use the following nProbe parameters to test Layer-7 visibility of the applications:
$ sudo nprobe -i Gi1 -V 10 -n 10.0.0.1:2055 -T="%SAMPLING_INTERVAL %IN_BYTES %IN_PKTS %IPV4_SRC_ADDR %IPV4_DST_ADDR %IPV4_NEXT_HOP %L4_SRC_PORT %L4_DST_PORT %SRC_TOS %TCP_FLAGS %PROTOCOL %IP_PROTOCOL_VERSION %DIRECTION %FLOW_START_MILLISECONDS %FLOW_END_MILLISECONDS %IN_SRC_MAC %OUT_DST_MAC %flowStartSeconds %flowEndSeconds %BIFLOW_DIRECTION %L7_PROTO %L7_PROTO_NAME %L7_PROTO_CATEGORY %INTERFACE_NAME" -t 60 -d 15 -l 60
- n: collector address
- V: Flow export version 10 - IPFIX, 9 (v9), 5 (v5)
- T: Flow template definition. Contrary to NetFlow v5 where the flow format is fixed, NetFlow V9 and IPFIX flows have a custom format that can be specified at runtime using this option.
-t: maximum flow lifetime. Regardless of the flow duration, a flow that has been active for more that the specified maximum lifetime is considered expired and it will be emitted. Further packets belonging to the same flow will be accounted on a new flow.
-d: maximum flow idle lifetime. A flow is over when the last packet received is older that the maximum flow idle lifetime.
-l: maximum queue timeout. It specifies the maximum amount of time that a flow can be queued waiting to be exported. Use this option in order to try to pack several flows into fewer packets, but at the same time have an upper bound timeout for queuing flows into the probe.
Tested Applications:
- Game Urban Terror
- Team viewer
- Youtube
- Skype
- Skype Call
- Tor
Note: NProbe Pro must be properly licensed otherwise it is limited to 25000 flows.
1.1 Unknown Application Detection
Even though nProbe can recognize more than 250 apps, there are still apps that it is unable to detect. When the application is not recognized, the Layer 7 protocol number is 0 and the Layer 7 protocol name is unknown. To simulate this scenario, I started the Urban Terror client on a local machine and connected the client to a game server running on the socket 91.121.66.83:27960 (UDP).
The Layer 7 protocol category value is 55:6e:73:70:65:63:69:66:69:65:64, which gives us the value "Unspecified" after converting from hexadecimal to ASCII (Figure 2).
Figure 2 - IPFIX Flow with Unknown Application - Game Urban Terror
Flow 1 with destination IP address 91.121.66.83 (UT server) inside the Ethernet frame 5, found in the Wireshark capture urban_terror.zip can be easily detected using the following display filter:
cflow.dstaddr == 91.121.66.83
1.2 Facebook
The Wireshark capture facebook.zip contains the Ethernet frame 9, the IPFIX flow 8 (Figure 3) with detected Facebook application. The L7 protocol number is 119 (Facebook), the category is Social network. The IP address of the Facebook server is 31.13.84.15, TCP port 443 (HTTPS).
Figure 3 - IPFIX Flow with Facebook Application
The Wireshark display filter for Facebook detection is cflow.pie.ntop.l7_proto_name == "Facebook".
1.3 TeamViewer Remote Access
The TeamViewer remote desktop application can connect two computers located behind NAT. The L7 protocol number is 148, the protocol name is Teamviewer and the category is Remote access (Figure 4). The IP address of the Teamviewer server is 188.172.246.165, the destination TCP port 5938. The flow with detected TeamViewer application is number 7, inside the Ethernet frame 1 in the file team_viewer.zip.
Figure 4 - IPFIX Flow with TeamViewer Application
The Wireshark display filter for TeamViewer is cflow.pie.ntop.l7_proto_name == "TeamViewer"
1.4 Youtube
The nProbe can also detect Youtube service, which certainly needs no introduction. The L7 protocol number is 124, the protocol name is TLS.Y ouTube and the category is Media (Figure 5). The IP address of the Teamviewer server is 172.217.23.238, the destination TCP port 443. The flow 8 with detected Youtube service is inside the Ethernet frame 6, Wireshark capture youtube.zip.
Figure 5 - IPFIX Flow with YouTube Application
Wireshark display filter:
cflow.pie.ntop.l7_proto_name == "TLS.YouTube"
1.5 Skype Application and Calls
Skype application (nProbe l7 protocol number 125, l7 name TLS.Skype, category VOIP) can be also detected by nProbe (Figure 6). The IPFIX message with flow number 6 inside Ethernet frame number 3 is in the Wireshark capture skype.zip
Figure 6 - IPFIX Flow with Skype Application
Wireshark display filter:
cflow.pie.ntop.l7_proto_name == "TLS.Skype"
Besides the Skype application detection, nProbe is also able to detect a skype call in a network traffic and report it in IPFIX messages. (Figure 7). The flow number 6 is inside Ethernet frame 27 in the Wireshark capture skype_call.zip. The IP address of the Skype server is 20.202.4.53 and the port is UDP 3478. The L7 protocol number definied by nProbe is 37, the L7 application name is STUN.SkypeCall and the category is VOIP.
Figure 7 - IPFIX Flow with Skype Call
Wireshark display filter:
cflow.pie.ntop.l7_proto_name == "STUN.SkypeCall"
1.6 Tor Browser
As the last step, we will use nProbe to detect anonymous Internet browsing with Tor. nProbe L7 protocol name is TLS.Tor, L7 protocol number is 163 and the category is VPN. The flow number 4 is inside Ethernet frame 2 in the Wireshark capture tor.zip.
Figure 8 - IPFIX Flow with Tor Application
Wireshark display filter:
cflow.pie.ntop.l7_proto_name == "TLS.Tor"
2. nProbe PLugins
Optional plugins e.g. HTTP, DNS, DHCP, FTP, SIP/RTP etc. extend the functionality of nProbe. For example, the HTTP plugin allows the nProbe application to decode HTTP traffic and report HTTP sites accessed, the mime-types or return code etc. Now, let's discuss the nProbe versions and licences. Basically, there are three versions of nProbe available:
- nProbe Pro
- nProbe Enterprise M
- nProbe Enterprise S
For nProbe Pro version, individual plugins must be licensed( except the BGP plugin).
The nProbe Enterprise M license is the new version of nProbe Pro that includes all plugins (Figure 9). The version nProbe Enterprise S is the new name of the former nProbe Pro license with DNS and HTTP plugins only. To test HTTP traffic decoding we either need nProbe Pro with HTTP plugin license or the nProbe Enterprise S edition. Of course, Enterprise M edition would be overkill in our scenario.
Figure 9 - nProbe Versions and Licensing
Now, we can start nProbe instance with the following HTTP options:
$ sudo nprobe -i Gi1 -V 10 -n 10.0.0.1:2055 -T="%SAMPLING_INTERVAL %IN_BYTES %IN_PKTS %IPV4_SRC_ADDR %IPV4_DST_ADDR %IPV4_NEXT_HOP %L4_SRC_PORT %L4_DST_PORT %SRC_TOS %TCP_FLAGS %PROTOCOL %IP_PROTOCOL_VERSION %DIRECTION %FLOW_START_MILLISECONDS %FLOW_END_MILLISECONDS %IN_SRC_MAC %OUT_DST_MAC %flowStartSeconds %flowEndSeconds %BIFLOW_DIRECTION %L7_PROTO %L7_PROTO_NAME %L7_PROTO_CATEGORY %INTERFACE_NAME %HTTP_URL %HTTP_METHOD %HTTP_RET_CODE %HTTP_UA %HTTP_MIME %HTTP_HOST %HTTP_VIA" -t 60 -d 15 -l 60
IPFIX message with decoded HTTP traffic is shown on the Figure 10. Notice the HTTP traffic information such as HTTP URL, method, return code, user agent etc.
Figure 10 - HTTP Traffic Decoded in IPFIX Flow
The Ethernet frame 4, flow number 1 contains information about HTTP traffic inside the Wireshark capture http.zip.
Conclusion
nProbe is a great tool and the de facto standard among NetFlowv9/IPFIX exporters. At the same time it does not cost too much, e.g. nProbe Enterrpise M costs only 938,61 EUR. For this price you get all available plugins.