Part2 - Monitoring Network Traffic with ntopng and nProbe

In the Part1 we have covered compilation of ntopng on Ubuntu 18.04.1 Server and installation of nProbe on Raspberry Pi 3. We have also configured Cisco Catalyst switch 3550 for traffic mirroring. The source of the traffic is the interface Fa0/3 where PC is connected and the destination port is Fa0/24 with connected Raspberry Pi 3B. This tutorial goes further and it covers configuration of the both ntopong and nProbe. First, let's have a look at the network topology.

Picture 1 - Network Topology

nProbe is running on Raspberry Pi 3B with the IP address 172.17.100.50/16. ntpong is running on Ubuntu 18.04.1 Server as the VirtualBox guest with the IP address 172.17.100.7/16. Our goal is to monitor network traffic from all devices connected to the ports of Cisco Catalyst switch 3550. In fact, we are monitoring only Windows 7 machine that is connected to Fa0/3.

Note: Without valid license, nProbe is working in demo mode and it is limited to show only 25000 flows export.

nProbe can work in two modes - poll mode and push mode. In a poll mode, ntopng dynamically subscribes to the probe via ZMQ, telling the probe what type of flow data it is interested in. The probe sends ntopng only this information, without sending all flows to ntopng as probes do. This practice optimises network traffic and limits the CPU cycles to those really necessary to carry on to collect flows. Below is the configuration of ntopng and nProbe in a poll mode.

# nprobe -n none --zmq "tcp://*:1234"

# ntopng -i "tcp://172.17.100.50:1234"

Note: in case you encounter problem with user, start ntopng with the option --dont-change-user.

In push mode, nProbe sends data to ntpong (collector) as soon as possible. This architecture is suboptimal as the probe is pushing the same data to all collectors (i.e. collector X cannot tell the probe that it is interested only to HTTP-based flows, but it has to collect everything and discard unneeded information). Also because in case a new collector has to be added, the probe has to be reconfigured. Below is our configuration for push mode.

# nprobe -i eth0 --zmq "tcp://172.17.100.7:5556" --zmq-probe-mode

# ntopng -i "tcp://*:5556c"

ntopng subscribes to nProbe for second-based throughput statistics, so that you will now see in ntopng realtime throughput statistics even if you configure the probe to aggregate flows to 1 min or more. In this case we had to add a ‘c’ on the ntopng side to declare it as collector endpoint.

1. Testing Push Mode

Login to Ubuntu server and start ntopng as following:

$ sudo su
# ntopng -i "tcp://*:5556c"

Picture 2 - Running Collector - ntpong on Ubuntu Server 18.04.1

Note: To start ntopng after boot, copy & paste the following lines to CLI (without hashtag at the beginning of the lines).

# echo -e '#!/bin/bash\n' > /etc/rc.local
# echo '/usr/local/bin/ntopng -i "tcp://*:5556c"' >> /etc/rc.local
# echo 'exit 0' >> /etc/rc.local
# chmod +x /etc/rc.local

Login to Raspberry Pi 3B and start nProbe as following. As we do not insert license, nProbe is working in demo mode, limited to 2500 flows export (Picture 3).

$ sudo su
# nprobe -i eth0 --zmq "tcp://172.17.100.7:5556" --zmq-probe-mode

Picture 3 - nProbe Demo Mode

Note: To start nProbe after boot, copy & paste the following lines to CLI (without hashtag at the beginning of the lines).

# echo -e '#!/bin/bash\n' > /etc/rc.local
# echo 'nprobe -i eth0 --zmq "tcp://172.17.100.7:5556" --zmq-probe-mode' >> /etc/rc.local
# echo 'exit 0' >> /etc/rc.local
# chmod +x /etc/rc.local

Now, we can login to ntopng web interface listening on 172.17.100.7/16 port 3000. During the first login, you will be prompted to change the password for the user admin.

You can easily find top talkers in our network 172.17.0.0/16 (Picture 4) or to get information about top application protocols used (Picture 5).

Picture 4 - Top Talkers

Picture 5 - Top Application Protocols

The Picture 6 shows active flows (Output is truncated).

Picture 6 - Active Flows

You can even click the IP address to get additional information about the IP address (Picture 7). Notice statistics such as transferred bytes. Captured traffic (pcap file) is also available for download.

Picture 7 -  Details about Host 172.17.100.10

You can also check hosts by countries (Picture 8).

Picture 8 - Hosts by Countries

Picture 9 reveals information about interfaces ntopng is listening to. In our case, it is the interface tcp://*:5556c. You can also find here information about active probes.

Picture 9 - Information about Interfaces ntopng is Listening 

Conclusion
ntopong and nProbe create a very advanced flow-collection architecture, that addressed concerns of using the flow paradigm over the Internet. They give users maximum flexibility now available with traditional probes/collectors.