nfdump suite¶
Introduction¶
This page has information about the nfdump suite. It is intended to be publicly useful
The name "nfdump suite" is potentially misleading because nfdump is one of six Netflow utilities in the suite: nfcapd, nfdump, nfexpire, nfprofile, nfreplay and nftrack. nfdump translates the binary netlow record files created by nfcapd (Netflow capture daemon) into human readable form
The nfdump suite was closely related to the now defunct nfsen. Nfsen was a web GUI which used nfdump to generate data for interactive display
Related documentation¶
- https://github.com/phaag/nfdump
- man pages
nfdump setup¶
This is actually about setting up an nfcapd service but the default names include nfdump
semop() error in bookkeeper.c line ...: Invalid argument"¶
These error messages were fixed by adding ...
RemoveIPC=no... to /etc/systemd/logind.conf and effecting it by
systemctl daemon-reload systemctl restart systemd-logind.service
Setup itself¶
Example for "edge1" using directory sub hierarchy scheme number 1. Directory sub hierarchy schemes are listed in the nfcapd man page
The network device had been set up to send netflow packets to the nfdump suite server. That was specific to the type of device so is not shown here
Setup took a long time because no complete guide was found. Hence this page
The following procedure was assembled from notes so may not be exactly right but it should be a good starting point- Created user nfcapd
- Created directory /var/cache/nfdump/edge1 nfcapd:nfcapd rwxr-xr-x
- Set up netflow record expiry. As user nfcapd
nfexpire -t $((365*3))d -u /var/cache/nfdump/edge1
- Created /etc/nfdump/edge1.conf by copying /etc/nfdump and editing
root@storage3.iciti.av:~# diff /etc/nfdump/default.conf /etc/nfdump/edge1.conf ... < #cache_directory=/var/cache/nfdump < #user=root < #group=root --- > cache_directory=/var/cache/nfdump/edge1 > user=nfcapd > group=nfcapd 28c56 < options='-l /var/cache/nfdump -p 2055' --- > options='-e -g nfcapd -l /var/cache/nfdump/edge1 -p 2055 -S1 -u nfcapd'
- Fixed a /run permissions difficulty by creating /etc/systemd/system/nfdump@.service
diff /lib/systemd/system/nfdump@.service /etc/systemd/system/nfdump@.service 10,11c10,11 < ExecStart=/usr/bin/nfcapd -D -P /run/nfcapd.%I.pid $options < PIDFile=/run/nfcapd.%I.pid --- > ExecStart=/usr/bin/nfcapd -D -P /tmp/nfcapd.%I.pid $options > PIDFile=/tmp/nfcapd.%I.pid
- Created a templated service
ln -s /etc/systemd/system/nfdump@.service /etc/systemd/system/nfdump@edge1.service systemctl daemon-reload systemctl start nfdump@edge1.service systemctl enable nfdump@edge1.service
Logcheck¶
cat /etc/logcheck/ignore.d.server/local-nfdump
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ nfcapd\[[[:digit:]]+\]: Add extension:
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ nfcapd\[[[:digit:]]+\]: Bound to IPv
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ nfcapd\[[[:digit:]]+\]: Current size: .*, Current lifetime: .*, Number of files: [[:digit:]]+$
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ nfcapd\[[[:digit:]]+\]: Ident: .* Flows: [[:digit:]]+, Packets: [[:digit:]]+, Bytes: [[:digit:]]+, Sequence Errors: 0, Bad Packets: 0$
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ nfcapd\[[[:digit:]]+\]: Init IPFIX: Max number of IPFIX tags: [[:digit:]]+$
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ nfcapd\[[[:digit:]]+\]: Limits: Filesize .*, Lifetime .*, Watermark: [[:digit:]]+%$
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ nfcapd\[[[:digit:]]+\]: Process_v9: New exporter: SysID: [[:digit:]]+, Domain: [[:digit:]]+, IP:
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ nfcapd\[[[:digit:]]+\]: Process_v9: \[[[:digit:]]+\] Add template [[:digit:]]+$
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ nfcapd\[[[:digit:]]+\]: Run expire on
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ nfcapd\[[[:digit:]]+\]: Signal launcher$
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ nfcapd\[[[:digit:]]+\]: Startup\.$
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ nfcapd\[[[:digit:]]+\]: Terminating nfcapd\.$
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ nfcapd\[[[:digit:]]+\]: Total ignored packets: 0$
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ nfcapd\[[[:digit:]]+\]: expire completed - nothing to expire\.$