Clickhouse
From Initech Technical Wiki
You'll find my collection of scripts and things related to Clickhouse here.
pmacct
This is a collection of configurations and scripts which effectively enable netflows/IPFIX records to be collected and imported into a clickhouse database.
Importing netflows
- pmacct uses nfacct.conf to collect netflows/IPFIX records and output them to /tmp/nfacct.json in JSONEachRow format and then triggers the nfacct-clickhouse-import script.
- The nfacct-clickhouse-import script pre-filters the /tmp/nfacct.json file using grep and PCRE patterns stored in nfacct_drop_patterns and imports what is left into clickhouse into the netflow.nfacct table.
Visualisation
I use grafana to visualise the data using this dashboard, you'll need the | piechart plugin, never mind about the | friends don't let friends abuse pie-charts