From Initech Technical Wiki
Jump to: navigation, search

You'll find my collection of scripts and things related to Clickhouse here.


This is a collection of configurations and scripts which effectively enable netflows/IPFIX records to be collected and imported into a clickhouse database.

Importing netflows

  1. pmacct uses nfacct.conf to collect netflows/IPFIX records and output them to /tmp/nfacct.json in JSONEachRow format and then triggers the nfacct-clickhouse-import script.
  2. The nfacct-clickhouse-import script pre-filters the /tmp/nfacct.json file using grep and PCRE patterns stored in nfacct_drop_patterns and imports what is left into clickhouse into the netflow.nfacct table.


I use grafana to visualise the data using this dashboard, you'll need the piechart plugin, never mind about friends don't let friends abuse pie-charts

XC5 Softswitch

This is a collection of scripts which export CDR records from an XC5 softswitch, import them into clickhouse and visualise them using a grafana dashboard.

Importing CDRs

  1. This step is triggered by cron (i run mine every minute) and calls a php script called clickhouse-cdr-import.php which logs into the XC5 JSON api, exports the CDRs and imports them into clickhouse into the softswitch.cdr table


Again I use grafana to visualise the data using this dashboard, and again you'll need the piechart plugin.