Difference between revisions of "Clickhouse"

Revision as of 21:54, 18 October 2019

You'll find my collection of scripts and things related to Clickhouse here.

pmacct

This is a collection of configurations and scripts which effectively enable netflows/IPFIX records to be collected and imported into a clickhouse database.

Importing netflows

pmacct uses nfacct.conf to collect netflows/IPFIX records and output them to /tmp/nfacct.json in JSONEachRow format and then triggers the nfacct-clickhouse-import script.
The nfacct-clickhouse-import script pre-filters the /tmp/nfacct.json file using grep and PCRE patterns stored in nfacct_drop_patterns and imports what is left into clickhouse into the netflow.nfacct table.

Visualisation

I use grafana to visualise the data using this dashboard, you'll need the piechart plugin, never mind about friends don't let friends abuse pie-charts

Revision as of 21:53, 18 October 2019 (view source) Timprice (talk \| contribs) ← Older edit		Revision as of 21:54, 18 October 2019 (view source) Timprice (talk \| contribs) Newer edit →
Line 9:		Line 9:

	=== Visualisation ===		=== Visualisation ===
−	I use grafana to visualise the data using this [[clickhouse-netflow-dashboard\|dashboard]], you'll need the [https://grafana.com/grafana/plugins/grafana-piechart-panel \| piechart] plugin, never mind about ~~the~~ [https://grafana.com/blog/2015/12/04/friends-dont-let-friends-abuse-pie-charts/ \| friends don't let friends abuse pie-charts]	+	I use grafana to visualise the data using this [[clickhouse-netflow-dashboard\|dashboard]], you'll need the [https://grafana.com/grafana/plugins/grafana-piechart-panel\| piechart] plugin, never mind about [https://grafana.com/blog/2015/12/04/friends-dont-let-friends-abuse-pie-charts/\| friends don't let friends abuse pie-charts]