Zabbix 4.2 data processing and more Sponsors Gold sponsors Co-organizer 2 What is Zabbix? Zabbix is a universal Open Source enterprise level monitoring solution 3 4 We have found a good balance between giving away free software and having a sustainable growth 5 Free Software Services Services that save your time and money Free Software Services Services that save your time and money Customers in 75+ countries 8 Some of our users 9 Zabbix Real-time Alerting & Data collection problem Visualization Remediation detection 10 Where we are currently? 3.0 LTS 3.2 3.4 4.0 LTS 11 Zabbix 4.0 LTS 12 Data preprocessing { “host": { "name": "Linux #2356", "vms": [{ “name”: “vm034”, “network": { "read": “0xfa673”, "write": “0x45b30” } }, { “name”: “vm076”, “network": { "read": “0x76ab”, “write": “0xff3a” } }] } } 13 Data preprocessing "write": “0x45b30” } }, { “name”: “vm076”, “network": { "read": “0x76ab”, 0x76ab “write": “0xff3a” } }] JSON Path: .host.vms[1].network.read } } 14 Data preprocessing 0x76ab JSON Path: .host.vms[1].network.read 15 Data preprocessing 0x76ab 76ab Regexp: 0x(.*) 16 Data preprocessing 0x76ab 76ab 330379 Hex to Decimal 17 Data preprocessing 0x76ab 76ab 330379 338308096 KB -> bytes 18 Data preprocessing 19 Efficiency Zabbix server mysql[status] 20 Efficiency Master item 21 Efficiency Dependent items 22 Efficiency mysql[questions] mysql[status] Zabbix server mysql[reads] 23 Efficiency mysql[questions] mysql[status] Zabbix server mysql[reads] Performance, less user parameters, all logic in templates 24 Limitations 4.0 LTS 4.2 Not flexible enough for all use cases Relies on user parameters Not anymore! No support of preprocessing for LLD 25 Zabbix 4.2 ETA: March, 2019 26 What is the ultimate goal? 4.2 4.4 5.0 LTS No limits for data processing Versioning for templates Self-contained templates Host and template tags Official guidelines Official templates 27 Validation, discarding 1 values and setting errors 28 Validation In range Matches regular expression Does not match regular expression Check for error in JSON Check for error in XML Check for error using regular expression 29 On fail Preprocessing Discard value Set value to Set error to 30 Discard incorrect values N/A N/A (dropped!) 34 C 34 C 33 C 33 C 32 C 32 C 30 C 30 C 123 F Zabbix 123 F(dropped!) 31 Discard out of range values N/A N/A (dropped!) 34 34 33 33 999 999 (dropped!) 32 32 30 Zabbix 30 32 Use default if does not match OK OK ERR2 NOT OK(default!) OK OK OK OK ERR4 NOT OK(default!) NOT OK Zabbix NOT OK 33 Set error message if not in range 34 34 33 33 32 32 30 30 999 NOT SUPPORTED Zabbix Error:sensor hardware failure, out of range temperature. 34 Extract error message { “status”: “on”} on { “status”: “off”} off { “status”: “off”} off { “err”: “Node no NOT SUPPORTED longer exists”} Zabbix Error: Node no longer exists 35 2 Throttling 36 Process on value change 0 0 0 (dropped) 1 1 1 (dropped) 1 (dropped) 0 Zabbix 0 0 (dropped) 37 Subsampling 0s 0 0s 0 1s 1 1s 1 2s 0 2s 0 (dropped) 3s 0 (dropped) 4s 0 (dropped) 5s 0 6s 0 Zabbix (dropped) 7s 0 7s 0 (dropped) 8s 0 (dropped) 9s 0 38 Who is doing throttling 0 0 0 0 0 0 0 (dropped) 1 1 1 1 1 1 1 (dropped) 1 1 1 (dropped) 0 Agent 0 Proxy 0 0 0 0 0 Server (dropped) 39 Prometheus: 3 JS is not enough 40 Prometheus # HELP cpu_usage_system Telegraf collected metric # TYPE cpu_usage_system gauge cpu_usage_system{cpu="cpu-total",host="host1"} 1.1940298507220641 cpu_usage_system{cpu="cpu0",host="host1"} 1.1940298507220641 cpu_usage_system{cpu="cpu1",host="host1"} 1.1340298507220641 41 Prometheus Prometheus PromQL cpu_usage_system{cpu=“cpu-total"} # HELP cpu_usage_system Telegraf collected metric # TYPE cpu_usage_system gauge cpu_usage_system{cpu="cpu-total",host="host1"} 1.1940298507220641 1.1940298507220641 cpu_usage_system{cpu="cpu0",host="host1"} 1.1940298507220641 cpu_usage_system{cpu="cpu1",host="host1"} 1.1340298507220641 42 Discovery Prometheus PromQL wmi_logical_disk_free_bytes{volume =~”[A-C]:”} [ # HELP wmi_logical_disk_free_bytes Free space in bytes { (LogicalDisk.PercentFreeSpace) "name": "wmi_logical_disk_free_bytes", # TYPE wmi_logical_disk_free_bytes gauge "help": "Free space in bytes wmi_logical_disk_free_bytes{volume="C:"} 3.5180249088e+11 (LogicalDisk.PercentFreeSpace)", wmi_logical_disk_free_bytes{volume="D:"} 2.627731456e+09 "type": "gauge", wmi_logical_disk_free_bytes{volume="HarddiskVolume4"} 4.598e+08 "labels": { "volume": "C:" }, "value": "3.5180249088e+11", "line_raw": "wmi_logical_disk_free_bytes{volume=\"C:\"} 3.5180249088e+11" } ] 43 HTTP endpoint monitoring TEXT HTML JSON HTTP check XML Pre-processing History Prometheus Multiple metrics HTTP data processing 44 4 Ultimate flexibility 45 Challenges Converting C to F return (input - 32 ) * 5 / 9; 46 Challenges return (input.match(new RegExp( Word count param, 'g')) || []).length; 47 Challenges var data = JSON.parse(input); var count = 0; function iterate(obj) { for (var key in obj) { if (typeof(obj[key]) == 'object') { iterate(obj[key]); continue; JSON filtering } if (key == 'id' && obj[key] == param) count++; } } iterate(data); return count; 48 Ultimate test 49 50 No user parameters, no shell scripts, thus easy maintenance Super efficient, less impact on monitored devices due to bulk requests All complex logic is part of the template Templates becomes self-contained Services 5 TimescaleDB 52 Storage options MySQL PostgreSQL Oracle History DB2 ElasticSearch Zabbix 53 Getting performance TimescaleDB MySQL PostgreSQL Oracle DB2 Partitioning 54 And much more! 55 Zabbix 4.2 roadmap Zabbix 4.2 56 Zabbix capabilities 57 58 More than 300 integrations 59 60 61 62 It is a moderated list of all known solutions from community, vendors and partners! 63 https://share.zabbix.com https://www.zabbix.com/integrations 64 Thank you! Some of the used icons made by Freepik from www.flaticon.com.