Zabbix3.4.5新特性:历史数据支持Elasticsearch(网上多篇结合)

注:本篇为网上多篇结合,原因为本人照着网上的博客做了一遍,均为失败,在结合了多篇之后才搞定了这个新功能。

众所周知,zabbix后台使用的数据库为mysql或pgsql,这两个数据库因为性能问题当数据量特别大时,会出现查询迟缓的问题。故zabbix会让你设定一个归档时间,超过这个归档时间的数据就会就行颗粒度放大(我也不知道我在写的什么,懂的自然懂),但是当你真的某天有个需求需要查询过去某一个具体时间的监控信息时,这个放大的颗粒就不会那么精确。鉴于此zabbix对于历史数据引入了用elasticsearch来存储,这就是3.4.5的新特性。

特性功能:

Zabbix自3.4.5rc1版本开始支持Elasticsearch作为历史数据存储,17年12月28日发布了3.4.5

 

部署Elasticsearch(略)

Zabbix3.4.0升级至3.4.5

注: 由于Zabbix3.4.5对libcurl库要求在7.28.0或者更高,不然报如下错误:

cannot initialize history storage: cURL library support >= 7.28.0 is required for Elasticsearch history backend

所以先升级curl

# curl --version
curl 7.29.0 (x86_64-redhat-linux-gnu) libcurl/7.29.0 NSS/3.28.4 zlib/1.2.7 libidn/1.28 libssh2/1.4.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz unix-sockets
#wget http://repo.zabbix.com/zabbix/3.4/rhel/7/x86_64/zabbix-release-3.4-1.el7.centos.noarch.rpm
#wget http://repo.zabbix.com/zabbix/3.4/rhel/7/x86_64/zabbix-release-3.4-2.el7.noarch.rpm
#rpm -ivh zabbix-release-3.4-1.el7.centos.noarch.rpm
#rpm -ivh zabbix-release-3.4-2.el7.noarch.rpm

如果安装过程中有冲突的就把冲突的包yum remove掉,注意备份

#yum install zabbix-server-mysql zabbix-web-mysql -y

老样子,有冲突的就备份后yum remove

Zabbix_server配置支持Elasticsearch

vim /etc/zabbix/zabbix_server.conf
HistoryStorageURL=http://x.x.x.x:9200
HistoryStorageTypes=uint,dbl,str,log,text
里面的其他配置照旧

说明:

Elasticsearch支持的监控项类型:uint,dbl,str,log,text
监控项数据类型|数据库表|对应Elasticsearch类型:

image

Zabbix Web配置历史数据读Elasticsearch

修改配置文件vim conf/zabbix.conf.php
1. 开头的global $db改为global $DB, $HISTORY;
2. 如果不同类型使用不同的ES集群,可以按如下进行配置
$HISTORY['url']   = [
    'uint' => 'http://localhost1:9200',
    'text' => 'http://localhost2:9200'
];
$HISTORY['types'] = ['uint', 'text'];
3. 所有类型使用相同ES集群
$HISTORY['url']   = 'http://x.x.x.x:9200';
$HISTORY['types'] = ['str', 'text', 'log', 'uint', 'dbl'];

設定elasticsearch上zabbix資料的型態

# cd /opt/software/zabbix-3.4.6/database/elasticsearch/
# ll
total 4
-rw-r--r-- 1 1001 1001 3142 Jan 15 17:11 elasticsearch.map
# more elasticsearch.map
# uint mapping
 
{
   "settings" : {
      "index" : {
         "number_of_replicas" : 1,
         "number_of_shards" : 5
      }
   },
   "mappings" : {
      "values" : {
         "properties" : {
            "itemid" : {
               "type" : "long"
            },
            "clock" : {
               "format" : "epoch_second",
               "type" : "date"
            },
            "value" : {
               "type" : "long"
            }
         }
      }
   }
}
 
 
# dbl mapping
 
{
   "settings" : {
      "index" : {
         "number_of_replicas" : 1,
         "number_of_shards" : 5
      }
   },
   "mappings" : {
      "values" : {
         "properties" : {
            "itemid" : {
               "type" : "long"
            },
            "clock" : {
               "format" : "epoch_second",
               "type" : "date"
            },
            "value" : {
               "type" : "double"
            }
         }
      }
   }
}
 
 
# str mapping
 
{
   "settings" : {
      "index" : {
         "number_of_replicas" : 1,
         "number_of_shards" : 5
      }
   },
   "mappings" : {
      "values" : {
         "properties" : {
            "itemid" : {
               "type" : "long"
            },
            "clock" : {
               "format" : "epoch_second",
               "type" : "date"
            },
            "value" : {
               "fields" : {
                  "analyzed" : {
                     "index" : true,
                     "type" : "text",
                     "analyzer" : "standard"
                  }
               },
               "index" : false,
               "type" : "text"
            }
         }
      }
   }
}
 
 
# text mapping
 
{
   "settings" : {
      "index" : {
         "number_of_replicas" : 1,
         "number_of_shards" : 5
      }
   },
   "mappings" : {
      "values" : {
         "properties" : {
            "itemid" : {
               "type" : "long"
            },
            "clock" : {
               "format" : "epoch_second",
               "type" : "date"
            },
            "value" : {
               "fields" : {
                  "analyzed" : {
                     "index" : true,
                     "type" : "text",
                     "analyzer" : "standard"
                  }
               },
               "index" : false,
               "type" : "text"
            }
         }
      }
   }
}
 
 
# log mapping
 
{
   "settings" : {
      "index" : {
         "number_of_replicas" : 1,
         "number_of_shards" : 5
      }
   },
   "mappings" : {
      "values" : {
         "properties" : {
            "itemid" : {
               "type" : "long"
            },
            "clock" : {
               "format" : "epoch_second",
               "type" : "date"
            },
            "value" : {
               "fields" : {
                  "analyzed" : {
                     "index" : true,
                     "type" : "text",
                     "analyzer" : "standard"
                  }
               },
               "index" : false,
               "type" : "text"
            }
         }
      }
   }
}
# curl -XPUT http://x.x.x.x:9200/uint -d '
{
    "settings" : {
       "index" : {
          "number_of_replicas" : 1,
          "number_of_shards" : 5
       }
    },
    "mappings" : {
       "values" : {
          "properties" : {
             "itemid" : {
                "type" : "long"
             },
             "clock" : {
                "format" : "epoch_second",
                "type" : "date"
             },
             "value" : {
                "type" : "long"
             }
           }
        }
     }
}
'
{"acknowledged":true,"shards_acknowledged":true}
# curl -XPUT http://x.x.x.x:9200/dbl -d '
{
   "settings" : {
      "index" : {
         "number_of_replicas" : 1,
         "number_of_shards" : 5
      }
   },
   "mappings" : {
      "values" : {
         "properties" : {
            "itemid" : {
               "type" : "long"
            },
            "clock" : {
               "format" : "epoch_second",
               "type" : "date"
            },
            "value" : {
               "type" : "double"
            }
         }
      }
   }
}
'
{"acknowledged":true,"shards_acknowledged":true}
# curl -XPUT http://x.x.x.x:9200/log -d '
 {
    "settings" : {
       "index" : {
          "number_of_replicas" : 1,
          "number_of_shards" : 5
       }
    },
    "mappings" : {
       "values" : {
          "properties" : {
             "itemid" : {
                "type" : "long"
             },
             "clock" : {
                "format" : "epoch_second",
                "type" : "date"
             },
             "value" : {
                "fields" : {
                   "analyzed" : {
                      "index" : true,
                      "type" : "text",
                      "analyzer" : "standard"
                   }
                },
                "index" : false,
                "type" : "text"
             }
          }
       }
    }
 }
 '
{"acknowledged":true,"shards_acknowledged":true}
# curl -XPUT http://x.x.x.x:9200/text -d '
 {
    "settings" : {
       "index" : {
          "number_of_replicas" : 1,
          "number_of_shards" : 5
       }
    },
    "mappings" : {
       "values" : {
          "properties" : {
             "itemid" : {
                "type" : "long"
             },
             "clock" : {
                "format" : "epoch_second",
                "type" : "date"
             },
             "value" : {
                "fields" : {
                   "analyzed" : {
                      "index" : true,
                      "type" : "text",
                      "analyzer" : "standard"
                   }
                },
                "index" : false,
                "type" : "text"
             }
          }
       }
    }
 }
 '
{"acknowledged":true,"shards_acknowledged":true}
# curl -XPUT http://x.x.x.x:9200/str -d '
 {
    "settings" : {
       "index" : {
          "number_of_replicas" : 1,
          "number_of_shards" : 5
       }
    },
    "mappings" : {
       "values" : {
          "properties" : {
             "itemid" : {
                "type" : "long"
             },
             "clock" : {
                "format" : "epoch_second",
                "type" : "date"
             },
             "value" : {
                "fields" : {
                   "analyzed" : {
                      "index" : true,
                      "type" : "text",
                      "analyzer" : "standard"
                   }
                },
                "index" : false,
                "type" : "text"
             }
          }
       }
    }
 }
 '
{"acknowledged":true,"shards_acknowledged":true}
# /etc/init.d/zabbix_server start
Starting zabbix_server:                                    [  OK  ]
# /etc/init.d/httpd restart

(錯了要把資料全清掉才能重做)

删除方法:

curl -XDELETE http://x.x.x.x:9200/app_name

测试

  1. Zabbix配置ES成功后,通过Kibana可以看到:image
  2. 创建索引
  3. image
  4. image
  5. image
  6. 通过Zabbix Web访问是否正常显示数据

发表评论

电子邮件地址不会被公开。 必填项已用*标注

This blog is kept spam free by WP-SpamFree.