前言
大数据分为实时处理和离线处理,通常我们会采用`SparkStreaming`程序直连`Kafka`把数据写入到`Hbase`中,热更新`Hbase`,但作为列式存储的`Hbase`存在严重依赖rowkeys弊端,
在设计表时需要考虑使用rowkeys提高查询速度,phoenix采用空间换时间方式可以架设在`Hbase`上,但`Phoenix`构建索引也需要消耗大量的时间,最终我们调研采用`Apache Geode`.
`Apache Geode`为商业版`Gemfire`开源版,`Gemfire`在12306支持所有的火车票的订单的缓存,可以抗住亿万并发写入和并发读取.
`Geode`架构分为Locator和Server,Locator负责路由请求,Server服务写入数据.经过一段时间调研,决定采用`Geode`+`SparkStreaming`方式实时写入更新350W新能源车的数据.用于
大屏和客户接口实时查询.
安装Geode
下载Geode
- wget http://mirror.bit.edu.cn/apache/geode/1.11.0/apache-geode-1.11.0.tgz
- tar -zxvf apache-geode-1.11.0.tgz
- mv apache-geode-1.11.0 /opt/geode
配置Geode
修改配置文件gemfire.properties
#Thu Sep 27 17:26:36 PDT 2018
ack-severe-alert-threshold=0
ack-wait-threshold=50
archive-disk-space-limit=0
archive-file-size-limit=0
async-distribution-timeout=0
async-max-queue-size=100 #异步队列大小
async-queue-timeout=60000 #超时时间
bind-address=
cache-xml-file=cache.xml
cluster-configuration-dir=
cluster-ssl-ciphers=any
cluster-ssl-enabled=false
cluster-ssl-keystore=
cluster-ssl-keystore-password=
cluster-ssl-keystore-type=
cluster-ssl-protocols=any
cluster-ssl-require-authentication=true
cluster-ssl-truststore=
cluster-ssl-truststore-password=
conflate-events=server
conserve-sockets=true
delta-propagation=true
deploy-working-dir=/opt/geode/
disable-auto-reconnect=false
disable-tcp=false
distributed-system-id=-1
distributed-transactions=false
durable-client-id=
durable-client-timeout=3000
enable-cluster-configuration=true
enable-network-partition-detection=true
enable-time-statistics=false
enforce-unique-host=false
gateway-ssl-ciphers=any
gateway-ssl-enabled=false
gateway-ssl-keystore=
gateway-ssl-keystore-password=
gateway-ssl-keystore-type=
gateway-ssl-protocols=any
gateway-ssl-require-authentication=true
gateway-ssl-truststore=
gateway-ssl-truststore-password=
groups=
http-service-bind-address=
http-service-port=7070
http-service-ssl-ciphers=any
http-service-ssl-enabled=false
http-service-ssl-keystore=
http-service-ssl-keystore-password=
http-service-ssl-keystore-type=
http-service-ssl-protocols=any
http-service-ssl-require-authentication=false
http-service-ssl-truststore=
http-service-ssl-truststore-password=
jmx-manager=false
jmx-manager-access-file=
jmx-manager-bind-address=
jmx-manager-hostname-for-clients=
jmx-manager-http-port=7070
jmx-manager-password-file=
jmx-manager-port=1099
jmx-manager-ssl-ciphers=any
jmx-manager-ssl-enabled=false
jmx-manager-ssl-keystore=
jmx-manager-ssl-keystore-password=
jmx-manager-ssl-keystore-type=
jmx-manager-ssl-protocols=any
jmx-manager-ssl-require-authentication=true
jmx-manager-ssl-truststore=
jmx-manager-ssl-truststore-password=
jmx-manager-start=false
jmx-manager-update-rate=2000
load-cluster-configuration-from-dir=false
locator-wait-time=0
locators=
lock-memory=false
log-disk-space-limit=0
log-file=
log-file-size-limit=0
log-level=config
max-num-reconnect-tries=3
max-wait-time-reconnect=60000
mcast-address=239.192.81.1
mcast-flow-control=1048576, 0.25, 5000
mcast-port=0
mcast-recv-buffer-size=10485760 #广播缓存量
mcast-send-buffer-size=655350
mcast-ttl=32
member-timeout=50000
membership-port-range=1024-65535
memcached-bind-address=
memcached-port=0
memcached-protocol=ASCII
name=
off-heap-memory-size=5g
redis-bind-address=
redis-password=
redis-port=0
redundancy-zone=
remote-locators=
remove-unresponsive-client=false
roles=
security-client-accessor=
security-client-accessor-pp=
security-client-auth-init=
security-client-authenticator=
security-client-dhalgo=
security-log-file=
security-log-level=config
security-manager=
security-peer-auth-init=
security-peer-authenticator=
security-peer-verifymember-timeout=1000
security-post-processor=
security-shiro-init=
security-udp-dhalgo=
serializable-object-filter=\!*
server-bind-address=
server-ssl-ciphers=any
server-ssl-enabled=false
server-ssl-keystore=
server-ssl-keystore-password=
server-ssl-keystore-type=
server-ssl-protocols=any
server-ssl-require-authentication=true
server-ssl-truststore=
server-ssl-truststore-password=
socket-buffer-size=327680 #socket缓存
socket-lease-time=60000
ssl-ciphers=any
ssl-cluster-alias=
ssl-default-alias=
ssl-enabled-components=
ssl-endpoint-identification-enabled=false
ssl-gateway-alias=
ssl-jmx-alias=
ssl-keystore=
ssl-keystore-password=
ssl-keystore-type=
ssl-locator-alias=
ssl-protocols=any
ssl-require-authentication=true
ssl-server-alias=
ssl-truststore=
ssl-truststore-password=
ssl-truststore-type=
ssl-web-alias=
ssl-web-require-authentication=false
start-dev-rest-api=false
start-locator=
statistic-archive-file=
statistic-sample-rate=1000
statistic-sampling-enabled=true
tcp-port=0
thread-monitor-enabled=true
thread-monitor-interval-ms=60000
thread-monitor-time-limit-ms=30000
udp-fragment-size=60000
udp-recv-buffer-size=10485760
udp-send-buffer-size=655350
use-cluster-configuration=true
user-command-packages=
validate-serializable-objects=false
启动Geode
启动locator1
登陆192.168.1.120 启动locator1
- gfsh
- start locator –name=locator1 –port=10334 –locators=xxxx2[10334],xxxx1[10334]
启动locator2
登陆192.168.1.121 启动locator2
- gfsh
- start locator –name=locator2 –port=10334 –locators=xxxx2[10334],xxxx1[10334]
启动Server
登陆192.168.1.121 启动Server可以启动多个
- start server –name=server1 –initial-heap=8G –max-heap=8G –J=-XX:NewRatio=2 –J=-Xss512k –lock-memory=true –off-heap-memory-size=10G –server-port=40439 –locators=xxxx1[10334],xxxx2[10334] –properties-file=/opt/geode/config/gemfire.properties
--initial-heap初始化堆内存大小--max-heap=8G最大堆内存大小--J=-XX:NewRatio年轻代的Eden,Survivor大小比例--J=-Xss512k最大栈内存大小--lock-memory=true是否锁定内存--off-heap-memory-size堆外内存大小
启动pluster监控
start pulse –url=”http://10.10.21.9:7070/pulse”
Zabbix监控Geode
cluster-geode-plugin.sh脚本内容
#!/usr/bin/sh
#--------------------------------------------------------------------------------------------
# Expecting the following arguments in order -
# <host> = hostname/ip-address of Geode Cluster
# This is made available as a macro in host configuration.
# <port> = Port # Geode are available (default = 7070)
# This is made available as a macro in host configuration.
# <name_in_zabbix> = Name by which the Geode is configured in Zabbix.
# This is made available as a macro in host configuration.
#--------------------------------------------------------------------------------------------
COMMAND_LINE="$0 $*"
export SCRIPT_NAME="$0"
usage() {
echo "Usage: $SCRIPT_NAME <host> <port> <name_in_zabbix>"
}
if [ $# -ne 3 ]
then
usage ;
exit ;
fi
#--------------------------------------------------------------------------------------------
# First 2 parameters are required for connecting to Geode Locator
# The 3th parameter NAME_IN_ZABBIX is required to be sent back to Zabbix to identify the
# Zabbix host/entity for which these metrics are destined.
#--------------------------------------------------------------------------------------------
export GEODE_HOST=$1
export METRICS_PORT=$2
export NAME_IN_ZABBIX=$3
#--------------------------------------------------------------------------------------------
# Set the data output file and the log file from zabbix_sender
#--------------------------------------------------------------------------------------------
export DATA_FILE="/tmp/${NAME_IN_ZABBIX}_Geode_HEATH.txt"
export BAK_DATA_FILE="/tmp/${NAME_IN_ZABBIX}_Geode_HEATH_bak.txt"
export LOG_FILE="/tmp/${NAME_IN_ZABBIX}_Geode_HEATH.log"
#--------------------------------------------------------------------------------------------
# Use python to get the metrics data from Geode and use screen-scraping to extract
# metrics.
# The final result of screen scraping is a file containing data in the following format -
# <NAME_IN_ZABBIX> <METRIC_NAME> <METRIC_VALUE>
#--------------------------------------------------------------------------------------------
python `dirname $0`/GeodeMonitor.py $GEODE_HOST $METRICS_PORT $DATA_FILE $NAME_IN_ZABBIX
#--------------------------------------------------------------------------------------------
# Check the size of $DATA_FILE. If it is not empty, use zabbix_sender to send data to Zabbix.
#--------------------------------------------------------------------------------------------
if [[ -s $DATA_FILE ]]
then
zabbix_sender -vv -z 127.0.0.1 -i $DATA_FILE 2>>$LOG_FILE 1>>$LOG_FILE
echo -e "Successfully executed $COMMAND_LINE" >>$LOG_FILE
mv $DATA_FILE $BAK_DATA_FILE
echo "OK"
else
echo "Error in executing $COMMAND_LINE" >> $LOG_FILE
echo "ERROR"
fi
GeodeMonitor.py内容
__author__ = 'BQ'
import json
import urllib
import sys
import requests
def get_url_login_process_json(server_name, listen_port):
"""
login geode
:param server_name:
:param listen_port:
:return:
"""
if listen_port < 0:
print ("Invalid Port")
exit()
if not server_name:
print("Pass valid Hostname")
exit()
URL_LOGIN = "http://"+server_name+":"+str(listen_port)+"/pulse/login"
textmod ={'username':'admin','password':'admin'}
headers = {'User-agent':'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36'}
sessions = requests.session()
sessions_post = sessions.post(URL_LOGIN, data=textmod,json=True,headers=headers)
URL = "http://"+server_name+":"+str(listen_port)+"/pulse/pulseUpdate"
textmod1 ={'pulseData':'{"SystemAlerts":{"pageNumber":"1"},"ClusterDetails":{},"ClusterMembers":{},"ClusterMembersRGraph":{},"ClusterRegions":{},"ClusterKeyStatistics":{},"ClusterJVMPauses":{},"ClusterWANInfo":{},"ClusterMemoryUsage":{},"ClusterDiskThroughput":{},"PulseVersion":{}}'}
post = sessions.post(URL, data=textmod1,json=True,headers=headers)
content = json.loads(post.content)
geode_dict = {}
geode_dict["totalMembers"] = content['ClusterDetails']['totalMembers']
geode_dict["locators"] = content['ClusterDetails']['locators']
geode_dict["infoAlertCount"] = content['ClusterDetails']['infoAlertCount']
geode_dict["severeAlertCount"] = content['ClusterDetails']['severeAlertCount']
geode_dict["warningAlertCount"] = content['ClusterDetails']['warningAlertCount']
geode_dict["servers"] = content['ClusterDetails']['servers']
geode_dict["errorAlertCount"] = content['ClusterDetails']['errorAlertCount']
geode_dict["totalRegions"] = content['ClusterDetails']['totalRegions']
geode_dict["clients"] = content['ClusterDetails']['clients']
geode_dict["totalHeap"] = content['ClusterDetails']['totalHeap']
geode_dict["currentMemoryUsage"] = content['ClusterMemoryUsage']['currentMemoryUsage']
geode_dict["systemRegionEntryCount"] = content['ClusterRegions']['regions'][0]['systemRegionEntryCount']
geode_dict["putsRate"] = content['ClusterRegions']['regions'][0]['putsRate']
geode_dict["getsRate"] = content['ClusterRegions']['regions'][0]['getsRate']
geode_dict["entryCount"] = content['ClusterRegions']['regions'][0]['entryCount']
geode_dict["currentGCPause"] = content['ClusterJVMPauses']['currentGCPauses']
return geode_dict
# ---------------------------------
# Generate URL
# ---------------------------------
# This function converts the servername to URL which we need to query.
def get_url(server_name, listen_port):
"""
Generating URL to get the information
:param server_name:
:param listen_port:
:return:
"""
if listen_port < 0:
print ("Invalid Port")
exit()
if not server_name:
print("Pass valid Hostname")
exit()
URL = "http://"+server_name+":"+str(listen_port)+"/pulse/pulseUpdate"
URL = "http://"+server_name+":"+str(listen_port)+"/pulse/login"
textmod ={'pulseData':'{"SystemAlerts":{"pageNumber":"1"},"ClusterDetails":{},"ClusterMembers":{},"ClusterMembersRGraph":{},"ClusterRegions":{},"ClusterKeyStatistics":{},"ClusterJVMPauses":{},"ClusterWANInfo":{},"ClusterMemoryUsage":{},"ClusterDiskThroughput":{},"PulseVersion":{}}'}
textmod = urllib.urlencode(textmod)
data = requests.post(URL, textmod)
load = json.load(data.content)
members_ = load["ClusterDetails"]["totalMembers"]
print(members_)
return data.content
# ---------------------------------
# Load URL
# ---------------------------------
def load_url_as_dictionary(url):
"""
Loading JSON URL which we recieved
:param url:
:return:
"""
# Server URL to get JSON information
return json.load(urllib.urlopen(url))
def write_data_to_file(json, file_path, name_in_zabbix):
txt_file = open(file_path, 'w+')
for keys in json:
txt_file.writelines(name_in_zabbix +' '+ str(keys) +' '+ str(json[keys]) + '\n')
def usage():
print '''
Usage: $SCRIPT_NAME <YarnRM_host> <YarnRM_port> <file_path> <name_in_zabbix>
'''
if __name__ == '__main__':
if len(sys.argv) == 5:
geode_hostname = sys.argv[1]
geode_listen_port = sys.argv[2]
file_path = sys.argv[3]
nodename_in_zabbix = sys.argv[4]
process_json = get_url_login_process_json(geode_hostname, geode_listen_port)
write_data_to_file(process_json,file_path,nodename_in_zabbix)
else:
usage()
zabbix模板xml文件内容
<?xml version="1.0" encoding="UTF-8"?>
<zabbix_export>
<version>3.4</version>
<date>2020-03-17T09:23:50Z</date>
<groups>
<group>
<name>Geode</name>
</group>
</groups>
<templates>
<template>
<template>Template Cluster Geode</template>
<name>Template Cluster Geode</name>
<description/>
<groups>
<group>
<name>Geode</name>
</group>
</groups>
<applications>
<application>
<name>Geode Cluster</name>
</application>
</applications>
<items>
<item>
<name>连接到geode的客户端数量</name>
<type>2</type>
<snmp_community/>
<snmp_oid/>
<key>clients</key>
<delay>0</delay>
<history>10d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Geode Cluster</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>Geode Cluster数据采集器</name>
<type>10</type>
<snmp_community/>
<snmp_oid/>
<key>cluster-geode-plugin.sh[{$GEODE_HOST},{$GEODE_METRICS_PORT},{$ZABBIX_NAME}]</key>
<delay>30</delay>
<history>10d</history>
<trends>0</trends>
<status>0</status>
<value_type>4</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Geode Cluster</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>GeodeJVM停顿</name>
<type>2</type>
<snmp_community/>
<snmp_oid/>
<key>currentGCPause</key>
<delay>0</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Geode Cluster</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>当前内存使用量</name>
<type>2</type>
<snmp_community/>
<snmp_oid/>
<key>currentMemoryUsage</key>
<delay>0</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units>MB</units>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Geode Cluster</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>实体总数</name>
<type>2</type>
<snmp_community/>
<snmp_oid/>
<key>entryCount</key>
<delay>0</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Geode Cluster</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>错误报警数</name>
<type>2</type>
<snmp_community/>
<snmp_oid/>
<key>errorAlertCount</key>
<delay>0</delay>
<history>10d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Geode Cluster</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>get速率</name>
<type>2</type>
<snmp_community/>
<snmp_oid/>
<key>getsRate</key>
<delay>0</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Geode Cluster</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>region更新数据量</name>
<type>2</type>
<snmp_community/>
<snmp_oid/>
<key>infoAlertCount</key>
<delay>0</delay>
<history>10d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Geode Cluster</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>locators数目</name>
<type>2</type>
<snmp_community/>
<snmp_oid/>
<key>locators</key>
<delay>0</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Geode Cluster</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>put速率</name>
<type>2</type>
<snmp_community/>
<snmp_oid/>
<key>putsRate</key>
<delay>0</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Geode Cluster</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>servers数量</name>
<type>2</type>
<snmp_community/>
<snmp_oid/>
<key>servers</key>
<delay>0</delay>
<history>10d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Geode Cluster</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>severeAlertCount服务器修改次数</name>
<type>2</type>
<snmp_community/>
<snmp_oid/>
<key>severeAlertCount</key>
<delay>0</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Geode Cluster</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>当前Region实体总数</name>
<type>2</type>
<snmp_community/>
<snmp_oid/>
<key>systemRegionEntryCount</key>
<delay>0</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Geode Cluster</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>Geode总内存量</name>
<type>2</type>
<snmp_community/>
<snmp_oid/>
<key>totalHeap</key>
<delay>0</delay>
<history>10d</history>
<trends>365d</trends>
<status>0</status>
<value_type>0</value_type>
<allowed_hosts/>
<units>GB</units>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Geode Cluster</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>Geode成员数</name>
<type>2</type>
<snmp_community/>
<snmp_oid/>
<key>totalMembers</key>
<delay>0</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Geode Cluster</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>Region总数</name>
<type>2</type>
<snmp_community/>
<snmp_oid/>
<key>totalRegions</key>
<delay>0</delay>
<history>10d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Geode Cluster</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
<item>
<name>警告数</name>
<type>2</type>
<snmp_community/>
<snmp_oid/>
<key>warningAlertCount</key>
<delay>0</delay>
<history>90d</history>
<trends>365d</trends>
<status>0</status>
<value_type>3</value_type>
<allowed_hosts/>
<units/>
<snmpv3_contextname/>
<snmpv3_securityname/>
<snmpv3_securitylevel>0</snmpv3_securitylevel>
<snmpv3_authprotocol>0</snmpv3_authprotocol>
<snmpv3_authpassphrase/>
<snmpv3_privprotocol>0</snmpv3_privprotocol>
<snmpv3_privpassphrase/>
<params/>
<ipmi_sensor/>
<authtype>0</authtype>
<username/>
<password/>
<publickey/>
<privatekey/>
<port/>
<description/>
<inventory_link>0</inventory_link>
<applications>
<application>
<name>Geode Cluster</name>
</application>
</applications>
<valuemap/>
<logtimefmt/>
<preprocessing/>
<jmx_endpoint/>
<master_item/>
</item>
</items>
<discovery_rules/>
<httptests/>
<macros>
<macro>
<macro>{$GEODE_HOST}</macro>
<value>geode_host_ip_addr_or_hostname</value>
</macro>
<macro>
<macro>{$GEODE_METRICS_PORT}</macro>
<value>7070</value>
</macro>
<macro>
<macro>{$ZABBIX_NAME}</macro>
<value>name_of_this_host_in_zabbix</value>
</macro>
</macros>
<templates/>
<screens/>
</template>
</templates>
<triggers>
<trigger>
<expression>{Template Cluster Geode:servers.last()}<26</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name>Geode Sever存在宕机，请登陆服务器查看</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>4</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger>
<trigger>
<expression>{Template Cluster Geode:clients.last()}<3</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name>SparkStreaming程序堆积,导致geode客户端连接数量降低</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>1</priority>
<description/>
<type>0</type>
<manual_close>1</manual_close>
<dependencies/>
<tags/>
</trigger>
<trigger>
<expression>{Template Cluster Geode:totalMembers.nodata(120)}=1</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name>在过去的2分钟内未获取到主机[{HOSTNAME}]上Geode的任何数据，请检查数据采集器的日志或者查看该主机Geode运行状态</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>5</priority>
<description/>
<type>0</type>
<manual_close>1</manual_close>
<dependencies/>
<tags/>
</trigger>
</triggers>
</zabbix_export>
Grafana模板JSON文件
{
"__inputs": [
{
"name": "DS_BIANQI",
"label": "bianqi",
"description": "",
"type": "datasource",
"pluginId": "alexanderzobnin-zabbix-datasource",
"pluginName": "Zabbix"
}
],
"__requires": [
{
"type": "datasource",
"id": "alexanderzobnin-zabbix-datasource",
"name": "Zabbix",
"version": "1.0.0"
},
{
"type": "grafana",
"id": "grafana",
"name": "Grafana",
"version": "4.6.0"
},
{
"type": "panel",
"id": "graph",
"name": "Graph",
"version": ""
},
{
"type": "panel",
"id": "singlestat",
"name": "Singlestat",
"version": ""
}
],
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": "-- Grafana --",
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"type": "dashboard"
}
]
},
"editable": true,
"gnetId": null,
"graphTooltip": 2,
"hideControls": true,
"id": null,
"links": [],
"rows": [
{
"collapse": false,
"height": "250px",
"panels": [
{
"cacheTimeout": null,
"colorBackground": true,
"colorValue": false,
"colors": [
"#299c46",
"#cca300",
"#7eb26d"
],
"datasource": "${DS_BIANQI}",
"format": "none",
"gauge": {
"maxValue": 12,
"minValue": 0,
"show": true,
"thresholdLabels": false,
"thresholdMarkers": true
},
"id": 1,
"interval": null,
"links": [],
"mappingType": 1,
"mappingTypes": [
{
"name": "value to text",
"value": 1
},
{
"name": "range to text",
"value": 2
}
],
"maxDataPoints": 100,
"nullPointMode": "connected",
"nullText": null,
"postfix": "",
"postfixFontSize": "50%",
"prefix": "",
"prefixFontSize": "50%",
"rangeMaps": [
{
"from": "null",
"text": "N/A",
"to": "null"
}
],
"span": 3,
"sparkline": {
"fillColor": "rgba(31, 118, 189, 0.18)",
"full": false,
"lineColor": "rgb(31, 120, 193)",
"show": false
},
"tableColumn": "",
"targets": [
{
"application": {
"filter": "Geode Cluster"
},
"functions": [],
"group": {
"filter": "Geode集群"
},
"host": {
"filter": "Geode-Cluster-10.10.21.70"
},
"item": {
"filter": "Geode成员数"
},
"mode": 0,
"options": {
"showDisabledItems": false
},
"refId": "A"
}
],
"thresholds": "",
"title": "Geode成员数",
"type": "singlestat",
"valueFontSize": "80%",
"valueMaps": [
{
"op": "=",
"text": "N/A",
"value": "null"
}
],
"valueName": "current"
},
{
"cacheTimeout": null,
"colorBackground": true,
"colorValue": false,
"colors": [
"#299c46",
"#1f78c1",
"#7eb26d"
],
"datasource": "${DS_BIANQI}",
"format": "none",
"gauge": {
"maxValue": 10,
"minValue": 0,
"show": true,
"thresholdLabels": false,
"thresholdMarkers": true
},
"id": 3,
"interval": null,
"links": [],
"mappingType": 1,
"mappingTypes": [
{
"name": "value to text",
"value": 1
},
{
"name": "range to text",
"value": 2
}
],
"maxDataPoints": 100,
"nullPointMode": "connected",
"nullText": null,
"postfix": "",
"postfixFontSize": "50%",
"prefix": "",
"prefixFontSize": "50%",
"rangeMaps": [
{
"from": "null",
"text": "N/A",
"to": "null"
}
],
"span": 3,
"sparkline": {
"fillColor": "rgba(31, 118, 189, 0.18)",
"full": false,
"lineColor": "rgb(31, 120, 193)",
"show": false
},
"tableColumn": "",
"targets": [
{
"application": {
"filter": "Geode Cluster"
},
"functions": [],
"group": {
"filter": "Geode集群"
},
"host": {
"filter": "Geode-Cluster-10.10.21.70"
},
"item": {
"filter": "servers数量"
},
"mode": 0,
"options": {
"showDisabledItems": false
},
"refId": "A"
}
],
"thresholds": "",
"title": "servers数量",
"type": "singlestat",
"valueFontSize": "80%",
"valueMaps": [
{
"op": "=",
"text": "N/A",
"value": "null"
}
],
"valueName": "current"
},
{
"cacheTimeout": null,
"colorBackground": true,
"colorValue": false,
"colors": [
"#299c46",
"rgb(176, 135, 22)",
"#7eb26d"
],
"datasource": "${DS_BIANQI}",
"format": "none",
"gauge": {
"maxValue": 12,
"minValue": 0,
"show": true,
"thresholdLabels": false,
"thresholdMarkers": true
},
"id": 4,
"interval": null,
"links": [],
"mappingType": 1,
"mappingTypes": [
{
"name": "value to text",
"value": 1
},
{
"name": "range to text",
"value": 2
}
],
"maxDataPoints": 100,
"nullPointMode": "connected",
"nullText": null,
"postfix": "",
"postfixFontSize": "50%",
"prefix": "",
"prefixFontSize": "50%",
"rangeMaps": [
{
"from": "null",
"text": "N/A",
"to": "null"
}
],
"span": 3,
"sparkline": {
"fillColor": "rgba(31, 118, 189, 0.18)",
"full": false,
"lineColor": "rgb(31, 120, 193)",
"show": false
},
"tableColumn": "",
"targets": [
{
"application": {
"filter": "Geode Cluster"
},
"functions": [],
"group": {
"filter": "Geode集群"
},
"host": {
"filter": "Geode-Cluster-10.10.21.70"
},
"item": {
"filter": "Region总数"
},
"mode": 0,
"options": {
"showDisabledItems": false
},
"refId": "A"
}
],
"thresholds": "",
"title": "Region总数",
"type": "singlestat",
"valueFontSize": "80%",
"valueMaps": [
{
"op": "=",
"text": "N/A",
"value": "null"
}
],
"valueName": "current"
},
{
"cacheTimeout": null,
"colorBackground": true,
"colorValue": false,
"colors": [
"#299c46",
"#508642",
"#7eb26d"
],
"datasource": "${DS_BIANQI}",
"format": "none",
"gauge": {
"maxValue": 12,
"minValue": 0,
"show": true,
"thresholdLabels": false,
"thresholdMarkers": true
},
"hideTimeOverride": false,
"id": 5,
"interval": null,
"links": [],
"mappingType": 1,
"mappingTypes": [
{
"name": "value to text",
"value": 1
},
{
"name": "range to text",
"value": 2
}
],
"maxDataPoints": 100,
"nullPointMode": "connected",
"nullText": null,
"postfix": "",
"postfixFontSize": "50%",
"prefix": "",
"prefixFontSize": "50%",
"rangeMaps": [
{
"from": "null",
"text": "N/A",
"to": "null"
}
],
"span": 3,
"sparkline": {
"fillColor": "rgba(31, 118, 189, 0.18)",
"full": false,
"lineColor": "rgb(31, 120, 193)",
"show": false
},
"tableColumn": "",
"targets": [
{
"application": {
"filter": "Geode Cluster"
},
"functions": [],
"group": {
"filter": "Geode集群"
},
"host": {
"filter": "Geode-Cluster-10.10.21.70"
},
"item": {
"filter": "locators数目"
},
"mode": 0,
"options": {
"showDisabledItems": false
},
"refId": "A"
}
],
"thresholds": "",
"title": "locators数目",
"type": "singlestat",
"valueFontSize": "80%",
"valueMaps": [
{
"op": "=",
"text": "N/A",
"value": "null"
}
],
"valueName": "current"
}
],
"repeat": null,
"repeatIteration": null,
"repeatRowId": null,
"showTitle": false,
"title": "Dashboard Row",
"titleSize": "h6"
},
{
"collapse": false,
"height": 334,
"panels": [
{
"aliasColors": {
"连接到geode的客户端数量": "#c15c17"
},
"bars": true,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_BIANQI}",
"fill": 1,
"id": 6,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [],
"nullPointMode": "null",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 12,
"stack": false,
"steppedLine": false,
"targets": [
{
"application": {
"filter": "Geode Cluster"
},
"functions": [],
"group": {
"filter": "Geode集群"
},
"host": {
"filter": "Geode-Cluster-10.10.21.70"
},
"item": {
"filter": "连接到geode的客户端数量"
},
"mode": 0,
"options": {
"showDisabledItems": false
},
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "连接到geode的客户端数量",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
}
],
"repeat": null,
"repeatIteration": null,
"repeatRowId": null,
"showTitle": false,
"title": "Dashboard Row",
"titleSize": "h6"
},
{
"collapse": false,
"height": 299,
"panels": [
{
"aliasColors": {
"Geode总内存量": "#f9d9f9"
},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_BIANQI}",
"fill": 1,
"id": 7,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [],
"nullPointMode": "null",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 3,
"stack": false,
"steppedLine": false,
"targets": [
{
"application": {
"filter": "Geode Cluster"
},
"functions": [],
"group": {
"filter": "Geode集群"
},
"host": {
"filter": "Geode-Cluster-10.10.21.70"
},
"item": {
"filter": "Geode总内存量"
},
"mode": 0,
"options": {
"showDisabledItems": false
},
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "Geode总内存量",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "decgbytes",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
},
{
"aliasColors": {
"region更新数据量": "#1f78c1"
},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_BIANQI}",
"fill": 1,
"id": 8,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [],
"nullPointMode": "null",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 3,
"stack": false,
"steppedLine": false,
"targets": [
{
"application": {
"filter": "Geode Cluster"
},
"functions": [],
"group": {
"filter": "Geode集群"
},
"host": {
"filter": "Geode-Cluster-10.10.21.70"
},
"item": {
"filter": "region更新数据量"
},
"mode": 0,
"options": {
"showDisabledItems": false
},
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "region更新数据量",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
},
{
"aliasColors": {
"severeAlertCount服务器修改次数": "#f29191"
},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_BIANQI}",
"fill": 1,
"id": 9,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [],
"nullPointMode": "null",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 3,
"stack": false,
"steppedLine": false,
"targets": [
{
"application": {
"filter": "Geode Cluster"
},
"functions": [],
"group": {
"filter": "Geode集群"
},
"host": {
"filter": "Geode-Cluster-10.10.21.70"
},
"item": {
"filter": "GeodeJVM停顿"
},
"mode": 0,
"options": {
"showDisabledItems": false
},
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "Geode JVM停顿次数",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
},
{
"aliasColors": {
"错误报警数": "#bf1b00"
},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_BIANQI}",
"fill": 1,
"id": 10,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [],
"nullPointMode": "null",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 3,
"stack": false,
"steppedLine": false,
"targets": [
{
"application": {
"filter": "Geode Cluster"
},
"functions": [],
"group": {
"filter": "Geode集群"
},
"host": {
"filter": "Geode-Cluster-10.10.21.70"
},
"item": {
"filter": "错误报警数"
},
"mode": 0,
"options": {
"showDisabledItems": false
},
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "错误报警数",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
},
{
"aliasColors": {
"警告数": "#962d82",
"错误报警数": "#bf1b00"
},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_BIANQI}",
"fill": 1,
"id": 11,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [],
"nullPointMode": "null",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 3,
"stack": false,
"steppedLine": false,
"targets": [
{
"application": {
"filter": "Geode Cluster"
},
"functions": [],
"group": {
"filter": "Geode集群"
},
"host": {
"filter": "Geode-Cluster-10.10.21.70"
},
"item": {
"filter": "警告数"
},
"mode": 0,
"options": {
"showDisabledItems": false
},
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "警告数",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
},
{
"aliasColors": {
"put速率": "#629e51",
"警告数": "#962d82",
"错误报警数": "#bf1b00"
},
"bars": true,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_BIANQI}",
"fill": 1,
"id": 12,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [],
"nullPointMode": "null",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 3,
"stack": false,
"steppedLine": false,
"targets": [
{
"application": {
"filter": ""
},
"functions": [],
"group": {
"filter": "Geode集群"
},
"host": {
"filter": "Geode-Cluster-10.10.21.70"
},
"item": {
"filter": "put速率"
},
"mode": 0,
"options": {
"showDisabledItems": false
},
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "put速率",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
},
{
"aliasColors": {
"get速率": "#bf1b00",
"警告数": "#962d82",
"错误报警数": "#bf1b00"
},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_BIANQI}",
"fill": 1,
"id": 13,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [],
"nullPointMode": "null",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 3,
"stack": false,
"steppedLine": false,
"targets": [
{
"application": {
"filter": ""
},
"functions": [],
"group": {
"filter": "Geode集群"
},
"host": {
"filter": "Geode-Cluster-10.10.21.70"
},
"item": {
"filter": "get速率"
},
"mode": 0,
"options": {
"showDisabledItems": false
},
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "get速率",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
},
{
"aliasColors": {
"当前Region实体总数": "#cca300",
"警告数": "#962d82",
"错误报警数": "#bf1b00"
},
"bars": true,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_BIANQI}",
"fill": 1,
"id": 15,
"legend": {
"alignAsTable": false,
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [],
"nullPointMode": "null",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 3,
"stack": false,
"steppedLine": false,
"targets": [
{
"application": {
"filter": ""
},
"functions": [],
"group": {
"filter": "Geode集群"
},
"host": {
"filter": "Geode-Cluster-10.10.21.70"
},
"item": {
"filter": "当前Region实体总数"
},
"mode": 0,
"options": {
"showDisabledItems": false
},
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "当前Region实体总数",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "none",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
},
{
"aliasColors": {
"当前内存使用量": "#70dbed",
"警告数": "#962d82",
"错误报警数": "#bf1b00"
},
"bars": true,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_BIANQI}",
"fill": 1,
"id": 16,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [],
"nullPointMode": "null",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 6,
"stack": false,
"steppedLine": false,
"targets": [
{
"application": {
"filter": ""
},
"functions": [],
"group": {
"filter": "Geode集群"
},
"host": {
"filter": "Geode-Cluster-10.10.21.70"
},
"item": {
"filter": "当前内存使用量"
},
"mode": 0,
"options": {
"showDisabledItems": false
},
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "当前内存使用量",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "mbytes",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
},
{
"aliasColors": {
"region更新数据量": "#806eb7",
"警告数": "#962d82",
"错误报警数": "#bf1b00"
},
"bars": true,
"dashLength": 10,
"dashes": false,
"datasource": "${DS_BIANQI}",
"fill": 1,
"id": 18,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [],
"nullPointMode": "null",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"span": 6,
"stack": false,
"steppedLine": false,
"targets": [
{
"application": {
"filter": ""
},
"functions": [],
"group": {
"filter": "Geode集群"
},
"host": {
"filter": "Geode-Cluster-10.10.21.70"
},
"item": {
"filter": "region更新数据量"
},
"mode": 0,
"options": {
"showDisabledItems": false
},
"refId": "A"
}
],
"thresholds": [],
"timeFrom": null,
"timeShift": null,
"title": "region更新数据量",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
]
}
],
"repeat": null,
"repeatIteration": null,
"repeatRowId": null,
"showTitle": false,
"title": "Dashboard Row",
"titleSize": "h6"
}
],
"schemaVersion": 14,
"style": "dark",
"tags": [
"Geode"
],
"templating": {
"list": []
},
"time": {
"from": "now/d",
"to": "now"
},
"timepicker": {
"refresh_intervals": [
"5s",
"10s",
"30s",
"1m",
"5m",
"15m",
"30m",
"1h",
"2h",
"1d"
],
"time_options": [
"5m",
"15m",
"1h",
"6h",
"12h",
"24h",
"2d",
"7d",
"30d"
]
},
"timezone": "",
"title": "BDP-Geode集群监控",
"version": 24
}
GrafanaGeode监控截图
开发使用
创建Region
create region –name=user –type=REPLICATE 注意type类型,热更新非常非常频繁的请使用分片的方式
添加maven依赖
<!-- https://mvnrepository.com/artifact/org.apache.geode/geode-core -->
<dependency>
<groupid>org.apache.geode</groupid>
<artifactid>geode-core</artifactid>
<version>1.2.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.geode/geode-json -->
<dependency>
<groupid>org.apache.geode</groupid>
<artifactid>geode-json</artifactid>
<version>1.2.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.geode/geode-common -->
<dependency>
<groupid>org.apache.geode</groupid>
<artifactid>geode-common</artifactid>
<version>1.2.0</version>
</dependency>
<dependency>
<groupid>log4j</groupid>
<artifactid>log4j</artifactid>
<version>1.2.17</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-core -->
<dependency>
<groupid>org.apache.logging.log4j</groupid>
<artifactid>log4j-core</artifactid>
<version>2.8.2</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-api -->
<dependency>
<groupid>org.apache.logging.log4j</groupid>
<artifactid>log4j-api</artifactid>
<version>2.8.2</version>
</dependency>
新建UserBean.java类
import java.io.DataInput;
import java.io.DataOutput;
import java.io.IOException;
import java.util.Date;
import org.apache.geode.DataSerializable;
import org.apache.geode.DataSerializer;
public class UserBean implements DataSerializable{
/**
* 必须有这个方法，否则查询会报没有init()这个方法
* */
public UserBean() {
}
public UserBean(int id2, int i, String string, Date date) {
// TODO Auto-generated constructor stub
this.id=id2;
this.age=i;
this.name=string;
this.createTime=date;
}
/**
* id
* */
private int id;
/**
* 年龄
* */
private int age;
/**
* 姓名
* */
private String name;
/**
* 创建时间
* */
private Date createTime;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public int getAge() {
return age;
}
public void setAge(int age) {
this.age = age;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public Date getCreateTime() {
return createTime;
}
public void setCreateTime(Date createTime) {
this.createTime = createTime;
}
public void fromData(DataInput in) throws IOException, ClassNotFoundException {
this.id = in.readInt();
this.age=in.readInt();
this.name = in.readUTF();
this.createTime = DataSerializer.readDate(in);
}
public void toData(DataOutput out) throws IOException {
out.writeInt(this.id);
out.writeInt(this.age);
out.writeUTF(this.name);
DataSerializer.writeDate(this.createTime, out);
}
}
添加App.java
public class App
{
String regionName="user";
int userNum=10;
public static void main( String[] args )
{
new App().test();
}
public void test(){
Region region = null;
ClientCache cache = new ClientCacheFactory().addPoolLocator("192.168.60.50", 10334).create();
ClientRegionFactory rf = cache.createClientRegionFactory(ClientRegionShortcut.CACHING_PROXY);
//打开user表
region = rf.create(regionName);
System.out.println("开始创建用户");
create(region);
select(region);
System.out.println("完成创建用户");
System.out.println("开始修改用户");
update(region);
select(region);
System.out.println("完成修改用户");
System.out.println("开始删除用户");
delete(region,108);
select(region);
System.out.println("完成删除用户");
System.out.println("开始清空表");
truncate(region);
select(region);
System.out.println("完成清空表");
//关闭表
region.close();
}
/**
* 查询user表
* */
public void select(Region region){
try {
Object objList = region.query("select * from /"+regionName+" u where u.age > 15");
if(objList != null && objList instanceof ResultsBag){
Iterator iter = ((ResultsBag) objList).iterator();
while(iter.hasNext()){
UserBean userBean = (UserBean) iter.next();
System.out.println("User信息："+JSON.toJSONString(userBean));
}
}
Object obj = region.get(108);
if(obj != null && obj instanceof UserBean){
System.out.println("User108信息"+JSON.toJSONString(obj));
}
}catch (Exception e){
e.printStackTrace();
// logger.error("error occured.", e);
}
}
/**
* 增加10个
* */
public void create(Region region){
for (int i = 0; i < userNum; i++) {
int id = i+100;
region.put(id, new UserBean(id,10+i,"username:"+id,new Date()));
}
}
/**
* 修改
* */
public void update(Region region){
UserBean user108= (UserBean) region.get(108);
if(user108 != null && user108 instanceof UserBean){
System.out.println("User108信息"+JSON.toJSONString(user108));
//修改年龄为
user108.setAge(12);
region.put(user108.getId(), user108);
}
}
/**
* 删除某个用户
* */
public void delete(Region region,Integer id){
region.remove(id);
}
/**
* 清空表
* */
public void truncate(Region region){
region.clear();
}
}
maven打包上传该jar包Geode
deploy –jar=/tmp/hello.jar 如果需要卸载jar undeploy –jar=hello.jar
总结
本文大致介绍了Geode使用和简单的Demo,希望对读者起到抛砖引玉的作用.
参考引用
- http://blog.chinaunix.net/uid-11121450-id-5772480.html
- https://wjw465150.github.io/GeodeUserGuide/Geode_5_Managing_Apache_Geode.html#%E6%80%A7%E8%83%BD%E8%B0%83%E6%95%B4%E5%92%8C%E9%85%8D%E7%BD%AE
- https://gemfire.docs.pivotal.io/99/geode/tools_modules/gfsh/command-pages/alter.html#topic_E74ED23CB60342538B2175C326E7D758
