Monitoring Fundamentals
1. Hardware Monitoring
Inspects data center equipment, remote management cards, and IPMI.
2. System Monitoring
- CPU Load: Use
uptime(target < 3),top(ideal 30%-70%),vmstat,mpstat. - Memory: Use
free -m(utilization should be < 80%). - Disk: Use
df,iotop(for I/O process monitoring),iostat. - Network: Use
iftop -n. - Performance Tool:
nmonoffers comprehensive monitoring for testing.
3. Network Monitoring
Tools include iftop, webmaster utilities, SmokePing, and external monitoring services.
CPU Monitoring Concepts
Key Metrics
- Context Switching: The process scheduler's task switching.
- Run Queue Length: Number of processes waiting per time unit.
- Utilization: Percentage of CPU time used.
Performance Baselines
- Run queue length per core should be between 1-3 (e.g., 4 cores < 12).
- CPU utilization: 65%-70% user time (
us), 30%-35% system time (sy), 0%-5% idle (id).
Monitoring Tools
Common commands: top, vmstat 1 10, mpstat (requires sysstat package).
Zabbix Installation
Refer to the official documentation for package installation.
1. Add Zabbix 2.4 Yum Repository
rpm -ivh http://repo.zabbix.com/zabbix/2.4/rhel/6/x86_64/zabbix-release-2.4-1.el6.noarch.rpm
2. Install Zabbix Packages
yum install zabbix zabbix-server zabbix-get zabbix-web zabbix-server-mysql zabbix-web-mysql zabbix-agent
3. Initialize Database
mysql> CREATE DATABASE zabbix CHARACTER SET utf8 COLLATE utf8_bin;
mysql> GRANT ALL PRIVILEGES ON zabbix.* TO 'zabbix'@'localhost' IDENTIFIED BY 'zabbix123';
Populate the database schema and data:
cd /usr/share/doc/zabbix-server-mysql-2.4.8/create/
mysql -uroot -pRoot123 zabbix < schema.sql
mysql -uroot -pRoot123 zabbix < images.sql
mysql -uroot -pRoot123 zabbix < data.sql
4. Configure Timezone
Edit the Apache configuration for Zabbix:
vim /etc/httpd/conf.d/zabbix.conf
Add or modify the line:
php_value date.timezone Asia/Shanghai
5. Configure Zabbix Server Database Connection
Edit the server configuration file:
vim /etc/zabbix/zabbix_server.conf
Set the following parameters:
DBHost=172.16.1.214
DBName=zabbix
DBUser=zabbix
DBPassword=zabbix123
6. Start Services
/etc/init.d/httpd restart
/etc/init.d/zabbix-server start
Access the web interface at http://[server_ip]/zabbix/. Default login is Admin / zabbix.
7. Install and Configure Zabbix Agent
yum install zabbix-agent
vim /etc/zabbix/zabbix_agentd.conf
Set the server adress:
Server=172.16.1.211
8. Start Zabbix Agent
/etc/init.d/zabbix-agent start
Custom Monitoring with Zabbix
1. Add Custom Agent Check (Monitor Logged-in Users)
Edit the agent configuration file:
vim /etc/zabbix/zabbix_agentd.conf
Add a UserParameter:
UserParameter=login.user,uptime | awk '{print $6}'
Restart the agent for changes to take effect.
2. Test the Custom Item from Server
zabbix_get -s 172.16.1.210 -k login.user
Then, create a new Item in the Zabbix web interface using this key. You can also create graphs for this item.
3. Configure Custom Alert Scripts
Place scripts in /usr/lib/zabbix/alertscripts/. They must accept three parameters: recipient, subject, and body.
Example script sms.sh:
#!/bin/bash
RECIPIENT=$1
SUBJECT=$2
MESSAGE=$3
echo "$RECIPIENT $SUBJECT $MESSAGE" >> /tmp/sms.log
4. Configure Alert Actions in Web Interface
Use these macros in action operations:
{ALERT.SENDTO}- Recipient addres.{ALERT.SUBJECT}- Message subject.{ALERT.MESSAGE}- Detailed content.
For a trigger action, you can use a message like:
Status: {TRIGGER.STATUS}
Time: {EVENT.DATE} {EVENT.TIME}
Trigger: {TRIGGER.NAME}
Host: {HOST.NAME}
Item: {ITEM.NAME}: {ITEM.VALUE}
For a recovery action, use:
Status: {TRIGGER.STATUS}
Time: {EVENT.RECOVERY.DATE} {EVENT.RECOVERY.TIME}
Trigger: {TRIGGER.NAME}
Host: {HOST.NAME}
Item: {ITEM.NAME}: {ITEM.VALUE}
Monitoring MySQL with Percona Plugins
1. Install Percona Monitoring Plugins
yum install https://www.percona.com/downloads/percona-monitoring-plugins/percona-monitoring-plugins-1.1.7/binary/redhat/7/x86_64/percona-zabbix-templates-1.1.7-2.noarch.rpm
yum install percona-zabbix-templates php php-mysql
cp /var/lib/zabbix/percona/templates/userparameter_percona_mysql.conf /etc/zabbix/zabbix_agentd.d/
systemctl restart zabbix-agent
2. Import Zabbix Templates
Import the modified template files compatible with Zabbix 3.0+.
3. Configure MySQL Connection
Edit the PHP configuration script:
vim /var/lib/zabbix/percona/scripts/ss_get_mysql_stats.php.cnf
Set database credentials:
<?php
$mysql_user = 'root';
$mysql_pass = 's3cret';
?>
4. Test MySQL Monitoring
zabbix_get -s 172.16.20.11 -k MySQL.max-connections
Zabbix Agent Active Mode
Use active mode when the server queue has many delayed items or when monitoring over 300 hosts.
Configuration
Modify the agent configuration file:
#Server=172.16.100.1 # Comment out passive server setting
StartAgents=0 # Disable passive listening
ServerActive=172.16.100.1 # Address of Zabbix server/proxy
Hostname=tools01-uat # Must match host name as configured on server
In the server template, change item types to "Zabbix agent (active)".
Zabbix Proxy Deployment
1. Install Zabbix Proxy
yum install zabbix-proxy zabbix-proxy-mysql mariadb-server
systemctl start mariadb
2. Create and Initialize Proxy Database
mysql> CREATE DATABASE zabbix_proxy CHARACTER SET utf8;
mysql> GRANT ALL ON zabbix_proxy.* TO 'zabbix_proxy'@'localhost' IDENTIFIED BY 'zabbix_proxy';
Load the schema:
zcat /usr/share/doc/zabbix-proxy-mysql-*/schema.sql.gz | mysql -uzabbix_proxy -pzabbix_proxy zabbix_proxy
Zabbix Automation
1. Agent Auto-Registration
Configure the agent to enable auto-registration:
vim /etc/zabbix/zabbix_agentd.conf
ServerActive=172.16.100.1
Hostname=tools01-uat.hewomi.com
HostMetadataItem=system.uname
On the Zabbix server, create an "Auto registration" action to add new hosts.
2. Using Zabbix API for Automation
First, obtain an authentication token:
curl -s -X POST -H 'Content-Type: application/json-rpc' -d '{
"jsonrpc": "2.0",
"method": "user.login",
"params": {
"user": "Admin",
"password": "password"
},
"id": 1,
"auth": null
}' http://172.16.100.1/zabbix/api_jsonrpc.php
Use the returned token to make subsequent API calls, such as listing hosts or creating new ones.
3. Python Script Example for API Authentication
#!/usr/bin/env python
import requests
import json
url = 'http://192.168.56.11/zabbix/api_jsonrpc.php'
post_data = {
"jsonrpc": "2.0",
"method": "user.login",
"params": {
"user": "zhangsan",
"password": "123123"
},
"id": 1
}
post_header = {'Content-Type': 'application/json-rpc'}
response = requests.post(url, data=json.dumps(post_data), headers=post_header)
result = json.loads(response.text)
if 'result' in result:
print(result['result'])
else:
print('Login error')
After authentication, use the API to add hosts, assign templates, and manage groups programmatically.