Anomaly Detection

Overview

The anomaly detection module implements time series data based on statistics methods to detect possible exceptions in the data. The framework of this module is decoupled to flexibly replace different anomaly detection algorithms. In addition, this module can automatically select algorithms based on different features of time series data. It supports anomaly value detection, threshold detection, box plot detection, gradient detection, growth rate detection, fluctuation rate detection, and status conversion detection.

Usage Guide

Assume the metric collection system is running properly and the configuration file directory confpath has been initialized. You can run the following commands to enable this feature:

Enable only the anomaly detection function:

gs_dbmind service start --conf confpath --only-run anomaly_detection

View data for a metric on all nodes from timestamps1 to timestamps2:

gs_dbmind component anomaly_detection --conf confpath --action overview --metric metric_name --start-time timestamps1 --end-time timestamps2

View data for a metric on a specific node from timestamps1 to timestamps2:

gs_dbmind component anomaly_detection --conf confpath --action overview --metric metric_name --start-time timestamps1 --end-time timestamps2 --host ip_address --anomaly anomaly_type

View data for a metric on all nodes from timestamps1 to timestamps2 using a specific anomaly detection mode:

gs_dbmind component anomaly_detection --conf confpath --action overview --metric metric_name --start-time timestamps1 --end-time timestamps2 --anomaly anomaly_type

View data for a metric on a specific node from timestamps1 to timestamps2 using a specific anomaly detection mode:

gs_dbmind component anomaly_detection --conf confpath --action overview --metric metric_name --start-time timestamps1 --end-time timestamps2 --host ip_address --anomaly anomaly_type

Visualize data for a metric on all nodes from timestamps1 to timestamps2 using a specific anomaly detection mode:

gs_dbmind component anomaly_detection --conf confpath --action plot --metric metric_name --start-time timestamps1 --end-time timestamps2 --host ip_address --anomaly anomaly_type

Stop the running service:

gs_dbmind service stop --conf confpath

NOTE

When configuring anomaly detection parameters, ensure that the start-time is at least 30 seconds earlier than the end-time.

Obtaining Help Information

You can run the --help command to obtain the help information. For example:

gs_dbmind component anomaly_detection --help

The following information is displayed:

usage: anomaly_detection.py [-h] --action {overview,plot} -c CONF -m METRIC -s
                            START_TIME -e END_TIME [-H HOST] [-a ANOMALY]

Workload Anomaly detection: Anomaly detection of monitored metric.

optional arguments:
  -h, --help            show this help message and exit
  --action {overview,plot}
                        choose a functionality to perform
  -c CONF, --conf CONF  set the directory of configuration files
  -m METRIC, --metric METRIC
                        set the metric name you want to retrieve
  -s START_TIME, --start-time START_TIME
                        set the start time of for retrieving in ms
  -e END_TIME, --end-time END_TIME
                        set the end time of for retrieving in ms
  -H HOST, --host HOST  set a host of the metric, ip only or ip and port.
  -a ANOMALY, --anomaly ANOMALY
                        set a anomaly detector of the metric(increase_rate,
                        level_shift, spike, threshold)

Process finished with exit code 0

Command Reference

Table 1 Command Line Parameters

Parameter	Description	Value Range
-h, --help	Help command	-
--action	Action parameter	overview plot: visualization
-c, --conf	Configuration file directory	-
-m, --metric-name	Metric name to be displayed	-
-H, --host	Data source IP address used to filter data	IP address or IP address + port
-a, --anomaly	Anomaly detection mode used for filtering	-
-s, --start-time	Start time as a timestamp in milliseconds, or in the format %Y-%m-%d %H:%M:%S.	Positive integer or date and time format
-e, --end-time	End time as a timestamp in milliseconds, or in the format %Y-%m-%d %H:%M:%S.	Positive integer or date and time format

Troubleshooting

Overview scenario failure: Ensure the configuration file path is correct and the configuration information is complete. Verify the metric name, host IP address, and anomaly detection type are accurate, and check if the metric data exists within the specified start and end times.
Visualization scenario failure: Ensure the configuration file path is correct and the configuration information is complete. Verify the metric name, host IP address, and anomaly detection type are accurate, and check if the metric data exists within the specified start and end times.

Anomaly Detection ​

Overview ​

Usage Guide ​

Obtaining Help Information ​

Command Reference ​