Resource Management Preparation

Planning Resources

Before configuring resource load management, plan tenant resources based on service models. After services run for a period of time, you can adjust the configurations based on resource usage.

Assume that two departments in a large enterprise use the same cluster. openGauss puts system resources used by the same department in a group to isolate resources for different departments. Table 1 describes the resource plan.

Table 1 Tenant resource plan

Tenant Name	Parameter	Example Value
Tenant A	Sub-Class Cgroup	class_a
	Workload Cgroup	workload_a1 workload_a2
	Group resource pool	resource_pool_a
	Service resource pool	resource_pool_a1 resource_pool_a2
	Group user	tenant_a
	Service user	tenant_a1 tenant_a2
Tenant B	Sub-Class Cgroup	class_b
	Workload Cgroup	workload_b1 workload_b2
	Group resource pool	resource_pool_b
	Service resource pool	resource_pool_b1 resource_pool_b2
	Group user	tenant_b
	Service user	tenant_b1 tenant_b2

Enabling Resource Load Management

Background

This section describes how to configure parameters for resource load management.

Prerequisites

In openGauss, you can manage system resources only as a database administrator. Run the following statement to query user permissions:
```
openGauss=# SELECT rolname FROM pg_roles WHERE rolsystemadmin = 't';
 rolname
---------
 omm
 Jack
(2 rows)
```
Resource load management can be applied only to users with the login permission. Run the following statement to query user permissions:
```
openGauss=# SELECT rolname FROM pg_roles WHERE rolcanlogin = 't';
 rolname
---------
 omm
(1 row)
```

NOTICE:

If a user's login permission is revoked, the user's resource pool will be changed to default_pool. For details about default_pool, see Table 2.

Procedure

You can perform the following steps only as a database administrator to enable load management based on the resource pool. The following uses user omm as an example.

Enable resource pool–based load management.

gs_guc set -N all -I all -c "use_workload_manager=on"

Restart the database for the parameter settings to take effect.
```
gs_om -t stop && gs_om -t start
```

Setting a Cgroup

Background

The core of openGauss resource load management is resource pools. The first step of configuring a resource pool is to configure Cgroups in the environment. For details about Cgroup principles, see the product manual corresponding to your OS. For details about openGauss Cgroups, see Viewing Cgroup Information.

The Class Cgroup is a top-layer Cgroup for database service running. DefaultClass is a sub-category of the Class Cgroup and is automatically created when a cluster is deployed. The Medium Cgroup under DefaultClass contains running jobs that are triggered by the system. Resource configurations of Medium cannot be modified, and the jobs running on it are not controlled by resource management. Therefore, you are advised to create sub-Class and Workload Cgroups to control resource allocation.

Prerequisites

You are familiar with "Server Tools > gs_cgroup" and "Server Tools > gs_ssh" in Tool Reference.

Procedure

NOTE:

To control all the resources in openGauss, you need to create, update, and delete Cgroups on each node. Use gs_ssh (see "Server Tools > gs_ssh" in Tool Reference) to run commands in the steps below.
A Cgroup must be named as follows:
The names of sub-Class Cgroups and Workload Cgroups cannot contain columns (:).
Cgroups having the same name cannot be created.

Creating sub-Class and Workload Cgroups

Log in as the OS user omm to the primary node of openGauss.
Create sub-Class Cgroups class_a and class_b, and allocate 40% and 20% of Class CPU resources to them, respectively.
```
gs_ssh -c "gs_cgroup -c -S class_a -s 40"
```
```
gs_ssh -c "gs_cgroup -c -S class_b -s 20"
```
Create Workload Cgroups workload_a1 and workload_a2 under class_a, and allocate 20% and 60% of class_a CPU resources to them, respectively.
```
gs_ssh -c "gs_cgroup -c -S class_a -G workload_a1 -g 20 "
```
```
gs_ssh -c "gs_cgroup -c -S class_a -G workload_a2 -g 60 "
```
Create Workload Cgroups workload_b1 and workload_b2 under class_b, and allocate 50% and 40% of class_b CPU resources to them, respectively.
```
gs_ssh -c "gs_cgroup -c -S class_b -G workload_b1 -g 50 "
```
```
gs_ssh -c "gs_cgroup -c -S class_b -G workload_b2 -g 40 "
```

Adjusting resource quotas for Cgroups

Change the CPU resource quota for class_a to 30%.
```
gs_ssh -c "gs_cgroup -u -S class_a -s 30"
```
Change the CPU resource quota for workload_a1 under class_a to 30% of class_a CPU resources.
```
gs_ssh -c "gs_cgroup -u -S class_a -G workload_a1 -g 30"
```
NOTICE:
After adjustment, CPU resources allocated to workload_a1 should not be greater than those allocated to class_a. The name of a Workload Cgroup cannot be a default name of the Timeshare Cgroup, that is, Low, Medium, High, or Rush.

Deleting a Cgroup

Delete the Cgroup class_a.
```
gs_ssh -c "gs_cgroup -d  -S class_a"
```
After the command is executed, the Cgroup class_a is deleted.
NOTICE:
User root or a user with the root permission can delete the default Cgroups that can be accessed by a common user username by specifying -d and -U username. A common user can delete existing Class Cgroups by specifying -d and -S classname.

Viewing Cgroup Information

View Cgroup information in configuration files.

gs_cgroup -p

Cgroup configuration

gs_cgroup -p

Top Group information is listed:
GID:   0 Type: Top    Percent(%): 1000( 50) Name: Root                  Cores: 0-47
GID:   1 Type: Top    Percent(%):  833( 83) Name: Gaussdb:omm           Cores: 0-20
GID:   2 Type: Top    Percent(%):  333( 40) Name: Backend               Cores: 0-20
GID:   3 Type: Top    Percent(%):  499( 60) Name: Class                 Cores: 0-20

Backend Group information is listed:
GID:   4 Type: BAKWD  Name: DefaultBackend   TopGID:   2 Percent(%): 266(80) Cores: 0-20
GID:   5 Type: BAKWD  Name: Vacuum           TopGID:   2 Percent(%):  66(20) Cores: 0-20

Class Group information is listed:
GID:  20 Type: CLASS  Name: DefaultClass     TopGID:   3 Percent(%): 166(20) MaxLevel: 1 RemPCT: 100 Cores: 0-20
GID:  21 Type: CLASS  Name: class1           TopGID:   3 Percent(%): 332(40) MaxLevel: 2 RemPCT:  70 Cores: 0-20

Workload Group information is listed:
GID:  86 Type: DEFWD  Name: grp1:2           ClsGID:  21 Percent(%):  99(30) WDLevel:  2 Quota(%): 30 Cores: 0-5

Timeshare Group information is listed:
GID: 724 Type: TSWD   Name: Low              Rate: 1
GID: 725 Type: TSWD   Name: Medium           Rate: 2
GID: 726 Type: TSWD   Name: High             Rate: 4
GID: 727 Type: TSWD   Name: Rush             Rate: 8

Group Exception information is listed:
GID:  20 Type: EXCEPTION Class: DefaultClass
PENALTY: QualificationTime=1800 CPUSkewPercent=30

GID:  21 Type: EXCEPTION Class: class1
PENALTY: AllCpuTime=100 QualificationTime=2400 CPUSkewPercent=90

GID:  86 Type: EXCEPTION Group: class1:grp1:2
ABORT: BlockTime=1200 ElapsedTime=2400

Table 1 lists the Cgroup configuration shown in the above example.

Table 1 Cgroup configuration

GID	Type	Name	Percentage (%)	Remarks
0	Top Cgroup	Root	The number 1000 indicates that the total system resources are divided into 1000 pieces. The number 50 in the parentheses indicates 50% of I/O resources. openGauss does not control I/O resources through Cgroups. Therefore, the following Cgroup information is only about CPU.	-
1		Gaussdb:omm	Only one database program runs in a cluster. The default quota of the Gaussdb:omm Cgroup is 833. That is, the ratio of database programs to non-database programs is 5:1 (833:167).	-
2		Backend	The number 40 in the parentheses indicates that the Backend Cgroup takes up 40% of the resources of the Gaussdb:dbuser Cgroup. The number 60 in the parentheses indicates that the Class Cgroup takes up 60% of the resources of the Gaussdb:dbuser Cgroup.	-
3		Class		-
4	Backend Cgroup	DefaultBackend	The numbers 80 and 20 in the parentheses indicate the percentages of Backend Cgroup resources taken by the DefaultBackend and Vacuum groups, respectively.	TopGID: GID (2) of the Backend Cgroup in a Top Cgroup
5	Backend Cgroup	Vacuum		TopGID: GID (2) of the Backend Cgroup in a Top Cgroup
20	Class Cgroup	DefaultClass	The number 20 in the parentheses indicates that the DefaultClass Cgroup takes up 20% of the Class Cgroup resources. The number 40 in the parentheses indicates that the class1 Cgroup takes up 40% of the Class Cgroup resources. There are only two Class Cgroups currently. Therefore, the system resource quotas for the Class Cgroups (499) are allocated in the ratio of 20:40 (166:332).	TopGID: GID (3) of the Class Cgroups in a Top Cgroup to which the DefaultClass and class1 Cgroups belong. MaxLevel: maximum number of levels for Workload Cgroups in a Class Cgroup. This parameter is set to 1 for DefaultClass because it has no Workload Cgroups. RemPCT: percentage of remaining resources in a Class Cgroup after its resources are allocated to Workload Cgroups. For example, the percentage of remaining resources in the class1 Cgroup is 70%.
21	Class Cgroup	class1
86	Workload Cgroup	grp1:2 (This name is composed of the name of a Workload Cgroup and its level in the Class Cgroup. This grp1:2 Cgroup is the first Workload Cgroup under the class1 Cgroup, and its level is 2. Each Class Cgroup contains a maximum of 10 levels of Workload Cgroups.)	In this example, this Workload Cgroup takes up 30% of class1 Cgroup resources (332 x 30% = 99).	ClsGID: GID of the class1 Cgroup to which the Workload Cgroup belongs. WDLevel: level of the Workload Cgroup in the corresponding Class Cgroup.
724	Timeshare Cgroup	Low	-	Rate: rate of resources allocated to a Timeshare Cgroup. The Low Cgroup has the minimum rate 1 and the Rush Cgroup has the maximum rate 8. The resource rate for Rush:High:Medium:Low is 8:4:2:1 under a Timeshare Cgroup.
725		Medium	-
726		High	-
727		Rush	-

View the Cgroup tree in the OS.

Run the following command to query the structure of the Cgroup tree:

gs_cgroup -P

In the command output, shares indicates the value of cpu.shares, which specifies the dynamic quota of CPU resources in the OS, and cpus indicates the value of cpuset.cpus, which specifies the dynamic quota of CPUSET resources in the OS (number of cores that a Cgroup can use).

Mount Information:
cpu:/dev/cgroup/cpu
blkio:/dev/cgroup/blkio
cpuset:/dev/cgroup/cpuset
cpuacct:/dev/cgroup/cpuacct

Group Tree Information:
- Gaussdb:wangrui (shares: 5120, cpus: 0-20, weight: 1000)
        - Backend (shares: 4096, cpus: 0-20, weight: 400)
                - Vacuum (shares: 2048, cpus: 0-20, weight: 200)
                - DefaultBackend (shares: 8192, cpus: 0-20, weight: 800)
        - Class (shares: 6144, cpus: 0-20, weight: 600)
                - class1 (shares: 4096, cpus: 0-20, weight: 400)
                        - RemainWD:1 (shares: 1000, cpus: 0-20, weight: 100)
                                - RemainWD:2 (shares: 7000, cpus: 0-20, weight: 700)
                                        - Timeshare (shares: 1024, cpus: 0-20, weight: 500)
                                                - Rush (shares: 8192, cpus: 0-20, weight: 800)
                                                - High (shares: 4096, cpus: 0-20, weight: 400)
                                                - Medium (shares: 2048, cpus: 0-20, weight: 200)
                                                - Low (shares: 1024, cpus: 0-20, weight: 100)
                                - grp1:2 (shares: 3000, cpus: 0-5, weight: 300)
                        - TopWD:1 (shares: 9000, cpus: 0-20, weight: 900)
                - DefaultClass (shares: 2048, cpus: 0-20, weight: 200)
                        - RemainWD:1 (shares: 1000, cpus: 0-20, weight: 100)
                                - Timeshare (shares: 1024, cpus: 0-20, weight: 500)
                                        - Rush (shares: 8192, cpus: 0-20, weight: 800)
                                        - High (shares: 4096, cpus: 0-20, weight: 400)
                                        - Medium (shares: 2048, cpus: 0-20, weight: 200)
                                        - Low (shares: 1024, cpus: 0-20, weight: 100)
                        - TopWD:1 (shares: 9000, cpus: 0-20, weight: 900)

Obtain the Cgroup configuration in a system view.
1. Perform the steps in Using gsql to Connect to a Database.
2. Obtain the configuration about all Cgroups in the system.
```
openGauss=# SELECT * FROM gs_all_control_group_info;
```

Creating a Resource Pool

Background

openGauss creates resource pools to divide host resources. After resource load management is enabled, the default resource pool alone is insufficient to address the resource load management requirements of services. Therefore, new resource pools must be used to reallocate system resources for granular control purposes. Table 1 describes the features of a common resource pool.

Table 1 Features of a common resource pool

Resource Pool	Feature
Common resource pool (common scenario)	A common resource pool is associated with a Workload Cgroup, and its group resource pool must not be associated with the corresponding sub-Class Cgroup. For example, to create a service resource pool and associate it with class1:wd, ensure that its group resource pool has not been associated with class1. The default value of mem_percent is 0%. The sum of mem_percent for all common resource groups under the same group resource pool can be greater than or equal to 100%.

Resource Pool

Feature

Common resource pool (common scenario)

A common resource pool is associated with a Workload Cgroup, and its group resource pool must not be associated with the corresponding sub-Class Cgroup. For example, to create a service resource pool and associate it with class1:wd, ensure that its group resource pool has not been associated with class1.
The default value of mem_percent is 0%. The sum of mem_percent for all common resource groups under the same group resource pool can be greater than or equal to 100%.

After resource load management is enabled, the system automatically creates default_pool. If no resource pool is specified for a session or user, they will be automatically associated with default_pool. By default, default_pool is associated with the DefaultClass:Medium Cgroup and does not limit the number of concurrent services. Table 2 describes the attributes of default_pool.

Table 2 default_pool attributes

Attribute	Value	Description
respool_name	default_pool	Name of the resource pool
mem_percent	100	Maximum percentage of used memory
cpu_affinity	–1	CPU affinity (reserved)
control_group	DefaultClass:Medium	Cgroup associated with the resource pool
active_statements	–1	Maximum number of concurrent queries allowed by the resource pool. The value -1 indicates that the maximum number of concurrent queries cannot exceed INT_MAX.
max_dop	1	Concurrency level of execution operators after the SMP is enabled (reserved)
memory_limit	8GB	Upper memory usage limit (reserved)
parentid	0	OID of the parent resource pool
io_limits	0	Upper limit of IOPS. It is counted by ones for column storage and by 10 thousands for row storage. The value 0 indicates that the maximum number of concurrent queries cannot exceed INT_MAX.
io_priority	None	I/O priority set for jobs that consume many I/O resources. It takes effect when the I/O usage reaches 90%. None indicates there is no control.
nodegroup	installation	Name of the logical cluster to which the resource pool belongs. This parameter is invalid in a standalone system.
is_foreign	f	Specifies that the resource pool is not used for users outside the logical cluster. This parameter is invalid in a standalone system.

NOTICE:

default_pool cannot be modified in openGauss.

Prerequisites

You are familiar with the CREATE RESOURCE POOL, ALTER RESOURCE POOL, and DROP RESOURCE POOLsyntax.

Procedure

Creating a resource pool

Perform the steps in gsql connection and usage.
Create a group resource pool and associate it with the specified sub-Class Cgroup. In the following example, the group resource pool named resource_pool_a is associated with the class_a Cgroup.
```
openGauss=# CREATE RESOURCE POOL resource_pool_a WITH (control_group='class_a');
openGauss=# CREATE RESOURCE POOL resource_pool_b WITH (control_group='class_b');
CREATE RESOURCE POOL
```
Create a service resource pool and associate it with the specified Workload Cgroup. In the following example, the service resource pool named resource_pool_a1 is associated with the workload_a1 Cgroup.
```
openGauss=# CREATE RESOURCE POOL resource_pool_a1 WITH (control_group='class_a:workload_a1');
openGauss=# CREATE RESOURCE POOL resource_pool_a2 WITH (control_group='class_a:workload_a2');
openGauss=# CREATE RESOURCE POOL resource_pool_b1 WITH (control_group='class_b:workload_b1');
openGauss=# CREATE RESOURCE POOL resource_pool_b2 WITH (control_group='class_b:workload_b2');
CREATE RESOURCE POOL
```
NOTE:
- If you do not specify an associated Cgroup when creating a resource pool, the resource pool will be associated with the default Cgroup, which is the Timeshare Cgroup Medium under the DefaultClass Cgroup.
- The value of control_group is case-sensitive and must be contained in single quotation marks or double quotation marks.
- If a database user specifies the Timeshare string (Rush, High, Medium, or Low) in the syntax, for example, control_group is set to High, the resource pool will be associated with the High Timeshare Cgroup under DefaultClass.
- control_group allows you to create a Workload Cgroup, for example, class1:wd whose Cgroup level can also be appended, such as class1:wd:2. The Cgroup level must be within 1 to 10, but it is not used for Cgroup differentiation. In earlier versions, you can create Workload Cgroups with the same name and differentiate them by their levels. In the latest version, Cgroup names must be unique. If you have created duplicate Workload Cgroups in an earlier version, delete them to avoid confusion.

Managing resource pools

Modify resource pool attributes. In the following example, the Cgroup associated with the resource pool resource_pool_a2 is changed to class_a:workload_a1 (assuming that class_a:workload_a1 is not associated with any other resource pools).

openGauss=# ALTER RESOURCE POOL resource_pool_a2 WITH (control_group="class_a:workload_a1");
ALTER RESOURCE POOL

Deleting a resource pool

Delete a resource pool. For example, run the following command to delete the resource pool resource_pool_a2:

openGauss=# DROP RESOURCE POOL resource_pool_a2;
DROP RESOURCE POOL

NOTE:

The resource pool cannot be deleted if it is associated with a role.
In a multi-tenant scenario, deleting a group resource pool also deletes the related service resource pools. A resource pool can be deleted only when it is not associated with any users.

Viewing Resource Pool Information

NOTICE:

Do not use the INSERT, UPDATE, DELETE, and TRUNCATE statements in the pg_resource_pool system catalog that manages resource load.
Do not modify the memory_limit and cpu_affinity attributes of a resource pool.

Run the following command to view the information of all the resource pools of the current cluster:

openGauss=# SELECT * FROM PG_RESOURCE_POOL;

   respool_name   | mem_percent | cpu_affinity |    control_group    | active_statements | max_dop | memory_limit | parentid | io_limits | io_priority |  nodegroup   | is_foreign  | max_worker
------------------+-------------+--------------+---------------------+-------------------+---------+--------------+----------+-----------+--------------+--------------+------------
 default_pool     |         100 |           -1 | DefaultClass:Medium |                -1 |       1 | 8GB          |        0 |         0 | None        | installation | f  |
 resource_pool_a  |          20 |           -1 | class_a             |                10 |       1 | 8GB          |        0 |         0 | None        | installation | f  |
 resource_pool_b  |          20 |           -1 | class_b             |                10 |       1 | 8GB          |        0 |         0 | None        | installation | f  |
 resource_pool_a1 |          20 |           -1 | class_a:workload_a1 |                10 |       1 | 8GB          |    16970 |         0 | None        | installation | f  |
 resource_pool_a2 |          20 |           -1 | class_a:workload_a2 |                10 |       1 | 8GB          |    16970 |         0 | None        | installation | f  |
 resource_pool_b1 |          20 |           -1 | class_b:workload_b1 |                10 |       1 | 8GB          |    16971 |         0 | None        | installation | f  |
 resource_pool_b2 |          20 |           -1 | class_b:workload_b2 |                10 |       1 | 8GB          |    16971 |         0 | None        | installation | f  |
(7 rows)

View information about Cgroups associated with a resource pool. For details, see statistics-information-functions.

In the following example, resource_pool_a1 is the name of the resource pool.

openGauss=# SELECT * FROM gs_control_group_info('resource_pool_a1');

        name         |  class  |  workload   | type  | gid | shares | limits | rate | cpucores
---------------------+---------+-------------+-------+-----+--------+--------+------+----------
 class_a:workload_a1 | class_a | workload_a1 | DEFWD |  87 |     30 |      0 |    0 | 0-3
(1 row)

Table 3 gs_control_group_info attributes

Attribute	Value	Description
name	class_a:workload_a1	Class name and workload name
class	class_a	Class Cgroup name
workload	workload_a1	Workload Cgroup name
type	DEFWD	Cgroup type (Top, CLASS, BAKWD, DEFWD, and TSWD)
gid	87	Cgroup ID
shares	30	Percentage of CPU resources to those on the parent node
limits	0	Percentage of CPU cores to those on the parent node
rate	0	Allocation ratio in Timeshare
cpucores	0-3	CPU cores

Resource Management Preparation ​

Planning Resources ​

Enabling Resource Load Management ​

Background ​

Prerequisites ​

Procedure ​

Setting a Cgroup ​

Background ​

Prerequisites ​

Procedure ​

Viewing Cgroup Information ​

Creating a Resource Pool ​

Background ​

Prerequisites ​

Procedure ​

Viewing Resource Pool Information ​

Resource Management Preparation

Planning Resources

Enabling Resource Load Management

Background

Prerequisites

Procedure

Setting a Cgroup

Background

Prerequisites

Procedure

Viewing Cgroup Information

Creating a Resource Pool

Background

Prerequisites

Procedure

Viewing Resource Pool Information