管理 Pod 稳定性

本节提供有关如何使用服务质量、优先级类、驱逐阈值和资源监控来维护集群节点 Pod 稳定性的信息。

适用于 Kubernetes 的 Redis Enterprise

Kubernetes 集群管理系统资源的分配，并可以驱逐 Pod 以释放系统资源。 以下是配置 Redis Enterprise 节点 Pod 以保持 Pod 稳定性的一些方法：

保证服务质量

正在运行的 Pod 分配了服务质量度量，即 以下三个服务质量等级之一： 保证、可突发和尽力而为。 您可以确保将 Guaranteed 类分配给 Redis Enterprise 节点 Pod 通过遵循正确的指导方针。

要获取分配的 Guaranteed quality of service class （保证服务质量） 类：

  • Pod 中的每个容器都必须具有内存限制和内存请求，并且它们必须相同。
  • Pod 中的每个容器都必须具有 CPU 限制和 CPU 请求，并且它们必须相同。

如果 Redis Enterprise CRD 中未指定资源限制和请求，则 作员创建的默认版本满足这些要求。的 Redis Enterprise 集群 CRD， 否则，您必须在redisEnterpriseNodeResources部分。

Sidecar 容器还会影响 Pod 的服务质量类分配。

要检查任何正在运行的 Redis Enterprise 节点 Pod 的服务质量类，请运行：

kubectl get pod rec-0 --o jsonpath="{.status.qosClass}"

where rec-0 is the name of one of the pods associated with the Redis Enterprise cluster.

Using priority to protect from preemption

When a Redis Enterprise node pod is scheduled, it can be assigned a priority class with the priorityClassName property. This property value is the name of a priority class that must already exist within the cluster.

A sufficiently high priority will prevent other workloads with a lower priority from preempting the scheduling of Redis Enterprise Nodes. Similarly, a high value may also prevent eviction when lower priority workloads are deployed on the same cluster.

The successful use of this strategy involves first creating a priority class with a very large priority value:

apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: redis-enterprise-priority
value: 1000000000
globalDefault: false
description: "This priority class should be used for Redis Enterprise pods only."

Then, you refer to the priority class by name in your Redis Enterprise cluster CRD:

apiVersion: app.redislabs.com/v1
kind: RedisEnterpriseCluster
metadata:
  name: example-redisenterprisecluster
spec:
  size: 3
  priorityClassName: "redis-enterprise-priority"

Alternatively, you can also disable preemption entirely.

Managing eviction thresholds

Eviction thresholds are typically managed by kubelet arguments. You can set the thresholds:

We recommend that you:

  • Set the soft eviction threshold to be higher than the hard eviction threshold. The high soft threshold makes the node condition change earlier, and alerts the administrator.
  • Set eviction-max-pod-grace-period high enough to allow the RS pods to migrate the Redis databases before the pods are force killed.
  • Set the eviction-soft-grace-period high enough that the administrator (or a k8s auto-scaling mechanism) scales k8s up or out.

Monitoring for memory and disk usage

We recommend that you monitor the node for MemoryPressure and DiskPressure. When both of these conditions are true, then an eviction threshold is met and the pod is evicted.

To retrieve the flags, run the command:

$kubectl get nodes -o jsonpath='{range .items[*]}name:{.metadata.name}{"\t"}MemoryPressure:{.status.conditions[?(@.type == "MemoryPressure")].status}{"\t"}DiskPressure:{.status.conditions[?(@.type == "DiskPressure")].status}{"\n"}{end}'

name:gke-55d1ac88-213c	MemoryPressure:False	DiskPressure:False
name:gke-55d1ac88-vrpp	MemoryPressure:False	DiskPressure:False
name:gke-7253cc19-42g0	MemoryPressure:False	DiskPressure:False
RATE THIS PAGE
Back to top ↑