1.1 拓扑域划分
在超大规模集群中,可通过区域、可用区、数据中心、机柜等维度划分拓扑域,通过节点标签实现精准划分,示例如下:
# 按区域划分拓扑域(北京、南京区域)
kubectl label node k8s-master01 k8s-node01 region=beijing
kubectl label node k8s-node02 region=nanjing
# 按可用区划分(北京区域下分海淀、朝阳可用区)
kubectl label node k8s-master01 zone=beijing-haidian
kubectl label node k8s-node01 zone=beijing-chaoyang
# 按机房划分(海淀可用区下分不同机房)
kubectl label node k8s-master01 engineroom=beijing-haidian-c1
1.2 K8s 亲和力实战
1.2.1 同一个应用必须部署在不同的宿主机
通过podAntiAffinity的requiredDuringSchedulingIgnoredDuringExecution,强制应用副本不处于同一宿主机(拓扑域为宿主机 hostname)。
apiVersion: apps/v1
kind: Deployment
metadata:
name: diff-nodes # 部署名称
labels:
app: diff-nodes # 应用标签,用于关联亲和性规则
spec:
selector:
matchLabels:
app: diff-nodes # 选择匹配标签的Pod
replicas: 2 # 部署2个副本
template:
metadata:
labels:
app: diff-nodes # Pod标签,与亲和性规则匹配
spec:
affinity:
podAntiAffinity: # Pod反亲和性:避免同一应用副本在同一拓扑域
requiredDuringSchedulingIgnoredDuringExecution: # 调度时必须满足,运行时忽略
- labelSelector: # 选择匹配的Pod标签
matchExpressions:
- key: app # 标签键
operator: In # 匹配规则:包含指定值
values: [diff-nodes] # 匹配的标签值
topologyKey: kubernetes.io/hostname # 拓扑域键:按宿主机划分
containers:
- name: diff-nodes # 容器名称
image: registry.cn-beijing.aliyuncs.com/dotbalo/nginx:1.15.12 # 容器镜像
imagePullPolicy: IfNotPresent # 镜像拉取策略:本地有则不拉取
1.2.2 同一个应用尽量部署在不同的宿主机
通过podAntiAffinity的preferredDuringSchedulingIgnoredDuringExecution,优先让应用副本不处于同一宿主机,不强制要求。
apiVersion: apps/v1
kind: Deployment
metadata:
name: diff-nodes # 部署名称
labels:
app: diff-nodes # 应用标签
spec:
selector:
matchLabels:
app: diff-nodes # 匹配Pod标签
replicas: 2 # 2个副本
template:
metadata:
labels:
app: diff-nodes # Pod标签
spec:
affinity:
podAntiAffinity: # Pod反亲和性
preferredDuringSchedulingIgnoredDuringExecution: # 调度时优先满足,运行时忽略
- podAffinityTerm: # 亲和性条件
labelSelector:
matchExpressions:
- key: app
operator: In
values: [diff-nodes]
topologyKey: kubernetes.io/hostname # 按宿主机划分拓扑域
weight: 100 # 优先级权重(0-100,数值越高优先级越高)
containers:
- name: diff-nodes
image: registry.cn-beijing.aliyuncs.com/dotbalo/nginx:1.15.12
imagePullPolicy: IfNotPresent
1.2.3 同一个应用分布在不同的机房
通过Pod反亲和性,优先让应用副本分布在不同可用域(拓扑域为zone),提升服务高可用性。
apiVersion: apps/v1
kind: Deployment
metadata:
name: diff-zone # 部署名称(区分不同可用域)
labels:
app: diff-zone # 应用标签
spec:
selector:
matchLabels:
app: diff-zone # 匹配Pod标签
replicas: 3 # 3个副本,分布在不同可用域
template:
metadata:
labels:
app: diff-zone # Pod标签
spec:
affinity:
podAntiAffinity: # Pod反亲和性
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values: [diff-zone]
topologyKey: zone # 拓扑域键:按可用区划分
weight: 100 # 高优先级
containers:
- name: diff-nodes
image: registry.cn-beijing.aliyuncs.com/dotbalo/nginx:1.15.12
imagePullPolicy: IfNotPresent
1.2.4 应用尽量和缓存服务部署在同一个可用域
通过Pod亲和性,优先让应用与缓存服务部署在同一宿主机(拓扑域为hostname),提升访问性能。
# 1. 先部署缓存服务(标签为app: cache)
kubectl create deploy cache --image=registry.cn-beijing.aliyuncs.com/dotbalo/redis
# 2. 部署应用,通过亲和性关联缓存服务
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-app # 应用部署名称
spec:
replicas: 2 # 2个应用副本
selector:
matchLabels:
app: my-app # 应用Pod标签
template:
metadata:
labels:
app: my-app # Pod标签
spec:
affinity:
podAffinity: # Pod亲和性:应用与缓存服务尽量同域
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100 # 高优先级
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values: [cache] # 匹配缓存服务的标签
topologyKey: kubernetes.io/hostname # 按宿主机划分
containers:
- name: my-app
image: registry.cn-beijing.aliyuncs.com/dotbalo/nginx:1.15.12
imagePullPolicy: IfNotPresent
1.2.5 计算服务必须部署至高性能机器
通过nodeAffinity的requiredDuringSchedulingIgnoredDuringExecution,强制计算服务部署在指定高性能节点(标签disktype为ssd或nvme)。
apiVersion: apps/v1
kind: Deployment
metadata:
name: compute # 计算服务部署名称
spec:
replicas: 2 # 2个计算服务副本
selector:
matchLabels:
app: compute # 应用标签
template:
metadata:
labels:
app: compute # Pod标签
spec:
affinity:
nodeAffinity: # 节点亲和性:指定部署节点
requiredDuringSchedulingIgnoredDuringExecution: # 调度时必须满足
nodeSelectorTerms:
- matchExpressions:
- key: disktype # 节点标签键(标识高性能磁盘)
operator: In # 匹配规则:包含指定值
values: [ssd, nvme] # 高性能磁盘类型
containers:
- name: compute
image: registry.cn-beijing.aliyuncs.com/dotbalo/nginx:1.15.12
imagePullPolicy: IfNotPresent
1.2.6 计算服务尽量部署至高性能机器
通过nodeAffinity的preferredDuringSchedulingIgnoredDuringExecution,优先将计算服务部署在高性能节点,可设置权重区分优先级。
apiVersion: apps/v1
kind: Deployment
metadata:
name: compute # 计算服务部署名称
spec:
replicas: 2
selector:
matchLabels:
app: compute
template:
metadata:
labels:
app: compute
spec:
affinity:
nodeAffinity: # 节点亲和性
preferredDuringSchedulingIgnoredDuringExecution: # 优先满足
# 优先级100:优先部署在ssd节点
- weight: 100
preference:
matchExpressions:
- key: disktype
operator: In
values: [ssd]
# 优先级50:其次部署在nvme节点
- weight: 50
preference:
matchExpressions:
- key: disktype
operator: In
values: [nvme]
1.2.7 应用尽量不部署至低性能机器
通过nodeAffinity,优先让应用不部署在低性能节点(标签performance为low),使用NotIn匹配规则。
apiVersion: apps/v1
kind: Deployment
spec:
name: compute-intensive-app # 计算密集型应用名称
replicas: 3
selector:
matchLabels:
app: compute-intensive # 应用标签
template:
metadata:
labels:
app: compute-intensive # Pod标签
spec:
affinity:
nodeAffinity: # 节点亲和性
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
preference:
matchExpressions:
- key: performance # 节点性能标签
operator: NotIn # 匹配规则:不包含指定值
values: [low] # 低性能节点标签值
containers:
- name: compute-intensive
image: registry.cn-beijing.aliyuncs.com/dotbalo/nginx:1.15.12
imagePullPolicy: IfNotPresent
1.2.8 应用均匀分布在不同的机房
通过topologySpreadConstraints(拓扑域约束),确保应用副本在不同拓扑域(如宿主机)均匀分布,控制副本偏差。
apiVersion: apps/v1
kind: Deployment
metadata:
name: example-deployment # 部署名称
spec:
replicas: 3 # 3个副本,均匀分布
selector:
matchLabels:
app: example # 应用标签
template:
metadata:
labels:
app: example # Pod标签
spec:
topologySpreadConstraints: # 拓扑域约束(多个约束需全部满足)
- maxSkew: 1 # 最大副本偏差:任意拓扑域间副本数相差不超过1
whenUnsatisfiable: DoNotSchedule # 不满足约束时,不调度新Pod
topologyKey: kubernetes.io/hostname # 拓扑域键:按宿主机划分
labelSelector: # 应用约束的Pod标签选择器
matchLabels:
app: example
containers:
- name: example
image: registry.cn-beijing.aliyuncs.com/dotbalo/nginx:1.15.12
# 补充说明:
# whenUnsatisfiable可选值:
# - DoNotSchedule:不满足约束时,拒绝调度
# - ScheduleAnyway:不满足约束时,仍允许调度