https://blog.boychai.xyz/index.php/tag/%E8%B5%84%E6%BA%90%E9%99%90%E5%88%B6/ https://blog.boychai.xyz/index.php/archives/46/ 2022-12-20T02:57:00+00:00 LimitRange概述默认情况下，Kubernetes集群上的容器对计算资源维度没有任何限制，可能会导致个别容器资源过大导致影响其他容器正常工作，这时可以使用LimitRange定义容器默认CPU和内存请求值或者最大限制。LimitRange维度限制：限制容器配置requests.cpu/memory,limits.cpu/memory的最小、最大值限制容器配置requests.cpu/memory,limits.cpu/memory的默认值限制PVC配置requests.storage的最小、最大值使用前提LimitRange功能是一个准入控制插件，默认已经启用。检查是否开启LimitRange功能的方法如下：[root@master ~]# kubectl -n kube-system get pod|grep apiserver kube-apiserver-master.host.com 1/1 Running 28 (95m ago) 62d [root@master ~]# kubectl -n kube-system exec kube-apiserver-master.host.com -- kube-apiserver -h|grep enable-admission-plugins --admission-control strings Admission is divided into two phases. In the first phase, only mutating admission plugins run. In the second phase, only validating admission plugins run. The names in the below list may represent a validating plugin, a mutating plugin, or both. The order of plugins in which they are passed to this flag does not matter. Comma-delimited list of: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. (DEPRECATED: Use --enable-admission-plugins or --disable-admission-plugins instead. Will be removed in a future version.) --enable-admission-plugins strings admission plugins that should be enabled in addition to default enabled ones (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, PodSecurity, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, MutatingAdmissionWebhook, ValidatingAdmissionWebhook, ResourceQuota). Comma-delimited list of admission plugins: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. The order of plugins in this flag does not matter.在"--enable-admission-plugins"中寻找"LimitRanger"发现已经开启。资源清单计算资源最大、最小限制apiVersion: v1 kind: LimitRange metadata: name: cpu-memory-max-min namespace: test spec: limits: - max: cpu: 1 memory: 1Gi min: cpu: 100m memory: 100Mi type: Containermax里面是容器能设置limit的最大值，min里面是容器能设置request的最小值计算资源默认值apiVersion: v1 kind: LimitRange metadata: name: cpu-memory-max-min namespace: test spec: limits: - max: cpu: 1 memory: 1Gi min: cpu: 100m memory: 100Mi default: cpu: 500m memory: 500Mi defaultRequest: cpu: 100m memory: 100Mi type: Container"default"是设置limit的默认值，"defaultRequest"是设置request的默认值存储资源最大、最小限制apiVersion: v1 kind: LimitRange metadata: name: storage-max-min namespace: test spec: limits: - max: storage: 10Gi min: storage: 1Gi type: PersistentVolumeClaimpvc的使用维度状态[root@master cks]# kubectl get limits -n test NAME CREATED AT cpu-memory-max-min 2022-12-17T11:01:11Z storage-max-min 2022-12-17T10:59:56Z [root@master cks]# kubectl describe limits -n test Name: cpu-memory-max-min Namespace: test Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio ---- -------- --- --- --------------- ------------- ----------------------- Container memory 100Mi 1Gi 100Mi 500Mi - Container cpu 100m 1 100m 500m - Name: storage-max-min Namespace: test Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio ---- -------- --- --- --------------- ------------- ----------------------- PersistentVolumeClaim storage 1Gi 10Gi - - - https://blog.boychai.xyz/index.php/archives/45/ 2022-12-17T06:36:00+00:00 ResourceQuota概述当多个团队去共享使用一个Kubernetes集群时，会出现不均匀的资源使用，默认情况下资源先到先得，这个时候可以通过ResourceQuota来对命名空间资源使用总量做限制，从而解决这个问题。使用前提ResourceQuota功能是一个准入控制插件，默认已经启用。检查是否开启ResourceQuota功能的方法如下：[root@master ~]# kubectl -n kube-system get pod|grep apiserver kube-apiserver-master.host.com 1/1 Running 27 (17h ago) 61d [root@master ~]# kubectl -n kube-system exec kube-apiserver-master.host.com -- kube-apiserver -h|grep enable-admission-plugins --admission-control strings Admission is divided into two phases. In the first phase, only mutating admission plugins run. In the second phase, only validating admission plugins run. The names in the below list may represent a validating plugin, a mutating plugin, or both. The order of plugins in which they are passed to this flag does not matter. Comma-delimited list of: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. (DEPRECATED: Use --enable-admission-plugins or --disable-admission-plugins instead. Will be removed in a future version.) --enable-admission-plugins strings admission plugins that should be enabled in addition to default enabled ones (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, PodSecurity, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, MutatingAdmissionWebhook, ValidatingAdmissionWebhook, ResourceQuota). Comma-delimited list of admission plugins: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. The order of plugins in this flag does not matter. 在"--enable-admission-plugins"中寻找"ResourceQuota"发现已经开启。支持的资源支持的资源描述limits.cpu/memory所有Pod上限资源配置总量不超过该值 (所有非终止状态的Pod)requests.cpu/memory所有Pod请求资源配置总量不超过该值 (所有非终止状态的Pod)cpu/memory等同于requests.cpu/requests.memoryrequests.storage所有PVC请求容量总和不超过该值persistentvolumeclaims所有PVC数量总和不超过该值\<storage-class-name\>.storageclass.storage.k8s.io/requests.storage所有与\<storage-class-name\>相关的PVC请求容量总和不超过该值\<storage-class-name\>.storageclass.storage.k8s.io/persistentvolumeclaims所有与\<storage-class-name\>相关的PVC数量总和不超过该值pods、 count/deployments.apps、count/statfulsets.apps、count/services(services.loadbalancers、 services.nodeports)count/secrets、 count/configmaps、count/job.batch、count/cronjobs.batch创建资源数量不超过该值资源清单计算资源配额apiVersion: v1 kind: ResourceQuota metadata: name: compute-resources namespace: test spec: hard: requests.cpu: "1" requests.memory: 10Gi limits.cpu: "4" limits.memory: 20Gi存储资源配额apiVersion: v1 kind: ResourceQuota metadata: name: storage-resources namespace: test spec: hard: requests.storage: 10Gi managed-nfs-storage.storageclass.storage.k8s.io/requests.storage: 10Gi"managed-nfs-storage"是动态存储类的名称。对象数量配额apiVersion: v1 kind: ResourceQuota metadata: name: object-counts namespace: test spec: hard: pods: "10" count/deployments.apps: "3" count/services: "3"限制的是个数，命名空间的总数量不能超过该值。配额状态[root@master ~]# kubectl get quota -n test NAME AGE REQUEST LIMIT compute-resources 41m requests.cpu: 0/4, requests.memory: 0/10Gi limits.cpu: 0/6, limits.memory: 0/12Gi object-counts 4m6s count/deployments.apps: 0/3, count/services: 0/3, pods: 0/10 storage-resources 6m16s managed-nfs-storage.storageclass.storage.k8s.io/requests.storage: 0/10Gi, requests.storage: 0/10Gi通过上面的命令可以查看额配资源使用的情况。