- 已编辑
操作系统信息：物理机，Centos7.8.2003，4C/32G
Kubernetes版本信息：v1.16。多节点(1master 3node)。
KubeSphere版本信息：v2.1.1,在线安装。已有K8s安装。
问题是什么:openldap pod 处于pending状态，删除之后自动重启pod仍然pending。
[root@master local]# kubectl get pods -n kubesphere-system
NAME READY STATUS RESTARTS AGE
ks-account-6b7cd45ccd-fjrxg 0/1 Init:1/2 2 10d
ks-account-8585dc85bc-5h89l 0/1 Init:1/2 0 24h
ks-apigateway-75b49ddb5-pc6hx 1/1 Running 233 24h
ks-apiserver-6d994fd644-q7psq 1/1 Running 0 24h
ks-console-6f5846c74f-mk9mx 1/1 Running 0 24h
ks-controller-manager-896c578b-sbzdf 1/1 Running 1 24h
ks-installer-7d9fb945c7-92w8c 1/1 Running 5 208d
openldap-0 0/1 Pending 0 58m
redis-6fd6c6d6f9-7524n 检查openldap详细信息:
[root@master local]# kubectl describe pod openldap-0 -n kubesphere-system
Name: openldap-0
Namespace: kubesphere-system
Priority: 0
Node: <none>
Labels: app.kubernetes.io/instance=ks-openldap
app.kubernetes.io/name=openldap-ha
controller-revision-hash=openldap-5b89576789
statefulset.kubernetes.io/pod-name=openldap-0
Annotations: <none>
Status: Pending
IP:
IPs: <none>
Controlled By: StatefulSet/openldap
Containers:
openldap-ha:
Image: osixia/openldap:1.3.0
Port: 389/TCP
Host Port: 0/TCP
Args:
--copy-service
--loglevel=warning
Liveness: tcp-socket :389 delay=30s timeout=1s period=15s #success=1 #failure=3
Readiness: tcp-socket :389 delay=30s timeout=1s period=15s #success=1 #failure=3
Environment:
LDAP_ORGANISATION: kubesphere
LDAP_DOMAIN: kubesphere.io
LDAP_CONFIG_PASSWORD: admin
LDAP_ADMIN_PASSWORD: admin
LDAP_REPLICATION: false
LDAP_TLS: false
LDAP_REMOVE_CONFIG_AFTER_SETUP: true
MY_POD_NAME: openldap-0 (v1:metadata.name)
HOSTNAME: $(MY_POD_NAME).openldap
Mounts:
/etc/ldap/slapd.d from openldap-pvc (rw,path="ldap-config")
/var/lib/ldap from openldap-pvc (rw,path="ldap-data")
/var/run/secrets/kubernetes.io/serviceaccount from default-token-w29wh (ro)
Conditions:
Type Status
PodScheduled False
Volumes:
openldap-pvc:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: openldap-pvc-openldap-0
ReadOnly: false
default-token-w29wh:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-w29wh
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: CriticalAddonsOnly
node-role.kubernetes.io/master:NoSchedule
node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 5m27s (x55 over 60m) default-scheduler 0/3 nodes are available: 1 node(s) had taints that the pod didn't tolerate, 2 node(s) had volume node affinity conflict.看上去挂载卷节点亲缘关系冲突，查看持久卷信息
[root@master local]# kubectl get pv |grep 'openldap'
pvc-18b413c5-ca12-4d38-911a-fb53d6db9f83 2Gi RWO Delete Terminating kubesphere-system/openldap-pvc-openldap-0 local 211d
`
持久卷处于terminating状态挂起，查看持久卷描述信息：
`[root@master local]# kubectl describe pv pvc-18b413c5-ca12-4d38-911a-fb53d6db9f83
Name: pvc-18b413c5-ca12-4d38-911a-fb53d6db9f83
Labels: openebs.io/cas-type=local-hostpath
Annotations: pv.kubernetes.io/provisioned-by: openebs.io/local
Finalizers: [kubernetes.io/pv-protection]
StorageClass: local
Status: Terminating (lasts 4h13m)
Claim: kubesphere-system/openldap-pvc-openldap-0
Reclaim Policy: Delete
Access Modes: RWO
VolumeMode: Filesystem
Capacity: 2Gi
Node Affinity:
Required Terms:
Term 0: kubernetes.io/hostname in [master]
Message:
Source:
Type: LocalVolume (a persistent volume backed by local storage on a node)
Path: /var/openebs/local/pvc-18b413c5-ca12-4d38-911a-fb53d6db9f83
Events: <none>查看master节点标签
[root@master local]# kubectl describe node master
Name: master
Roles: master
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
kubernetes.io/arch=amd64
kubernetes.io/hostname=master
kubernetes.io/os=linux
node-role.kubernetes.io/master=
Annotations: alpha.kubernetes.io/provided-node-ip: 192.168.0.251
kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Mon, 12 Oct 2020 11:12:03 +0800
Taints: node.kubernetes.io/disk-pressure:NoSchedule
Unschedulable: false
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
NetworkUnavailable False Tue, 11 May 2021 11:36:04 +0800 Tue, 11 May 2021 11:36:04 +0800 CalicoIsUp Calico is running on this node
MemoryPressure False Tue, 11 May 2021 15:32:35 +0800 Thu, 04 Mar 2021 11:09:35 +0800 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure True Tue, 11 May 2021 15:32:35 +0800 Sat, 01 May 2021 03:19:56 +0800 KubeletHasDiskPressure kubelet has disk pressure
PIDPressure False Tue, 11 May 2021 15:32:35 +0800 Thu, 04 Mar 2021 11:09:35 +0800 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Tue, 11 May 2021 15:32:35 +0800 Thu, 04 Mar 2021 11:09:37 +0800 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.0.251
Hostname: master
Capacity:
cpu: 4
ephemeral-storage: 51175Mi
hugepages-1Gi: 0
hugepages-2Mi: 0
memory: 32612896Ki
pods: 110
Allocatable:
cpu: 3550m
ephemeral-storage: 48294789041
hugepages-1Gi: 0
hugepages-2Mi: 0
memory: 31760496Ki
pods: 110
System Info:
Machine ID: 94c90834363b4d6fa977f2b25b0d057b
System UUID: ECE90BC7-B6B8-EA11-9DA5-2CF05D19AFF5
Boot ID: c5ca10a3-af35-4c75-8725-8d9d032e103b
Kernel Version: 3.10.0-1127.el7.x86_64
OS Image: CentOS Linux 7 (Core)
Operating System: linux
Architecture: amd64
Container Runtime Version: docker://18.9.7
Kubelet Version: v1.16.7
Kube-Proxy Version: v1.16.7
PodCIDR: 10.233.64.0/24
PodCIDRs: 10.233.64.0/24
Non-terminated Pods: (3 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
kube-system kube-apiserver-master 250m (7%) 0 (0%) 0 (0%) 0 (0%) 211d
kube-system kube-controller-manager-master 200m (5%) 0 (0%) 0 (0%) 0 (0%) 211d
kube-system kube-scheduler-master 100m (2%) 0 (0%) 0 (0%) 0 (0%) 211d
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 550m (15%) 0 (0%)
memory 0 (0%) 0 (0%)
ephemeral-storage 0 (0%) 0 (0%)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Starting 56m kube-proxy, master Starting kube-proxy.
Normal Starting 41m kube-proxy, master Starting kube-proxy.
Normal Starting 26m kube-proxy, master Starting kube-proxy.
Normal Starting 11m kube-proxy, master Starting kube-proxy.
Warning EvictionThresholdMet 3m21s (x81037 over 10d) kubelet, master Attempting to reclaim ephemeral-storagemaster节点标签是有的，现在不明白这是什么问题了，openldap的配置文件yaml文件在ks-installer吗？没有找到
