操作系统信息
例如：虚拟机，Centos7.9，10C/4G，3主3节点
Kubernetes版本信息
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.12", GitCommit:"b058e1760c79f46a834ba59bd7a3486ecf28237d", GitTreeState:"clean", BuildDate:"2022-07-13T14:59:18Z", GoVersion:"go1.16.15", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.12", GitCommit:"b058e1760c79f46a834ba59bd7a3486ecf28237d", GitTreeState:"clean", BuildDate:"2022-07-13T14:53:39Z", GoVersion:"go1.16.15", Compiler:"gc", Platform:"linux/amd64"}
容器运行时
docker version
Client:
Version: 20.10.8
API version: 1.41
Go version: go1.16.6
Git commit: 3967b7d
Built: Fri Jul 30 19:50:40 2021
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.8
API version: 1.41 (minimum version 1.12)
Go version: go1.16.6
Git commit: 75249d8
Built: Fri Jul 30 19:55:09 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.4.9
GitCommit: e25210fe30a0a703442421b0f60afac609f950a3
runc:
Version: 1.0.1
GitCommit: v1.0.1-0-g4144b638
docker-init:
Version: 0.19.0
KubeSphere版本信息
kk安装，直接安装的最新版
问题是什么
kk安装出错，我用的nfs持久化储存，在redis和ks-apiserver的pod报错
我不用nfs用默认的持久化储存能正常安装
以下日志是ks-apiserver错误信息
Name: ks-apiserver-6d746b9899-2cdfv
Namespace: kubesphere-system
Priority: 0
Node: kube-master01/192.168.137.200
Start Time: Fri, 20 Jan 2023 16:43:49 +0800
Labels: app=ks-apiserver
pod-template-hash=6d746b9899
tier=backend
Annotations: cni.projectcalico.org/containerID: ba2b4394112cd45ca60ed6b33fcefeeecb1960d06a9563ba6e13276930da2dbb
cni.projectcalico.org/podIP: 10.233.85.3/32
cni.projectcalico.org/podIPs: 10.233.85.3/32
Status: Running
IP: 10.233.85.3
IPs:
IP: 10.233.85.3
Controlled By: ReplicaSet/ks-apiserver-6d746b9899
Containers:
ks-apiserver:
Container ID: docker://8a35538796e8411580ecc8eb379648fcbfd2208dab168b95c2cbc5cb3c434dd1
Image: registry.cn-beijing.aliyuncs.com/kubesphereio/ks-apiserver:v3.3.1
Image ID: docker-pullable://registry.cn-beijing.aliyuncs.com/kubesphereio/ks-apiserver@sha256:489dfa088bbb193b0a264ec93da7fe055e5c59288b6b5229f845ae19af684bdc
Port: 9090/TCP
Host Port: 0/TCP
Command:
ks-apiserver
--logtostderr=true
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 1
Started: Fri, 20 Jan 2023 16:47:10 +0800
Finished: Fri, 20 Jan 2023 16:47:10 +0800
Ready: False
Restart Count: 3
Limits:
cpu: 1
memory: 1Gi
Requests:
cpu: 20m
memory: 100Mi
Liveness: http-get http://:9090/kapis/version delay=15s timeout=15s period=10s #success=1 #failure=8
Environment:
KUBESPHERE_REDIS_PASSWORD: <set to the key 'auth' in secret 'redis-secret'> Optional: false
Mounts:
/etc/kubesphere/ from kubesphere-config (rw)
/etc/kubesphere/ingress-controller from ks-router-config (rw)
/etc/localtime from host-time (ro)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-4xh5p (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
ks-router-config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: ks-router-config
Optional: false
kubesphere-config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: kubesphere-config
Optional: false
host-time:
Type: HostPath (bare host directory volume)
Path: /etc/localtime
HostPathType:
kube-api-access-4xh5p:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: CriticalAddonsOnly op=Exists
node-role.kubernetes.io/master:NoSchedule
node.kubernetes.io/not-ready:NoExecute op=Exists for 60s
node.kubernetes.io/unreachable:NoExecute op=Exists for 60s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 3m54s default-scheduler Successfully assigned kubesphere-system/ks-apiserver-6d746b9899-2cdfv to kube-master01
Warning FailedMount 2m50s (x8 over 3m53s) kubelet MountVolume.SetUp failed for volume "kubesphere-config" : configmap "kubesphere-config" not found
Warning FailedMount 110s kubelet Unable to attach or mount volumes: unmounted volumes=[kubesphere-config], unattached volumes=[ks-router-config kubesphere-config host-time kube-api-access-4xh5p]: timed out waiting for the condition
Normal Pulling 99s kubelet Pulling image "registry.cn-beijing.aliyuncs.com/kubesphereio/ks-apiserver:v3.3.1"
Normal Pulled 83s kubelet Successfully pulled image "registry.cn-beijing.aliyuncs.com/kubesphereio/ks-apiserver:v3.3.1" in 16.510855291s
Normal Created 64s (x3 over 83s) kubelet Created container ks-apiserver
Normal Started 64s (x3 over 83s) kubelet Started container ks-apiserver
Normal Pulled 64s (x2 over 82s) kubelet Container image "registry.cn-beijing.aliyuncs.com/kubesphereio/ks-apiserver:v3.3.1" already present on machine
Warning BackOff 46s (x6 over 81s) kubelet Back-off restarting failed container
以下错误信息是redis报错信息：
Name: redis-74f975c764-wmw66
Namespace: kubesphere-system
Priority: 0
Node: kube-master01/192.168.137.200
Start Time: Fri, 20 Jan 2023 16:43:06 +0800
Labels: app=redis
pod-template-hash=74f975c764
tier=database
version=redis-4.0
Annotations: cni.projectcalico.org/containerID: c307f196bf5e8613a800f7ebb6a40e7f1a3d988325631157d8a078d0d0837e9d
cni.projectcalico.org/podIP: 10.233.85.1/32
cni.projectcalico.org/podIPs: 10.233.85.1/32
Status: Running
IP: 10.233.85.1
IPs:
IP: 10.233.85.1
Controlled By: ReplicaSet/redis-74f975c764
Init Containers:
init:
Container ID: docker://a248ba9e1591b0faee729d5bf7e39c281b33e0a9fa787e516f13bd876a1f4378
Image: registry.cn-beijing.aliyuncs.com/kubesphereio/redis:5.0.14-alpine
Image ID: docker-pullable://registry.cn-beijing.aliyuncs.com/kubesphereio/redis@sha256:1a3c609295332f1ce603948142a132656c92a08149d7096e203058533c415b8c
Port: <none>
Host Port: <none>
Command:
sh
-c
cat /tmp/redis/redis.conf | sed "s/REDIS_PASSWORD/$KUBESPHERE_REDIS_PASSWORD/" > /data/redis.conf
State: Terminated
Reason: Completed
Exit Code: 0
Started: Fri, 20 Jan 2023 16:43:14 +0800
Finished: Fri, 20 Jan 2023 16:43:14 +0800
Ready: True
Restart Count: 0
Environment:
KUBESPHERE_REDIS_PASSWORD: <set to the key 'auth' in secret 'redis-secret'> Optional: false
Mounts:
/data from redis-pvc (rw,path="redis-data")
/tmp/redis from redis-config (ro)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-x6jzm (ro)
Containers:
redis:
Container ID: docker://be7c629e2fb90c6a0abeacd62aa2732e9f681f7789a5d09d1916dec186a5f41a
Image: registry.cn-beijing.aliyuncs.com/kubesphereio/redis:5.0.14-alpine
Image ID: docker-pullable://registry.cn-beijing.aliyuncs.com/kubesphereio/redis@sha256:1a3c609295332f1ce603948142a132656c92a08149d7096e203058533c415b8c
Port: 6379/TCP
Host Port: 0/TCP
Args:
/data/redis.conf
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 1
Started: Fri, 20 Jan 2023 16:48:52 +0800
Finished: Fri, 20 Jan 2023 16:48:52 +0800
Ready: False
Restart Count: 6
Limits:
cpu: 1
memory: 1000Mi
Requests:
cpu: 20m
memory: 100Mi
Environment: <none>
Mounts:
/data from redis-pvc (rw,path="redis-data")
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-x6jzm (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
redis-pvc:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: redis-pvc
ReadOnly: false
redis-config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: redis-configmap
Optional: false
kube-api-access-x6jzm:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: CriticalAddonsOnly op=Exists
node-role.kubernetes.io/master:NoSchedule
node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 9m57s default-scheduler 0/6 nodes are available: 6 pod has unbound immediate PersistentVolumeClaims.
Normal Scheduled 9m56s default-scheduler Successfully assigned kubesphere-system/redis-74f975c764-wmw66 to kube-master01
Normal Pulling 9m55s kubelet Pulling image "registry.cn-beijing.aliyuncs.com/kubesphereio/redis:5.0.14-alpine"
Normal Pulled 9m48s kubelet Successfully pulled image "registry.cn-beijing.aliyuncs.com/kubesphereio/redis:5.0.14-alpine" in 6.797414417s
Normal Created 9m48s kubelet Created container init
Normal Started 9m48s kubelet Started container init
Normal Started 9m2s (x4 over 9m47s) kubelet Started container redis
Normal Pulled 8m20s (x5 over 9m48s) kubelet Container image "registry.cn-beijing.aliyuncs.com/kubesphereio/redis:5.0.14-alpine" already present on machine
Normal Created 8m20s (x5 over 9m48s) kubelet Created container redis
Warning BackOff 4m47s (x25 over 9m46s) kubelet Back-off restarting failed container
kubekey的配置详情：
apiVersion: kubekey.kubesphere.io/v1alpha2
kind: Cluster
metadata:
name: sample
spec:
hosts:
- {name: kube-master01, address: 192.168.137.200, internalAddress: 192.168.137.200, user: root, password: "root"}
- {name: kube-master02, address: 192.168.137.201, internalAddress: 192.168.137.201, user: root, password: "root"}
- {name: kube-master03, address: 192.168.137.205, internalAddress: 192.168.137.205, user: root, password: "root"}
- {name: kube-node01, address: 192.168.137.202, internalAddress: 192.168.137.202, user: root, password: "root"}
- {name: kube-node02, address: 192.168.137.203, internalAddress: 192.168.137.203, user: root, password: "root"}
- {name: kube-node03, address: 192.168.137.204, internalAddress: 192.168.137.204, user: root, password: "root"}
roleGroups:
etcd:
- kube-master01
- kube-master02
- kube-master03
control-plane:
- kube-master01
- kube-master02
- kube-master03
worker:
- kube-node01
- kube-node02
- kube-node03
controlPlaneEndpoint:
## Internal loadbalancer for apiservers
internalLoadbalancer: haproxy
domain: lb.kubesphere.local
address: ""
port: 6443
kubernetes:
version: v1.22.12
clusterName: cluster.local
autoRenewCerts: true
containerManager: docker
etcd:
type: kubekey
network:
plugin: calico
kubePodsCIDR: 10.233.64.0/18
kubeServiceCIDR: 10.233.0.0/18
## multus support. https://github.com/k8snetworkplumbingwg/multus-cni
multusCNI:
enabled: false
registry:
privateRegistry: ""
namespaceOverride: ""
registryMirrors: []
insecureRegistries: []
addons:
- name: nfs-client
namespace: kube-system
sources:
chart:
name: nfs-client-provisioner
repo: https://charts.kubesphere.io/main
valuesFile: /home/nfs-conf/nfs-client.yaml
