安装longhorn报错这个需要安装此插件,请参考,在安装完后pod删除后可以启动。
其中一台的longhorn-csi-plugin中的一个容器起不来报错如下,请协助看下
其中一台的longhorn-csi-plugin中的一个容器起不来报错如下,请协助看下
sealyun方式部署kubernetes v18.6 3master+3node
操作系统:centos7.6
参考以下链接部署:
https://kubesphere.com.cn/forum/d/2250-sealos-longhorn-kubesphere-v3-0-0
请协助看下
longhorn的pod启动都没问题,pvc没挂载上应该是
我遇到了相同的问题!我执行 kubectl -n kubesphere-system logs redis-ha-server-0 config-init 命令后输出如下:
kubectl get po -A
kubectl describe po -n kubesphere-system redis-ha-server-0
kubectl -n kubesphere-system logs redis-ha-server-0 config-init
造成这个问题的原因:
1、领导告知 k8s 上的系统不可用;
2、检查发现 1个master 节点磁盘超 85%且存在2000多个Evicted状态的pod
3、通过脚本删除Evicted状态的pod;
4、发现redis-ha-server中第0个存在为题,第1、2个正常;
5、通过ks面板调整redis-ha-server数量为0,ks面板被强制踢出,且无法再次登录;
6、通过kubectl edit 调整 redis-ha-server有状态副本集数量;
7、redis-ha-server-0还是有问题,且ks无法登陆;
我也遇到这个问题,k8s v1.21.11 多master集群上安装kubesphere v3.2.1,ks-apiserver和redis-ha-server-0起不来
这是在测试环境部署的,准备引入生产环境。
帮忙看下,感谢!
[root@ks-master3 heketi]# kubectl get pods -n kubesphere-system
NAME READY STATUS RESTARTS AGE
ks-apiserver-5578d6cf6f-nvgck 0/1 CrashLoopBackOff 10 33m
ks-apiserver-5578d6cf6f-qlwzp 0/1 CrashLoopBackOff 10 33m
ks-apiserver-794b78d766-mxqxr 0/1 CrashLoopBackOff 10 33m
ks-console-65f4d44d88-86lmg 1/1 Running 0 37m
ks-console-65f4d44d88-8ffkr 1/1 Running 0 37m
ks-console-65f4d44d88-t8pr8 1/1 Running 0 37m
ks-controller-manager-7598dd7db7-2dwd4 1/1 Running 0 33m
ks-controller-manager-7598dd7db7-fgdfx 1/1 Running 0 33m
ks-controller-manager-7598dd7db7-jpwhk 1/1 Running 0 32m
ks-installer-85dcfff87d-t6vsc 1/1 Running 0 48m
redis-ha-haproxy-868fdbddd4-g6zj6 1/1 Running 0 40m
redis-ha-haproxy-868fdbddd4-rbqv6 1/1 Running 0 40m
redis-ha-haproxy-868fdbddd4-rtgms 1/1 Running 0 40m
redis-ha-server-0 0/2 Init:0/1 0 40m
kubectl -n kubesphere-system describe pod redis-ha-server-0
...
status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/f4dc9c60-8742-45f5-9541-0f41c235d4d7/volumes/kubernetes.io~glusterfs/pvc-e9c083df-bb41-4580-b89d-ec8edfdcd20b --scope -- mount -t glusterfs -o auto_unmount,backup-volfile-servers=192.168.142.138:192.168.142.139:192.168.142.96,log-file=/var/lib/kubelet/plugins/kubernetes.io/glusterfs/pvc-e9c083df-bb41-4580-b89d-ec8edfdcd20b/redis-ha-server-0-glusterfs.log,log-level=ERROR 192.168.142.96:vol_5dba7beb565c1bf41dc19563531d9b97 /var/lib/kubelet/pods/f4dc9c60-8742-45f5-9541-0f41c235d4d7/volumes/kubernetes.io~glusterfs/pvc-e9c083df-bb41-4580-b89d-ec8edfdcd20b
Output: Running scope as unit run-31764.scope.
mount: unknown filesystem type 'glusterfs'
, the following error information was pulled from the glusterfs log to help diagnose this issue: could not open log file for pod redis-ha-server-0
Warning FailedMount 10m (x18 over 38m) kubelet (combined from similar events): MountVolume.SetUp failed for volume "pvc-e9c083df-bb41-4580-b89d-ec8edfdcd20b" : mount failed: mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/f4dc9c60-8742-45f5-9541-0f41c235d4d7/volumes/kubernetes.io~glusterfs/pvc-e9c083df-bb41-4580-b89d-ec8edfdcd20b --scope -- mount -t glusterfs -o auto_unmount,backup-volfile-servers=192.168.142.138:192.168.142.139:192.168.142.96,log-file=/var/lib/kubelet/plugins/kubernetes.io/glusterfs/pvc-e9c083df-bb41-4580-b89d-ec8edfdcd20b/redis-ha-server-0-glusterfs.log,log-level=ERROR 192.168.142.138:vol_5dba7beb565c1bf41dc19563531d9b97 /var/lib/kubelet/pods/f4dc9c60-8742-45f5-9541-0f41c235d4d7/volumes/kubernetes.io~glusterfs/pvc-e9c083df-bb41-4580-b89d-ec8edfdcd20b
Output: Running scope as unit run-25511.scope.
mount: unknown filesystem type 'glusterfs'
, the following error information was pulled from the glusterfs log to help diagnose this issue: could not open log file for pod redis-ha-server-0
Warning FailedMount 4m40s (x9 over 31m) kubelet Unable to attach or mount volumes: unmounted volumes=[data], unattached volumes=[config data kube-api-access-w85dk]: timed out waiting for the condition
Warning FailedMount 7s (x5 over 38m) kubelet Unable to attach or mount volumes: unmounted volumes=[data], unattached volumes=[kube-api-access-w85dk config data]: timed out waiting for the condition
kubectl -n kubesphere-system logs -f ks-apiserver-5578d6cf6f-nvgck
[root@ks-master1 kubesphere]# kubectl -n kubesphere-system logs -f ks-apiserver-5578d6cf6f-nvgck
W0325 10:19:02.245683 1 client_config.go:615] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
W0325 10:19:02.253978 1 client_config.go:615] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
W0325 10:19:02.280574 1 metricsserver.go:238] Metrics API not available.
Error: failed to connect to redis service, please check redis status, error: EOF
2022/03/25 10:19:02 failed to connect to redis service, please check redis status, error: EOF
ks-apiserver 起不来是因为redis没起来,redis没起来是因为集群里存储不正常,看起来这个集群里的存储是glusterfs,建议先把存储调好,这些服务应该就可以正常运行了。
peteradmin 你应该检查一下你的存储系统:unknown filesystem type ‘glusterfs’
大佬可以帮我看下么,我是用kk离线装的k8s和kubesphere,k8s没有问题,但是kubesphere的redis一直起不来,不知道怎么回事]
zhanglihao 同样遇到这个问题,怎么解决的
zhanglihao 我也遇到这个问题了,我的解决办法是先单节点安装,完了再增加节点