pod/openpitrix-iam-service 起不来

Qqcloud · 2020年6月1日

zheng1 那我已经安装上了openpitrix，可所有服务都异常，查看namespace也确实是openpitrix-system，那请教接下来怎么操作？

zheng1 · 2020年6月1日

qcloud 可以先尝试清除openpitrix， https://openpitrix.io/install#%E6%B8%85%E7%90%86%E7%8E%AF%E5%A2%83
然后重新部署到新的namespace。部署完成之后再回到ks-installer启用KubeSphere内置的应用商店

Qqcloud · 2020年6月1日

zheng1 我想分析一下，通过KubeSphere2.1.1安装后在 configmap 里开启 openpitrix 就可以自动安装，可查看pods

openpitrix-system hyperpitrix-release-app-job-499h7 0/1 Init:0/2 0 45h
openpitrix-system openpitrix-hyperpitrix-deployment-6df877b655-z9tbv 0/1 Init:0/11 0 45h
openpitrix-system openpitrix-rp-kubernetes-deployment-cbb6786f7-mlg9p 0/1 Init:½ 0 45h
[root@k8sphere01 _]#

[root@k8sphere01 _]# kubectl describe pods -n openpitrix-system hyperpitrix-release-app-job-499h7
Name: hyperpitrix-release-app-job-499h7
Namespace: openpitrix-system
Priority: 0
Node: k8sphere01/192.168.108.72
Start Time: Sat, 30 May 2020 11:20:41 +0800
Labels: app=hyperpitrix
controller-uid=c40a1994-d2a2-4015-8ec1-92b62822a65d
job=hyperpitrix-release-app
job-name=hyperpitrix-release-app-job
version=v0.4.8
Annotations: <none>
Status: Pending
IP: 10.233.118.30
IPs:
IP: 10.233.118.30
Controlled By: Job/hyperpitrix-release-app-job
Init Containers:
wait-app-manager:
Container ID: docker://bee6e8209a1b9f264ad534d3ad948be0021e926898e6d54cd8334c43ef8060d1
Image: alpine:3.10.4
Image ID: docker://sha256:af341ccd2df8b0e2d67cf8dd32e087bfda4e5756ebd1c76bbf3efa0dc246590e
Port: <none>
Host Port: <none>
Command:
sh
-c
until nc -z openpitrix-app-manager.openpitrix-system.svc 9102; do echo “waiting for app-manager”; sleep 2; done;
State: Running
Started: Sat, 30 May 2020 11:20:42 +0800
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-cbzwb (ro)
wait-attachment-manager:
Container ID:

Image: alpine:3.10.4
Image ID:

Port: <none>
Host Port: <none>
Command:
sh
-c
until nc -z openpitrix-attachment-manager.openpitrix-system.svc 9122; do echo “waiting for attachment-manager”; sleep 2; done;
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-cbzwb (ro)
Containers:
hyperpitrix-release-app-job:
Container ID:

Image: openpitrix/release-app:v0.4.3
Image ID:

Port: <none>
Host Port: <none>
Command:
release-app
-path
/data/helm-pkg/
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-cbzwb (ro)
Conditions:
Type Status
Initialized False
Ready False
ContainersReady False
PodScheduled True
Volumes:
default-token-cbzwb:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-cbzwb
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events: <none>
[root@k8sphere01 _]#

openpitrix-system hyperpitrix-release-app-job-499h7 0/1 Init:0/2 0 45h
openpitrix-system openpitrix-hyperpitrix-deployment-6df877b655-z9tbv 0/1 Init:0/11 0 45h
openpitrix-system openpitrix-rp-kubernetes-deployment-cbb6786f7-mlg9p 0/1 Init:½ 0 45h
[root@k8sphere01 _]#

[root@k8sphere01 _]# kubectl describe pods -n openpitrix-system hyperpitrix-release-app-job-499h7
Name: hyperpitrix-release-app-job-499h7
Namespace: openpitrix-system
Priority: 0
Node: k8sphere01/192.168.108.72
Start Time: Sat, 30 May 2020 11:20:41 +0800
Labels: app=hyperpitrix
controller-uid=c40a1994-d2a2-4015-8ec1-92b62822a65d
job=hyperpitrix-release-app
job-name=hyperpitrix-release-app-job
version=v0.4.8
Annotations: <none>
Status: Pending
IP: 10.233.118.30
IPs:
IP: 10.233.118.30
Controlled By: Job/hyperpitrix-release-app-job
Init Containers:
wait-app-manager:
Container ID: docker://bee6e8209a1b9f264ad534d3ad948be0021e926898e6d54cd8334c43ef8060d1
Image: alpine:3.10.4
Image ID: docker://sha256:af341ccd2df8b0e2d67cf8dd32e087bfda4e5756ebd1c76bbf3efa0dc246590e
Port: <none>
Host Port: <none>
Command:
sh
-c
until nc -z openpitrix-app-manager.openpitrix-system.svc 9102; do echo “waiting for app-manager”; sleep 2; done;
State: Running
Started: Sat, 30 May 2020 11:20:42 +0800
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-cbzwb (ro)
wait-attachment-manager:
Container ID:

Image: alpine:3.10.4
Image ID:

Port: <none>
Host Port: <none>
Command:
sh
-c
until nc -z openpitrix-attachment-manager.openpitrix-system.svc 9122; do echo “waiting for attachment-manager”; sleep 2; done;
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-cbzwb (ro)
Containers:
hyperpitrix-release-app-job:
Container ID:

Image: openpitrix/release-app:v0.4.3
Image ID:

Port: <none>
Host Port: <none>
Command:
release-app
-path
/data/helm-pkg/
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-cbzwb (ro)
Conditions:
Type Status
Initialized False
Ready False
ContainersReady False
PodScheduled True
Volumes:
default-token-cbzwb:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-cbzwb
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events: <none>
[root@k8sphere01 _]#

这种问题怎么去分析原因，并解决

zheng1 · 2020年6月1日

qcloud describe一下openpitrix-hyperpitrix-deployment-6df877b655-z9tbv 这个pod，需要知道它为什么没起来

Qqcloud · 2020年6月1日

[root@k8sphere01 conf]# kubectl describe pods -n openpitrix-system openpitrix-hyperpitrix-deployment-75d7664679-v7lgf
Name: openpitrix-hyperpitrix-deployment-75d7664679-v7lgf
Namespace: openpitrix-system
Priority: 0
Node: k8sphere01/192.168.108.72
Start Time: Mon, 01 Jun 2020 10:15:34 +0800
Labels: app=openpitrix
component=openpitrix-hyperpitrix
pod-template-hash=75d7664679
version=v0.4.8
Annotations: kubectl.kubernetes.io/restartedAt: 2020-06-01T10:15:33+08:00
Status: Pending
IP: 10.233.118.35
IPs:
IP: 10.233.118.35
Controlled By: ReplicaSet/openpitrix-hyperpitrix-deployment-75d7664679
Init Containers:
wait-mysql:
Container ID: docker://23a1708902c267442d8f0f8f79a9f4385284a1fad7a6d3edb0d3393df804a28e
Image: alpine:3.10.4
Image ID: docker://sha256:af341ccd2df8b0e2d67cf8dd32e087bfda4e5756ebd1c76bbf3efa0dc246590e
Port: <none>
Host Port: <none>
Command:
sh
-c
until nc -z mysql.kubesphere-system.svc 3306; do echo “waiting for mysql”; sleep 2; done;
State: Running
Started: Mon, 01 Jun 2020 10:15:39 +0800
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-cbzwb (ro)
wait-etcd:
Container ID:

Image: alpine:3.10.4
Image ID:

Port: <none>
Host Port: <none>
Command:
sh
-c
until nc -z etcd.kubesphere-system.svc 2379; do echo “waiting for etcd”; sleep 2; done;
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-cbzwb (ro)
openpitrix-db-init:
Container ID:

Image: openpitrix/openpitrix:flyway-v0.4.8
Image ID:

Port: <none>
Host Port: <none>
Command:
/flyway/sql/ddl/ddl_init.sh
-hmysql.kubesphere-system.svc
-uroot
–connect-timeout=5
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment:
PASSWORD: password
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-cbzwb (ro)
openpitrix-app-db-ctrl:
Container ID:

Image: openpitrix/openpitrix:flyway-v0.4.8
Image ID:

Port: <none>
Host Port: <none>
Command:
/flyway/flyway
-X
-url=jdbc:mysql://mysql.kubesphere-system.svc/app
-user=root
-validateOnMigrate=false
-locations=filesystem:/flyway/sql/app
migrate
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment:
FLYWAY_PASSWORD: password
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-cbzwb (ro)
openpitrix-attachment-db-ctrl:
Container ID:

Image: openpitrix/openpitrix:flyway-v0.4.8
Image ID:

Port: <none>
Host Port: <none>
Command:
/flyway/flyway
-X
-url=jdbc:mysql://mysql.kubesphere-system.svc/attachment
-user=root
-validateOnMigrate=false
-locations=filesystem:/flyway/sql/attachment
migrate
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment:
FLYWAY_PASSWORD: password
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-cbzwb (ro)
openpitrix-cluster-db-ctrl:
Container ID:

Image: openpitrix/openpitrix:flyway-v0.4.8
Image ID:

Port: <none>
Host Port: <none>
Command:
/flyway/flyway
-X
-url=jdbc:mysql://mysql.kubesphere-system.svc/cluster
-user=root
-validateOnMigrate=false
-locations=filesystem:/flyway/sql/cluster
migrate
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment:
FLYWAY_PASSWORD: password
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-cbzwb (ro)
openpitrix-isv-db-ctrl:
Container ID:

Image: openpitrix/openpitrix:flyway-v0.4.8
Image ID:

Port: <none>
Host Port: <none>
Command:
/flyway/flyway
-X
-url=jdbc:mysql://mysql.kubesphere-system.svc/isv
-user=root
-validateOnMigrate=false
-locations=filesystem:/flyway/sql/isv
migrate
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment:
FLYWAY_PASSWORD: password
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-cbzwb (ro)
openpitrix-job-db-ctrl:
Container ID:

Image: openpitrix/openpitrix:flyway-v0.4.8
Image ID:

Port: <none>
Host Port: <none>
Command:
/flyway/flyway
-X
-url=jdbc:mysql://mysql.kubesphere-system.svc/job
-user=root
-validateOnMigrate=false
-locations=filesystem:/flyway/sql/job
migrate
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment:
FLYWAY_PASSWORD: password
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-cbzwb (ro)
openpitrix-repo-db-ctrl:
Container ID:

Image: openpitrix/openpitrix:flyway-v0.4.8
Image ID:

Port: <none>
Host Port: <none>
Command:
/flyway/flyway
-X
-url=jdbc:mysql://mysql.kubesphere-system.svc/repo
-user=root
-validateOnMigrate=false
-locations=filesystem:/flyway/sql/repo
migrate
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment:
FLYWAY_PASSWORD: password
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-cbzwb (ro)
openpitrix-runtime-db-ctrl:
Container ID:

Image: openpitrix/openpitrix:flyway-v0.4.8
Image ID:

Port: <none>
Host Port: <none>
Command:
/flyway/flyway
-X
-url=jdbc:mysql://mysql.kubesphere-system.svc/runtime
-user=root
-validateOnMigrate=false
-locations=filesystem:/flyway/sql/runtime
migrate
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment:
FLYWAY_PASSWORD: password
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-cbzwb (ro)
openpitrix-task-db-ctrl:
Container ID:

Image: openpitrix/openpitrix:flyway-v0.4.8
Image ID:

Port: <none>
Host Port: <none>
Command:
/flyway/flyway
-X
-url=jdbc:mysql://mysql.kubesphere-system.svc/task
-user=root
-validateOnMigrate=false
-locations=filesystem:/flyway/sql/task
migrate
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment:
FLYWAY_PASSWORD: password
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-cbzwb (ro)
Containers:
hyperpitrix:
Container ID:

Image: openpitrix/openpitrix:v0.4.8
Image ID:

Ports: 9100/TCP, 9102/TCP, 9108/TCP, 9122/TCP, 9104/TCP, 9118/TCP, 9106/TCP, 9101/TCP, 9108/TCP, 9121/TCP, 9103/TCP, 9107/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP
Command:
hyperpitrix
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Limits:
cpu: 500m
memory: 500Mi
Requests:
cpu: 100m
memory: 100Mi
Environment:
OPENPITRIX_GRPC_SHOW_ERROR_CAUSE: true
OPENPITRIX_LOG_LEVEL: debug
OPENPITRIX_ETCD_ENDPOINTS: etcd.kubesphere-system.svc:2379
OPENPITRIX_MYSQL_HOST: mysql.kubesphere-system.svc
OPENPITRIX_ATTACHMENT_ENDPOINT: http://minio.kubesphere-system.svc:9000
OPENPITRIX_ATTACHMENT_BUCKET_NAME: openpitrix-attachment
OPENPITRIX_MYSQL_PASSWORD: password
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-cbzwb (ro)
Conditions:
Type Status
Initialized False
Ready False
ContainersReady False
PodScheduled True
Volumes:
default-token-cbzwb:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-cbzwb
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message

Normal Scheduled <unknown> default-scheduler Successfully assigned openpitrix-system/openpitrix-hyperpitrix-deployment-75d7664679-v7lgf to k8sphere01
Normal Pulled 23m kubelet, k8sphere01 Container image “alpine:3.10.4” already present on machine
Normal Created 23m kubelet, k8sphere01 Created container wait-mysql
Normal Started 23m kubelet, k8sphere01 Started container wait-mysql
[root@k8sphere01 conf]#

Qqcloud · 2020年6月1日

zheng1 一直在等待 wait-mysql

zheng1 · 2020年6月1日

qcloud 检查下kubesphere这个namespace下面的mysql是否正常

Qqcloud · 2020年6月1日

zheng1 怎么检查？能远程看一下不？

Qqcloud · 2020年6月1日

qcloud 已发邮件

Qqcloud · 2020年6月1日

zheng1 谢谢已解决该问题。确实是mysql的问题，但因其状态是running，就忽略掉这点而没有去看详细信息。

DDD123 · 2021年4月2日

qcloud

你好我这也出现不能访问mysql，是怎么处理的？

pod/openpitrix-iam-service 起不来

Qqcloud

zheng1K零S

Qqcloud

zheng1K零S

Qqcloud

Qqcloud

zheng1K零S

Qqcloud

Qqcloud

Qqcloud

DDD123