• 安装部署
  • 镜像拉取异常-中国地区进行kubespherev.3.4.1部署异常

创建部署问题时,请参考下面模板,你提供的信息越多,越容易及时获得解答。
你只花一分钟创建的问题,不能指望别人花上半个小时给你解答。
发帖前请点击 发表主题 右边的 预览(👀) 按钮,确保帖子格式正确。

操作系统信息
虚拟机 ,Ubuntu22.04,4C/8G

Kubernetes版本信息
v1.23 三个节点。

容器运行时
Docker version 24.0.6, build ed223bc

KubeSphere版本信息
v3.4.1。在线安装。全套安装。

问题是什么
在中国地区,无法访问Google的场景,参考官网文档操作时,遇到以下问题,

  1. 部分镜像直接获取中国地区资源,部分pod的镜像强制拉取国外资源,请问该如何规避强制拉取国外资源的问题。
  2. 本地有镜像时,部分pod依旧强制拉取国外资源,导致POD创建失败,该如何排查处理。
root@master:~# kubectl get pods -A
NAMESPACE           NAME                                           READY   STATUS         RESTARTS        AGE
kube-system         calico-kube-controllers-67fbf89557-whdmf       1/1     Running        0               2m35s
kube-system         calico-node-hrwzl                              1/1     Running        0               2m35s
kube-system         calico-node-qsx6d                              1/1     Running        0               2m35s
kube-system         calico-node-sjsn2                              1/1     Running        0               2m35s
kube-system         coredns-757cd945b-9kpvt                        1/1     Running        0               2m43s
kube-system         coredns-757cd945b-n4lfm                        1/1     Running        0               2m43s
kube-system         kube-apiserver-master                          1/1     Running        0               2m56s
kube-system         kube-controller-manager-master                 1/1     Running        1 (2m12s ago)   2m56s
kube-system         kube-proxy-qr5tl                               1/1     Running        0               2m40s
kube-system         kube-proxy-rhshr                               1/1     Running        0               2m43s
kube-system         kube-proxy-trfjd                               1/1     Running        0               2m40s
kube-system         kube-scheduler-master                          1/1     Running        0               2m56s
kube-system         nodelocaldns-fndsw                             1/1     Running        0               2m40s
kube-system         nodelocaldns-gvzhs                             1/1     Running        0               2m40s
kube-system         nodelocaldns-s554r                             1/1     Running        0               2m43s
kube-system         openebs-localpv-provisioner-7bbcf865cd-mwhd9   1/1     Running        0               2m33s
kube-system         snapshot-controller-0                          0/1     ErrImagePull   0               28s
kubesphere-system   ks-installer-ddbcf44f8-pdmj8                   1/1     Running        0               101s
root@master:~# kubectl describe pod -n kube-system snapshot-controller-0
Name:         snapshot-controller-0
Namespace:    kube-system
Priority:     0
Node:         node1/192.168.209.134
Start Time:   Tue, 06 May 2025 15:17:44 +0000
Labels:       app=snapshot-controller
              controller-revision-hash=snapshot-controller-7d87fc7c78
              statefulset.kubernetes.io/pod-name=snapshot-controller-0
Annotations:  cni.projectcalico.org/containerID: bec7a6010bdb5ec199ec14a71b4ba18e7c6ca2571414933a07db754289bbc1db
              cni.projectcalico.org/podIP: 10.233.90.3/32
              cni.projectcalico.org/podIPs: 10.233.90.3/32
Status:       Pending
IP:           10.233.90.3
IPs:
  IP:           10.233.90.3
Controlled By:  StatefulSet/snapshot-controller
Containers:
  snapshot-controller:
    Container ID:
    Image:         csiplugin/snapshot-controller:v4.0.0
    Image ID:
    Port:          <none>
    Host Port:     <none>
    Args:
      --v=5
      --leader-election=false
    State:          Waiting
      Reason:       ImagePullBackOff
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-xg6k9 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  kube-api-access-xg6k9:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  47s                default-scheduler  Successfully assigned kube-system/snapshot-controller-0 to node1
  Normal   BackOff    31s                kubelet            Back-off pulling image "csiplugin/snapshot-controller:v4.0.0"
  Warning  Failed     31s                kubelet            Error: ImagePullBackOff
  Normal   Pulling    17s (x2 over 46s)  kubelet            Pulling image "csiplugin/snapshot-controller:v4.0.0"
  Warning  Failed     2s (x2 over 31s)   kubelet            Failed to pull image "csiplugin/snapshot-controller:v4.0.0": rpc error: code = Unknown desc = Error response from daemon: Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
  Warning  Failed     2s (x2 over 31s)   kubelet            Error: ErrImagePull

不止一个pod出现镜像异常

root@master:~# kubectl get pods -A
NAMESPACE                      NAME                                             READY   STATUS              RESTARTS        AGE
kube-system                    calico-kube-controllers-67fbf89557-whdmf         1/1     Running             0               7m53s
kube-system                    calico-node-hrwzl                                1/1     Running             0               7m53s
kube-system                    calico-node-qsx6d                                1/1     Running             0               7m53s
kube-system                    calico-node-sjsn2                                1/1     Running             0               7m53s
kube-system                    coredns-757cd945b-9kpvt                          1/1     Running             0               8m1s
kube-system                    coredns-757cd945b-n4lfm                          1/1     Running             0               8m1s
kube-system                    kube-apiserver-master                            1/1     Running             0               8m14s
kube-system                    kube-controller-manager-master                   1/1     Running             1 (7m30s ago)   8m14s
kube-system                    kube-proxy-qr5tl                                 1/1     Running             0               7m58s
kube-system                    kube-proxy-rhshr                                 1/1     Running             0               8m1s
kube-system                    kube-proxy-trfjd                                 1/1     Running             0               7m58s
kube-system                    kube-scheduler-master                            1/1     Running             0               8m14s
kube-system                    nodelocaldns-fndsw                               1/1     Running             0               7m58s
kube-system                    nodelocaldns-gvzhs                               1/1     Running             0               7m58s
kube-system                    nodelocaldns-s554r                               1/1     Running             0               8m1s
kube-system                    openebs-localpv-provisioner-7bbcf865cd-mwhd9     1/1     Running             0               7m51s
kube-system                    snapshot-controller-0                            0/1     ImagePullBackOff    0               5m46s
kubesphere-controls-system     default-http-backend-696d6bf54f-gwk6b            0/1     ImagePullBackOff    0               4m51s
kubesphere-monitoring-system   kube-state-metrics-774ff789cf-brlgq              0/3     ErrImagePull        0               3m48s
kubesphere-monitoring-system   node-exporter-fkt24                              0/2     ImagePullBackOff    0               3m48s
kubesphere-monitoring-system   node-exporter-g7xwd                              0/2     ErrImagePull        0               3m48s
kubesphere-monitoring-system   node-exporter-kv47l                              0/2     ImagePullBackOff    0               3m48s
kubesphere-monitoring-system   notification-manager-operator-85d67fdc46-4vlcn   0/2     ErrImagePull        0               3m24s
kubesphere-monitoring-system   notification-manager-post-install-sjgbr          0/1     ImagePullBackOff    0               3m23s
kubesphere-monitoring-system   prometheus-operator-57c78bd7fb-hcj6k             0/2     ImagePullBackOff    0               3m49s
kubesphere-system              ks-apiserver-76bf8bb556-7c4xs                    0/1     ContainerCreating   0               4m51s
kubesphere-system              ks-console-768dbcdf9c-q57sg                      0/1     ImagePullBackOff    0               4m51s
kubesphere-system              ks-controller-manager-866bf75746-lsvgx           0/1     ContainerCreating   0               4m51s
kubesphere-system              ks-installer-ddbcf44f8-pdmj8                     1/1     Running             0               6m59s

在前三次部署时,还有一个minio的镜像拉取异常。我执行同样的操作,当前是第五次部署,依旧是部署异常。

当前部署出现新异常

PLAY RECAP *********************************************************************
localhost                  : ok=30   changed=22   unreachable=0    failed=0    skipped=17   rescued=0    ignored=0
Start installing monitoring
Start installing multicluster
Start installing openpitrix
Start installing network
**************************************************
Waiting for all tasks to be completed ...
task network status is successful  (1/4)
task openpitrix status is successful  (2/4)
task multicluster status is successful  (3/4)
task monitoring status is failed  (4/4)
**************************************************
Collecting installation results ...
root@master:~# kubectl get pods -A
NAMESPACE                      NAME                                             READY   STATUS             RESTARTS      AGE
kube-system                    calico-kube-controllers-67fbf89557-whdmf         1/1     Running            0             24m
kube-system                    calico-node-hrwzl                                1/1     Running            0             24m
kube-system                    calico-node-qsx6d                                1/1     Running            0             24m
kube-system                    calico-node-sjsn2                                1/1     Running            0             24m
kube-system                    coredns-757cd945b-9kpvt                          1/1     Running            0             25m
kube-system                    coredns-757cd945b-n4lfm                          1/1     Running            0             25m
kube-system                    kube-apiserver-master                            1/1     Running            0             25m
kube-system                    kube-controller-manager-master                   1/1     Running            1 (24m ago)   25m
kube-system                    kube-proxy-qr5tl                                 1/1     Running            0             25m
kube-system                    kube-proxy-rhshr                                 1/1     Running            0             25m
kube-system                    kube-proxy-trfjd                                 1/1     Running            0             25m
kube-system                    kube-scheduler-master                            1/1     Running            0             25m
kube-system                    nodelocaldns-fndsw                               1/1     Running            0             25m
kube-system                    nodelocaldns-gvzhs                               1/1     Running            0             25m
kube-system                    nodelocaldns-s554r                               1/1     Running            0             25m
kube-system                    openebs-localpv-provisioner-7bbcf865cd-mwhd9     1/1     Running            0             24m
kube-system                    snapshot-controller-0                            0/1     ImagePullBackOff   0             22m
kubesphere-controls-system     default-http-backend-696d6bf54f-gwk6b            0/1     ImagePullBackOff   0             21m
kubesphere-monitoring-system   kube-state-metrics-774ff789cf-brlgq              0/3     ImagePullBackOff   0             20m
kubesphere-monitoring-system   node-exporter-fkt24                              0/2     ImagePullBackOff   0             20m
kubesphere-monitoring-system   node-exporter-g7xwd                              0/2     ImagePullBackOff   0             20m
kubesphere-monitoring-system   node-exporter-kv47l                              0/2     ImagePullBackOff   0             20m
kubesphere-monitoring-system   notification-manager-operator-85d67fdc46-4vlcn   0/2     ImagePullBackOff   0             20m
kubesphere-monitoring-system   prometheus-operator-57c78bd7fb-hcj6k             0/2     ImagePullBackOff   0             20m
kubesphere-system              ks-apiserver-76bf8bb556-7c4xs                    0/1     ImagePullBackOff   0             21m
kubesphere-system              ks-console-768dbcdf9c-q57sg                      0/1     ImagePullBackOff   0             21m
kubesphere-system              ks-controller-manager-866bf75746-lsvgx           0/1     ImagePullBackOff   0             21m
kubesphere-system              ks-installer-ddbcf44f8-pdmj8                     1/1     Running            0             24m
root@master:~#

报错1,

root@master:~# kubectl describe pod -n kubesphere-system  ks-controller-manager-866bf75746-lsvgx
Name:         ks-controller-manager-866bf75746-lsvgx
Namespace:    kubesphere-system
Priority:     0
Node:         master/192.168.209.133
Start Time:   Tue, 06 May 2025 15:18:39 +0000
Labels:       app=ks-controller-manager
              pod-template-hash=866bf75746
              tier=backend
Annotations:  cni.projectcalico.org/containerID: 5ec541e30b9cb909f512f3c1cd240687f381622ec5ddd7e121c4f82e5026e90b
              cni.projectcalico.org/podIP: 10.233.70.3/32
              cni.projectcalico.org/podIPs: 10.233.70.3/32
Status:       Pending
IP:           10.233.70.3
IPs:
  IP:           10.233.70.3
Controlled By:  ReplicaSet/ks-controller-manager-866bf75746
Containers:
  ks-controller-manager:
    Container ID:
    Image:         kubesphere/ks-controller-manager:v3.4.1
    Image ID:
    Ports:         8080/TCP, 8443/TCP
    Host Ports:    0/TCP, 0/TCP
    Command:
      controller-manager
      --logtostderr=true
      --leader-elect=true
    State:          Waiting
      Reason:       ImagePullBackOff
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     1
      memory:  1000Mi
    Requests:
      cpu:        30m
      memory:     50Mi
    Environment:  <none>
    Mounts:
      /etc/kubesphere/ from kubesphere-config (rw)
      /etc/localtime from host-time (ro)
      /tmp/k8s-webhook-server/serving-certs from webhook-secret (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-fkzb2 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  kubesphere-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      kubesphere-config
    Optional:  false
  webhook-secret:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  ks-controller-manager-webhook-cert
    Optional:    false
  host-time:
    Type:          HostPath (bare host directory volume)
    Path:          /etc/localtime
    HostPathType:
  kube-api-access-fkzb2:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 CriticalAddonsOnly op=Exists
                             node-role.kubernetes.io/master:NoSchedule
                             node.kubernetes.io/not-ready:NoExecute op=Exists for 60s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 60s
Events:
  Type     Reason       Age                   From               Message
  ----     ------       ----                  ----               -------
  Normal   Scheduled    24m                   default-scheduler  Successfully assigned kubesphere-system/ks-controller-manager-866bf75746-lsvgx to master
  Warning  FailedMount  8m23s (x2 over 19m)   kubelet            Unable to attach or mount volumes: unmounted volumes=[kubesphere-config], unattached volumes=[host-time kube-api-access-fkzb2 kubesphere-config webhook-secret]: timed out waiting for the condition
  Warning  FailedMount  7m41s (x16 over 24m)  kubelet            MountVolume.SetUp failed for volume "kubesphere-config" : configmap "kubesphere-config" not found
  Warning  FailedMount  6m6s (x6 over 22m)    kubelet            Unable to attach or mount volumes: unmounted volumes=[kubesphere-config], unattached volumes=[kubesphere-config webhook-secret host-time kube-api-access-fkzb2]: timed out waiting for the condition
  Warning  Failed       5m8s                  kubelet            Failed to pull image "kubesphere/ks-controller-manager:v3.4.1": rpc error: code = Unknown desc = Error response from daemon: Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
  Warning  Failed       5m8s                  kubelet            Error: ErrImagePull
  Normal   BackOff      5m8s                  kubelet            Back-off pulling image "kubesphere/ks-controller-manager:v3.4.1"
  Normal   Pulling      4m (x3 over 5m38s)    kubelet            Pulling image "kubesphere/ks-controller-manager:v3.4.1"

报错2,

root@master:~# kubectl describe pod -n kubesphere-system   ks-console-768dbcdf9c-q57sg
Name:         ks-console-768dbcdf9c-q57sg
Namespace:    kubesphere-system
Priority:     0
Node:         master/192.168.209.133
Start Time:   Tue, 06 May 2025 15:18:39 +0000
Labels:       app=ks-console
              pod-template-hash=768dbcdf9c
              tier=frontend
Annotations:  cni.projectcalico.org/containerID: 230231ca77787297c3f265b4f7ee63ef2364e6ec95af7b98dc2ba16f856792f5
              cni.projectcalico.org/podIP: 10.233.70.1/32
              cni.projectcalico.org/podIPs: 10.233.70.1/32
Status:       Pending
IP:           10.233.70.1
IPs:
  IP:           10.233.70.1
Controlled By:  ReplicaSet/ks-console-768dbcdf9c
Containers:
  ks-console:
    Container ID:
    Image:          kubesphere/ks-console:v3.4.1
    Image ID:
    Port:           <none>
    Host Port:      <none>
    State:          Waiting
      Reason:       ImagePullBackOff
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     1
      memory:  1Gi
    Requests:
      cpu:        20m
      memory:     100Mi
    Liveness:     tcp-socket :8000 delay=15s timeout=15s period=10s #success=1 #failure=8
    Environment:  <none>
    Mounts:
      /etc/localtime from host-time (ro)
      /opt/kubesphere/console/server/local_config.yaml from ks-console-config (rw,path="local_config.yaml")
      /opt/kubesphere/console/server/sample from sample-bookinfo (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-vwsvq (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  ks-console-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      ks-console-config
    Optional:  false
  sample-bookinfo:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      sample-bookinfo
    Optional:  false
  host-time:
    Type:          HostPath (bare host directory volume)
    Path:          /etc/localtime
    HostPathType:
  kube-api-access-vwsvq:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 CriticalAddonsOnly op=Exists
                             node-role.kubernetes.io/master:NoSchedule
                             node.kubernetes.io/not-ready:NoExecute op=Exists for 60s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 60s
Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  25m                default-scheduler  Successfully assigned kubesphere-system/ks-console-768dbcdf9c-q57sg to master
  Warning  Failed     23m (x3 over 24m)  kubelet            Failed to pull image "kubesphere/ks-console:v3.4.1": rpc error: code = Unknown desc = Error response from daemon: Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
  Normal   Pulling    23m (x4 over 25m)  kubelet            Pulling image "kubesphere/ks-console:v3.4.1"
  Warning  Failed     22m (x4 over 24m)  kubelet            Error: ErrImagePull
  Warning  Failed     22m                kubelet            Failed to pull image "kubesphere/ks-console:v3.4.1": rpc error: code = Unknown desc = Error response from daemon: Get "https://registry-1.docker.io/v2/": dial tcp 103.230.123.190:443: connect: connection refused
  Warning  Failed     22m (x6 over 24m)  kubelet            Error: ImagePullBackOff
  Normal   BackOff    8s (x97 over 24m)  kubelet            Back-off pulling image "kubesphere/ks-console:v3.4.1"

报错3,

root@master:~# kubectl describe pod -n kubesphere-system   ks-console-768dbcdf9c-q57sg
Name:         ks-console-768dbcdf9c-q57sg
Namespace:    kubesphere-system
Priority:     0
Node:         master/192.168.209.133
Start Time:   Tue, 06 May 2025 15:18:39 +0000
Labels:       app=ks-console
              pod-template-hash=768dbcdf9c
              tier=frontend
Annotations:  cni.projectcalico.org/containerID: 230231ca77787297c3f265b4f7ee63ef2364e6ec95af7b98dc2ba16f856792f5
              cni.projectcalico.org/podIP: 10.233.70.1/32
              cni.projectcalico.org/podIPs: 10.233.70.1/32
Status:       Pending
IP:           10.233.70.1
IPs:
  IP:           10.233.70.1
Controlled By:  ReplicaSet/ks-console-768dbcdf9c
Containers:
  ks-console:
    Container ID:
    Image:          kubesphere/ks-console:v3.4.1
    Image ID:
    Port:           <none>
    Host Port:      <none>
    State:          Waiting
      Reason:       ImagePullBackOff
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     1
      memory:  1Gi
    Requests:
      cpu:        20m
      memory:     100Mi
    Liveness:     tcp-socket :8000 delay=15s timeout=15s period=10s #success=1 #failure=8
    Environment:  <none>
    Mounts:
      /etc/localtime from host-time (ro)
      /opt/kubesphere/console/server/local_config.yaml from ks-console-config (rw,path="local_config.yaml")
      /opt/kubesphere/console/server/sample from sample-bookinfo (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-vwsvq (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  ks-console-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      ks-console-config
    Optional:  false
  sample-bookinfo:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      sample-bookinfo
    Optional:  false
  host-time:
    Type:          HostPath (bare host directory volume)
    Path:          /etc/localtime
    HostPathType:
  kube-api-access-vwsvq:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 CriticalAddonsOnly op=Exists
                             node-role.kubernetes.io/master:NoSchedule
                             node.kubernetes.io/not-ready:NoExecute op=Exists for 60s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 60s
Events:
  Type     Reason     Age                 From               Message
  ----     ------     ----                ----               -------
  Normal   Scheduled  25m                 default-scheduler  Successfully assigned kubesphere-system/ks-console-768dbcdf9c-q57sg to master
  Warning  Failed     24m (x3 over 25m)   kubelet            Failed to pull image "kubesphere/ks-console:v3.4.1": rpc error: code = Unknown desc = Error response from daemon: Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
  Normal   Pulling    23m (x4 over 25m)   kubelet            Pulling image "kubesphere/ks-console:v3.4.1"
  Warning  Failed     23m (x4 over 25m)   kubelet            Error: ErrImagePull
  Warning  Failed     23m                 kubelet            Failed to pull image "kubesphere/ks-console:v3.4.1": rpc error: code = Unknown desc = Error response from daemon: Get "https://registry-1.docker.io/v2/": dial tcp 103.230.123.190:443: connect: connection refused
  Warning  Failed     23m (x6 over 25m)   kubelet            Error: ImagePullBackOff
  Normal   BackOff    47s (x97 over 25m)  kubelet            Back-off pulling image "kubesphere/ks-console:v3.4.1"

报错4,

root@master:~# kubectl describe pod -n kubesphere-monitoring-system prometheus-operator-57c78bd7fb-hcj6k
Name:         prometheus-operator-57c78bd7fb-hcj6k
Namespace:    kubesphere-monitoring-system
Priority:     0
Node:         node2/192.168.209.135
Start Time:   Tue, 06 May 2025 15:19:41 +0000
Labels:       app.kubernetes.io/component=controller
              app.kubernetes.io/name=prometheus-operator
              app.kubernetes.io/part-of=kube-prometheus
              app.kubernetes.io/version=v0.55.1
              pod-template-hash=57c78bd7fb
Annotations:  cni.projectcalico.org/containerID: edfd95076d185b87cb9d22ce97009b33234b5d7651eed50befd727dd8087459f
              cni.projectcalico.org/podIP: 10.233.96.5/32
              cni.projectcalico.org/podIPs: 10.233.96.5/32
              kubectl.kubernetes.io/default-container: prometheus-operator
Status:       Pending
IP:           10.233.96.5
IPs:
  IP:           10.233.96.5
Controlled By:  ReplicaSet/prometheus-operator-57c78bd7fb
Containers:
  prometheus-operator:
    Container ID:
    Image:         kubesphere/prometheus-operator:v0.55.1
    Image ID:
    Port:          8080/TCP
    Host Port:     0/TCP
    Args:
      --kubelet-service=kube-system/kubelet
      --prometheus-config-reloader=kubesphere/prometheus-config-reloader:v0.55.1
    State:          Waiting
      Reason:       ImagePullBackOff
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     200m
      memory:  200Mi
    Requests:
      cpu:        100m
      memory:     100Mi
    Environment:  <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-5mlp5 (ro)
  kube-rbac-proxy:
    Container ID:
    Image:         kubesphere/kube-rbac-proxy:v0.11.0
    Image ID:
    Port:          8443/TCP
    Host Port:     0/TCP
    Args:
      --logtostderr
      --secure-listen-address=:8443
      --tls-cipher-suites=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
      --upstream=http://127.0.0.1:8080/
    State:          Waiting
      Reason:       ImagePullBackOff
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     1
      memory:  100Mi
    Requests:
      cpu:        10m
      memory:     20Mi
    Environment:  <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-5mlp5 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  kube-api-access-5mlp5:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              kubernetes.io/os=linux
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason          Age                 From               Message
  ----     ------          ----                ----               -------
  Normal   Scheduled       25m                 default-scheduler  Successfully assigned kubesphere-monitoring-system/prometheus-operator-57c78bd7fb-hcj6k to node2
  Normal   SandboxChanged  24m                 kubelet            Pod sandbox changed, it will be killed and re-created.
  Warning  Failed          23m (x3 over 25m)   kubelet            Error: ErrImagePull
  Normal   Pulling         23m (x3 over 25m)   kubelet            Pulling image "kubesphere/kube-rbac-proxy:v0.11.0"
  Warning  Failed          22m (x3 over 24m)   kubelet            Failed to pull image "kubesphere/kube-rbac-proxy:v0.11.0": rpc error: code = Unknown desc = Error response from daemon: Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
  Warning  Failed          22m (x3 over 24m)   kubelet            Error: ErrImagePull
  Warning  Failed          22m                 kubelet            Error: ImagePullBackOff
  Normal   BackOff         22m                 kubelet            Back-off pulling image "kubesphere/kube-rbac-proxy:v0.11.0"
  Warning  Failed          22m                 kubelet            Error: ImagePullBackOff
  Normal   Pulling         22m (x4 over 25m)   kubelet            Pulling image "kubesphere/prometheus-operator:v0.55.1"
  Warning  Failed          22m (x4 over 25m)   kubelet            Failed to pull image "kubesphere/prometheus-operator:v0.55.1": rpc error: code = Unknown desc = Error response from daemon: Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
  Normal   BackOff         33s (x85 over 22m)  kubelet            Back-off pulling image "kubesphere/prometheus-operator:v0.55.1"

报错5,

root@master:~# kubectl describe pod -n kubesphere-controls-system default-http-backend-696d6bf54f-gwk6b
Name:         default-http-backend-696d6bf54f-gwk6b
Namespace:    kubesphere-controls-system
Priority:     0
Node:         node1/192.168.209.134
Start Time:   Tue, 06 May 2025 15:18:39 +0000
Labels:       app=kubesphere
              component=kubesphere-router
              pod-template-hash=696d6bf54f
Annotations:  cni.projectcalico.org/containerID: dfff8e9c166a06a6b0dd01384e89ad61191d6e5446846eb09994b226ec633cb6
              cni.projectcalico.org/podIP: 10.233.90.4/32
              cni.projectcalico.org/podIPs: 10.233.90.4/32
Status:       Pending
IP:           10.233.90.4
IPs:
  IP:           10.233.90.4
Controlled By:  ReplicaSet/default-http-backend-696d6bf54f
Containers:
  default-http-backend:
    Container ID:
    Image:          mirrorgooglecontainers/defaultbackend-amd64:1.4
    Image ID:
    Port:           8080/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       ImagePullBackOff
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     10m
      memory:  20Mi
    Requests:
      cpu:        10m
      memory:     20Mi
    Liveness:     http-get http://:8080/healthz delay=30s timeout=5s period=10s #success=1 #failure=3
    Environment:  <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-qvccj (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  kube-api-access-qvccj:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Guaranteed
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age                   From               Message
  ----     ------     ----                  ----               -------
  Normal   Scheduled  27m                   default-scheduler  Successfully assigned kubesphere-controls-system/default-http-backend-696d6bf54f-gwk6b to node1
  Warning  Failed     25m                   kubelet            Failed to pull image "mirrorgooglecontainers/defaultbackend-amd64:1.4": rpc error: code = Unknown desc = Error response from daemon: Get "https://registry-1.docker.io/v2/": dial tcp 67.228.235.93:443: i/o timeout
  Warning  Failed     23m (x4 over 27m)     kubelet            Error: ErrImagePull
  Warning  Failed     23m (x6 over 27m)     kubelet            Error: ImagePullBackOff
  Warning  Failed     12m (x5 over 27m)     kubelet            Failed to pull image "mirrorgooglecontainers/defaultbackend-amd64:1.4": rpc error: code = Unknown desc = Error response from daemon: Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
  Normal   Pulling    7m19s (x8 over 27m)   kubelet            Pulling image "mirrorgooglecontainers/defaultbackend-amd64:1.4"
  Normal   BackOff    2m22s (x89 over 27m)  kubelet            Back-off pulling image "mirrorgooglecontainers/defaultbackend-amd64:1.4"

我参考的文档是(https://kubesphere.io/zh/docs/v3.4/installing-on-linux/introduction/multioverview/),参考的是访问GitHub受限场景,严格配置了 export KKZONE=cn

严格按照文档操作指导进行的操作。