使用三台ubuntu18.04搭建了k8s和kubesphere，版本信息如下：
k8s：v1.18.8
kubesphere：v3.0.0
容器运行时：containerd

采用kk创建的集群，并安装好kubesphere，计划通过kk进行升级。升级命令为：
./kk upgrade –with-kubernetes v1.18.8 –with-kubesphere v3.1.0

v3.1.0
[master1 172.17.3.12] MSG:
namespace/kubesphere-system unchanged
namespace/kubesphere-monitoring-system unchanged
[master1 172.17.3.12] MSG:
namespace/kubesphere-system unchanged
serviceaccount/ks-installer unchanged
customresourcedefinition.apiextensions.k8s.io/clusterconfigurations.installer.kubesphere.io unchanged
clusterrole.rbac.authorization.k8s.io/ks-installer unchanged
clusterrolebinding.rbac.authorization.k8s.io/ks-installer unchanged
deployment.apps/ks-installer unchanged
Please wait for the installation to complete:    >>--->

Please wait for the installation to complete:   <---<<

WARN[12:29:28 CST] Task failed ...
WARN[12:29:28 CST] error: KubeSphere startup timeout.
Error: Failed to upgrade kubesphere: KubeSphere startup timeout.
Usage:
  kk upgrade [flags]

Flags:
      --download-cmd string      The user defined command to download the necessary binary files. The first param '%s' is output path, the second param '%s', is the URL (default "curl -L -o %s %s")
  -f, --filename string          Path to a configuration file
  -h, --help                     help for upgrade
      --skip-pull-images         Skip pre pull images
      --with-kubernetes string   Specify a supported version of kubernetes
      --with-kubesphere          Deploy a specific version of kubesphere (default v3.0.0)

Global Flags:
      --debug        Print detailed information (default true)
      --in-cluster   Running inside the cluster

Failed to upgrade kubesphere: KubeSphere startup timeout.

发现控制台启动超时，执行如下命令查看日志
kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath=‘{.items[0].metadata.name}’) -f

2021-06-24T03:22:52Z INFO     : shell-operator latest
2021-06-24T03:22:52Z INFO     : Use temporary dir: /tmp/shell-operator
2021-06-24T03:22:52Z INFO     : Initialize hooks manager ...
2021-06-24T03:22:52Z INFO     : Search and load hooks ...
2021-06-24T03:22:52Z INFO     : Load hook config from '/hooks/kubesphere/installRunner.py'
2021-06-24T03:22:52Z INFO     : HTTP SERVER Listening on 0.0.0.0:9115
2021-06-24T03:22:53Z INFO     : Load hook config from '/hooks/kubesphere/schedule.sh'
2021-06-24T03:22:53Z INFO     : Initializing schedule manager ...
2021-06-24T03:22:53Z INFO     : KUBE Init Kubernetes client
2021-06-24T03:22:53Z INFO     : KUBE-INIT Kubernetes client is configured successfully
2021-06-24T03:22:53Z INFO     : MAIN: run main loop
2021-06-24T03:22:53Z INFO     : MAIN: add onStartup tasks
2021-06-24T03:22:53Z INFO     : QUEUE add all HookRun@OnStartup
2021-06-24T03:22:53Z INFO     : Running schedule manager ...
2021-06-24T03:22:53Z INFO     : MSTOR Create new metric shell_operator_live_ticks
2021-06-24T03:22:53Z INFO     : MSTOR Create new metric shell_operator_tasks_queue_length
2021-06-24T03:22:53Z INFO     : GVR for kind 'ClusterConfiguration' is installer.kubesphere.io/v1alpha1, Resource=clusterconfigurations
2021-06-24T03:22:53Z INFO     : EVENT Kube event '53b1e5bc-92d5-4e2e-b32f-bb45f1153d49'
2021-06-24T03:22:53Z INFO     : QUEUE add TASK_HOOK_RUN@KUBE_EVENTS kubesphere/installRunner.py
2021-06-24T03:22:56Z INFO     : TASK_RUN HookRun@KUBE_EVENTS kubesphere/installRunner.py
2021-06-24T03:22:56Z INFO     : Running hook 'kubesphere/installRunner.py' binding 'KUBE_EVENTS' ...
Delete old cluster configuration successfully
Create cluster configuration successfully
2021-06-24T03:22:56Z INFO     : EVENT Kube event '53b1e5bc-92d5-4e2e-b32f-bb45f1153d49'
2021-06-24T03:22:56Z INFO     : QUEUE add TASK_HOOK_RUN@KUBE_EVENTS kubesphere/installRunner.py
2021-06-24T03:22:59Z INFO     : TASK_RUN HookRun@KUBE_EVENTS kubesphere/installRunner.py
2021-06-24T03:22:59Z INFO     : Running hook 'kubesphere/installRunner.py' binding 'KUBE_EVENTS' ...
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match 'all'

PLAY [localhost] ***************************************************************

TASK [download : include_tasks] ************************************************
skipping: [localhost]

TASK [download : Downloading items] ********************************************
skipping: [localhost]

TASK [download : Synchronizing container] **************************************
skipping: [localhost]

TASK [kubesphere-defaults : KubeSphere | Setting images' namespace override] ***
skipping: [localhost]

TASK [kubesphere-defaults : KubeSphere | Configuring defaults] *****************
ok: [localhost] => {
    "msg": "Check roles/kubesphere-defaults/defaults/main.yml"
}

TASK [preinstall : KubeSphere | Checking Kubernetes version] *******************
changed: [localhost]

TASK [preinstall : KubeSphere | Initing Kubernetes version] ********************
ok: [localhost]

TASK [preinstall : KubeSphere | Stopping if Kubernetes version is nonsupport] ***
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}

TASK [preinstall : KubeSphere | Checking StorageClass] *************************
changed: [localhost]

TASK [preinstall : KubeSphere | Stopping if StorageClass was not found] ********
skipping: [localhost]

TASK [preinstall : KubeSphere | Checking default StorageClass] *****************
changed: [localhost]

TASK [preinstall : KubeSphere | Stopping if default StorageClass was not found] ***
ok: [localhost] => {
    "changed": false,
    "msg": "All assertions passed"
}

TASK [preinstall : KubeSphere | Checking KubeSphere component] *****************
changed: [localhost]

TASK [preinstall : KubeSphere | Getting KubeSphere component version] **********
changed: [localhost]

TASK [preinstall : KubeSphere | Getting KubeSphere component version] **********
skipping: [localhost] => (item=ks-openldap)
skipping: [localhost] => (item=ks-redis)
skipping: [localhost] => (item=ks-minio)
skipping: [localhost] => (item=ks-openpitrix)
skipping: [localhost] => (item=elasticsearch-logging)
skipping: [localhost] => (item=elasticsearch-logging-curator)
skipping: [localhost] => (item=istio)
skipping: [localhost] => (item=istio-init)
skipping: [localhost] => (item=jaeger-operator)
skipping: [localhost] => (item=ks-jenkins)
skipping: [localhost] => (item=ks-sonarqube)
skipping: [localhost] => (item=logging-fluentbit-operator)
skipping: [localhost] => (item=uc)
skipping: [localhost] => (item=metrics-server)
localhost                  : ok=9    changed=5    unreachable=0    failed=0    skipped=6    rescued=0    ignored=0

PLAY RECAP *********************************************************************
localhost                  : ok=9    changed=5    unreachable=0    failed=0    skipped=6    rescued=0    ignored=0

[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match 'all'

PLAY [localhost] ***************************************************************

TASK [download : include_tasks] ************************************************
skipping: [localhost]

TASK [download : Downloading items] ********************************************
skipping: [localhost]

TASK [download : Synchronizing container] **************************************
skipping: [localhost]

TASK [kubesphere-defaults : KubeSphere | Setting images' namespace override] ***
skipping: [localhost]

TASK [kubesphere-defaults : KubeSphere | Configuring defaults] *****************
ok: [localhost] => {
    "msg": "Check roles/kubesphere-defaults/defaults/main.yml"
}

TASK [metrics-server : Metrics-Server | Getting metrics-server installation files] ***
changed: [localhost]

TASK [metrics-server : Metrics-Server | Creating manifests] ********************
ok: [localhost] => (item={'file': 'metrics-server.yaml'})

TASK [metrics-server : Metrics-Server | Checking Metrics-Server] ***************
changed: [localhost]

TASK [metrics-server : Metrics-Server | Uninstalling old metrics-server] *******
changed: [localhost]

TASK [metrics-server : Metrics-Server | Installing new metrics-server] *********
changed: [localhost]

TASK [metrics-server : Metrics-Server | Waitting for metrics.k8s.io ready] *****
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (60 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (59 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (58 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (57 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (56 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (55 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (54 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (53 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (52 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (51 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (50 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (49 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (48 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (47 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (46 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (45 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (44 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (43 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (42 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (41 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (40 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (39 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (38 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (37 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (36 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (35 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (34 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (33 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (32 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (31 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (30 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (29 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (28 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (27 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (26 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (25 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (24 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (23 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (22 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (21 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (20 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (19 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (18 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (17 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (16 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (15 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (14 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (13 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (12 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (11 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (10 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (9 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (8 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (7 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (6 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (5 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (4 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (3 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (2 retries left).
FAILED - RETRYING: Metrics-Server | Waitting for metrics.k8s.io ready (1 retries left).
fatal: [localhost]: FAILED! => {"attempts": 60, "changed": true, "cmd": "/usr/local/bin/kubectl get apiservice | grep metrics.k8s.io\n", "delta": "0:00:00.150931", "end": "2021-06-24 03:33:37.107400", "rc": 0, "start": "2021-06-24 03:33:36.956469", "stderr": "", "stderr_lines": [], "stdout": "v1beta1.metrics.k8s.io                 kube-system/metrics-server   False (MissingEndpoints)   10m", "stdout_lines": ["v1beta1.metrics.k8s.io                 kube-system/metrics-server   False (MissingEndpoints)   10m"]}

PLAY RECAP *********************************************************************
localhost                  : ok=6    changed=4    unreachable=0    failed=1    skipped=4    rescued=0    ignored=0