创建部署问题时，请参考下面模板，你提供的信息越多，越容易及时获得解答。
你只花一分钟创建的问题，不能指望别人花上半个小时给你解答。
发帖前请点击发表主题右边的预览(👀) 按钮，确保帖子格式正确。

操作系统信息
虚拟机 Centos7.9 4C/8G

Kubernetes版本信息
v1.20.15。二进制部署，一主三从。

容器运行时
docker，版本: 20.10.9

KubeSphere版本信息
v3.3.2。在线安装。已有K8s安装

部署历程

情况一、kubernetes集群规划

二进制部署

一主二从，特别说明: master节点没有部署kubelet和kube-proxy服务，一开始没打算让master节点承担worker任务

kubesphere安装

--javascripttypescriptbashsqljsonhtmlcssccppjavarubypythongorustmarkdown

$$
参考官方部署文档: 因为我的kubernetes集群可以访问公网，所以基于在线方式进行安装 # 下载安装文件 curl -L -O https://github.com/kubesphere/ks-installer/releases/download/v3.3.1/cluster-configuration.yaml curl -L -O https://github.com/kubesphere/ks-installer/releases/download/v3.3.1/kubesphere-installer.yaml 注: 不启用多集群配置，安装正常
$$

启用多集群管理，配置主集群后开始安装

--javascripttypescriptbashsqljsonhtmlcssccppjavarubypythongorustmarkdown

$$
报错如下: Start installing monitoring Start installing multicluster Start installing openpitrix Start installing network ************************************************** Waiting for all tasks to be completed … task network status is successful (¼) task openpitrix status is successful (2/4) task monitoring status is successful (¾) task multicluster status is failed (4/4) ************************************************** Collecting installation results … Task ‘multicluster’ failed: ****************************************************************************************************************************************************** { “counter”: 65, “created”: “2023-03-21T04:19:43.002351”, “end_line”: 67, “event”: “runner_on_failed”, “event_data”: { “duration”: 3126.285635, “end”: “2023-03-21T04:19:43.002208”, “event_loop”: null, “host”: “localhost”, “ignore_errors”: null, “play”: “localhost”, “play_pattern”: “localhost”, “play_uuid”: “b220dc5c-bcb3-72c3-01b0-000000000005”, “playbook”: “/kubesphere/playbooks/multicluster.yaml”, “playbook_uuid”: “852e640e-0f4e-4d8f-9d55-a5b9608c5b3f”, “remote_addr”: “127.0.0.1”, “res”: { “ansible_no_log”: false, “attempts”: 10, “changed”: true, “cmd”: “/usr/local/bin/helm upgrade –install kubefed /kubesphere/kubesphere/kubefed/kubefed -f /kubesphere/kubesphere/kubefed/custom-values-kubefed.yaml –namespace kube-federation-system –wait –timeout 1800s\n”, “delta”: “0:01:54.017540”, “end”: “2023-03-21 12:19:42.920736”, “invocation”: { “module_args”: { “raw_params”: “/usr/local/bin/helm upgrade –install kubefed /kubesphere/kubesphere/kubefed/kubefed -f /kubesphere/kubesphere/kubefed/custom-values-kubefed.yaml –namespace kube-federation-system –wait –timeout 1800s\n”, “_uses_shell”: true, “argv”: null, “chdir”: null, “creates”: null, “executable”: null, “removes”: null, “stdin”: null, “stdin_add_newline”: true, “strip_empty_ends”: true, “warn”: true } }, “msg”: “non-zero return code”, “rc”: 1, “start”: “2023-03-21 12:17:48.903196”, “stderr”: "Error: UPGRADE FAILED: cannot patch \“clusterroles.rbac.authorization.k8s.io\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: context deadline exceeded && cannot patch \“configmaps\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: context deadline exceeded && cannot patch \“deployments.apps\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: context deadline exceeded && cannot patch \“ingresses.networking.k8s.io\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: context deadline exceeded && cannot patch \“jobs.batch\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: context deadline exceeded && cannot patch \“namespaces\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: context deadline exceeded && cannot patch \“replicasets.apps\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: context deadline exceeded && cannot patch \“secrets\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: context deadline exceeded && cannot patch \“serviceaccounts\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: context deadline exceeded && cannot patch \“services\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: context deadline exceeded && cannot patch \“kubefed\” with kind KubeFedConfig: Internal error occurred: failed calling webhook \“kubefedconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-kubefedconfig?timeout=10s\”: context deadline exceeded", “stderr_lines”: [ "Error: UPGRADE FAILED: cannot patch \“clusterroles.rbac.authorization.k8s.io\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: context deadline exceeded && cannot patch \“configmaps\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: context deadline exceeded && cannot patch \“deployments.apps\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: context deadline exceeded && cannot patch \“ingresses.networking.k8s.io\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: context deadline exceeded && cannot patch \“jobs.batch\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: context deadline exceeded && cannot patch \“namespaces\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: context deadline exceeded && cannot patch \“replicasets.apps\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: context deadline exceeded && cannot patch \“secrets\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: context deadline exceeded && cannot patch \“serviceaccounts\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: context deadline exceeded && cannot patch \“services\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: context deadline exceeded && cannot patch \“kubefed\” with kind KubeFedConfig: Internal error occurred: failed calling webhook \“kubefedconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-kubefedconfig?timeout=10s\”: context deadline exceeded" ], “stdout”: "", “stdout_lines”: [] }, “resolved_action”: “command”, “role”: “ks-multicluster”, “start”: “2023-03-21T03:27:36.716573”, “task”: “Kubefed | Initing kube-federation-system”, “task_action”: “command”, “task_args”: "", “task_path”: “/kubesphere/installer/roles/ks-multicluster/tasks/main.yml:51”, “task_uuid”: “b220dc5c-bcb3-72c3-01b0-00000000001f”, “uuid”: “45090cf4-d26b-4d5d-ba55-a66b44d5c50b” }, “parent_uuid”: “b220dc5c-bcb3-72c3-01b0-00000000001f”, “pid”: 3087, “runner_ident”: “multicluster”, “start_line”: 66, “stdout”: "fatal: [localhost]: FAILED! => {\“attempts\”: 10, \“changed\”: true, \“cmd\”: \“/usr/local/bin/helm upgrade –install kubefed /kubesphere/kubesphere/kubefed/kubefed -f /kubesphere/kubesphere/kubefed/custom-values-kubefed.yaml –namespace kube-federation-system –wait –timeout 1800s\n\”, \“delta\”: \“0:01:54.017540\”, \“end\”: \“2023-03-21 12:19:42.920736\”, \“msg\”: \“non-zero return code\”, \“rc\”: 1, \“start\”: \“2023-03-21 12:17:48.903196\”, \“stderr\”: \"Error: UPGRADE FAILED: cannot patch \\“clusterroles.rbac.authorization.k8s.io\\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: context deadline exceeded && cannot patch \\“configmaps\\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: context deadline exceeded && cannot patch \\“deployments.apps\\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: context deadline exceeded && cannot patch \\“ingresses.networking.k8s.io\\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: context deadline exceeded && cannot patch \\“jobs.batch\\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: context deadline exceeded && cannot patch \\“namespaces\\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: context deadline exceeded && cannot patch \\“replicasets.apps\\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: context deadline exceeded && cannot patch \\“secrets\\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: context deadline exceeded && cannot patch \\“serviceaccounts\\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: context deadline exceeded && cannot patch \\“services\\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: context deadline exceeded && cannot patch \\“kubefed\\” with kind KubeFedConfig: Internal error occurred: failed calling webhook \\“kubefedconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-kubefedconfig?timeout=10s\\”: context deadline exceeded\", \“stderr_lines\”: [\"Error: UPGRADE FAILED: cannot patch \\“clusterroles.rbac.authorization.k8s.io\\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: context deadline exceeded && cannot patch \\“configmaps\\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: context deadline exceeded && cannot patch \\“deployments.apps\\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: context deadline exceeded && cannot patch \\“ingresses.networking.k8s.io\\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: context deadline exceeded && cannot patch \\“jobs.batch\\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: context deadline exceeded && cannot patch \\“namespaces\\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: context deadline exceeded && cannot patch \\“replicasets.apps\\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: context deadline exceeded && cannot patch \\“secrets\\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: context deadline exceeded && cannot patch \\“serviceaccounts\\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: context deadline exceeded && cannot patch \\“services\\” with kind FederatedTypeConfig: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: context deadline exceeded && cannot patch \\“kubefed\\” with kind KubeFedConfig: Internal error occurred: failed calling webhook \\“kubefedconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-kubefedconfig?timeout=10s\\”: context deadline exceeded\"], \“stdout\”: \“\”, \“stdout_lines\”: []}", “uuid”: “45090cf4-d26b-4d5d-ba55-a66b44d5c50b” }
$$

排查历程

1.

查找并学习kubefed联邦集群的资料

2.

排查kubernetes集群本身状态，发现集群网络都是正常的

3.

排查kubernetes集群的准入插件是不是少了，发现也是正常的

4.

期间多次重置集群，多次重复安装，陷入迷茫。去社区、微信群、官方邮箱等地方寻求帮助，都没有得到有效的结果

5.

反复翻阅官方文档，官方文档的ack安装文档突然让我闪出一个关注点，那就是ack集群的master节点也是作为worker节点加入到集群的。所以我就抱着试试的态度，把我的master节点也部署kubelet和kube-proxy服务，并加入到集群

情况二、重新规划kubernetes集群

二进制部署，一主三从，master节点部署kubelet和kube-proxy服务，加入集群

kubesphere安装

--javascripttypescriptbashsqljsonhtmlcssccppjavarubypythongorustmarkdown

$$
参考官方部署文档: 因为我的kubernetes集群可以访问公网，所以基于在线方式进行安装 # 下载安装文件 curl -L -O https://github.com/kubesphere/ks-installer/releases/download/v3.3.1/cluster-configuration.yaml curl -L -O https://github.com/kubesphere/ks-installer/releases/download/v3.3.1/kubesphere-installer.yaml 注: 不启用多集群配置，安装正常
$$

启用多集群管理，配置主集群后开始安装

--javascripttypescriptbashsqljsonhtmlcssccppjavarubypythongorustmarkdown

$$
报错如下: Start installing monitoring Start installing multicluster Start installing openpitrix Start installing network ************************************************** Waiting for all tasks to be completed … task network status is successful (¼) task openpitrix status is successful (2/4) task monitoring status is successful (¾) task multicluster status is failed (4/4) ************************************************** Collecting installation results … Task ‘multicluster’ failed: ****************************************************************************************************************************************************** { “counter”: 65, “created”: “2023-03-23T08:30:29.765963”, “end_line”: 67, “event”: “runner_on_failed”, “event_data”: { “duration”: 659.827499, “end”: “2023-03-23T08:30:29.765746”, “event_loop”: null, “host”: “localhost”, “ignore_errors”: null, “play”: “localhost”, “play_pattern”: “localhost”, “play_uuid”: “f2b68e09-a924-cb57-afdb-000000000005”, “playbook”: “/kubesphere/playbooks/multicluster.yaml”, “playbook_uuid”: “755bdafd-56f0-4d8b-a6c9-7b6534c172c3”, “remote_addr”: “127.0.0.1”, “res”: { “ansible_no_log”: false, “attempts”: 10, “changed”: true, “cmd”: “/usr/local/bin/helm upgrade –install kubefed /kubesphere/kubesphere/kubefed/kubefed -f /kubesphere/kubesphere/kubefed/custom-values-kubefed.yaml –namespace kube-federation-system –wait –timeout 1800s\n”, “delta”: “0:00:04.732699”, “end”: “2023-03-23 16:30:29.678012”, “invocation”: { “module_args”: { “raw_params”: “/usr/local/bin/helm upgrade –install kubefed /kubesphere/kubesphere/kubefed/kubefed -f /kubesphere/kubesphere/kubefed/custom-values-kubefed.yaml –namespace kube-federation-system –wait –timeout 1800s\n”, “_uses_shell”: true, “argv”: null, “chdir”: null, “creates”: null, “executable”: null, “removes”: null, “stdin”: null, “stdin_add_newline”: true, “strip_empty_ends”: true, “warn”: true } }, “msg”: “non-zero return code”, “rc”: 1, “start”: “2023-03-23 16:30:24.945313”, “stderr”: "Error: UPGRADE FAILED: failed to create resource: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: x509: certificate signed by unknown authority (possibly because of \“crypto/rsa: verification error\” while trying to verify candidate authority certificate \“kubefed-admission-webhook-ca\”)", “stderr_lines”: [ "Error: UPGRADE FAILED: failed to create resource: Internal error occurred: failed calling webhook \“federatedtypeconfigs.core.kubefed.io\”: Post \“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\”: x509: certificate signed by unknown authority (possibly because of \“crypto/rsa: verification error\” while trying to verify candidate authority certificate \“kubefed-admission-webhook-ca\”)" ], “stdout”: "", “stdout_lines”: [] }, “resolved_action”: “command”, “role”: “ks-multicluster”, “start”: “2023-03-23T08:19:29.938247”, “task”: “Kubefed | Initing kube-federation-system”, “task_action”: “command”, “task_args”: "", “task_path”: “/kubesphere/installer/roles/ks-multicluster/tasks/main.yml:51”, “task_uuid”: “f2b68e09-a924-cb57-afdb-00000000001f”, “uuid”: “78adcb4f-c0a3-437d-9cec-87d28e6f7811” }, “parent_uuid”: “f2b68e09-a924-cb57-afdb-00000000001f”, “pid”: 3511, “runner_ident”: “multicluster”, “start_line”: 66, “stdout”: "fatal: [localhost]: FAILED! => {\“attempts\”: 10, \“changed\”: true, \“cmd\”: \“/usr/local/bin/helm upgrade –install kubefed /kubesphere/kubesphere/kubefed/kubefed -f /kubesphere/kubesphere/kubefed/custom-values-kubefed.yaml –namespace kube-federation-system –wait –timeout 1800s\n\”, \“delta\”: \“0:00:04.732699\”, \“end\”: \“2023-03-23 16:30:29.678012\”, \“msg\”: \“non-zero return code\”, \“rc\”: 1, \“start\”: \“2023-03-23 16:30:24.945313\”, \“stderr\”: \"Error: UPGRADE FAILED: failed to create resource: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: x509: certificate signed by unknown authority (possibly because of \\“crypto/rsa: verification error\\” while trying to verify candidate authority certificate \\“kubefed-admission-webhook-ca\\”)\", \“stderr_lines\”: [\"Error: UPGRADE FAILED: failed to create resource: Internal error occurred: failed calling webhook \\“federatedtypeconfigs.core.kubefed.io\\”: Post \\“https://kubefed-admission-webhook.kube-federation-system.svc:443/validate-federatedtypeconfigs?timeout=10s\\”: x509: certificate signed by unknown authority (possibly because of \\“crypto/rsa: verification error\\” while trying to verify candidate authority certificate \\“kubefed-admission-webhook-ca\\”)\"], \“stdout\”: \“\”, \“stdout_lines\”: []}", “uuid”: “78adcb4f-c0a3-437d-9cec-87d28e6f7811” }
$$

排查历程

1.

这个时候是很开心的，因为报错不一样的，而且很明确是因为证书问题。不再是之前的报错了。

2.

这个时候去到社区查找，经过多次找寻，终于找到了同样的报错isses，然后按照给出的解决方法

--javascripttypescriptbashsqljsonhtmlcssccppjavarubypythongorustmarkdown

$$
isses链接: https://kubesphere.io/forum/d/6774-v321-k8s/2 解决方法: 修改向apiserver注册的根证书，和webhook里secret使用的保持一致 validatingwebhookconfiguration/validations.core.kubefed.io里的caBundle mutatingwebhookconfiguration/mutation.core.kubfefed.io里的caBundle secrets/kubefed-admission-webhook-serving-cert中的tls.crt 这个现象不知道怎么导致的，把tls.crt复制覆盖掉这两处caBundle即可
$$

开始解决

查找 secrets/kubefed-admission-webhook-serving-cert

--javascripttypescriptbashsqljsonhtmlcssccppjavarubypythongorustmarkdown

$$
kubectl get secret -A |grep kubefed-admission-webhook-serving-cert 获取tls.crt文件内容
$$

修改validatingwebhookconfiguration/validations.core.kubefed.io的caBundle

--javascripttypescriptbashsqljsonhtmlcssccppjavarubypythongorustmarkdown

$$
kubectl get validatingwebhookconfiguration -A |grep validations.core.kubefed.io 注意: 这个文件里面有好几处的caBundle，都需要修改
$$

修改mutatingwebhookconfiguration/mutation.core.kubfefed.io里的caBundle

--javascripttypescriptbashsqljsonhtmlcssccppjavarubypythongorustmarkdown

$$
kubectl get mutatingwebhookconfiguration -A |grep mutation.core.kubfefed.io 注意: 这个文件里面只有一处caBundle
$$

然后重启ks-installer，重新来一遍安装即可。（我一开始是重启kubefed-controller服务，但是这个是不对的）

--javascripttypescriptbashsqljsonhtmlcssccppjavarubypythongorustmarkdown

$$
成功信息，多集群配置成功 Start installing monitoring Start installing multicluster Start installing openpitrix Start installing network ************************************************** Waiting for all tasks to be completed … task network status is successful (¼) task openpitrix status is successful (2/4) task monitoring status is successful (¾) task multicluster status is successful (4/4) ************************************************** Collecting installation results … ##################################################### ### Welcome to KubeSphere! ### #####################################################
$$