多节点部署时，node 节点注册到集群中时，API Server 认证失败

/usr/local/bin/kubeadm join --config /etc/kubernetes/kubeadm-client.conf -v=10

I1128 14:54:08.720562  102964 token.go:202] [discovery] Failed to connect to API Server "lb.kubesphere.local:6443": token id "l7mzck" is invalid for this cluster or it has expired. Use "kubeadm token create" on the control-plane node to create a new valid token

hongming

所有机器统一做了操作

1. 关闭 firewalld
2. 禁用 se’linux
3. 开启 chronyd 时间同步
4. date 命令确认时间一致性

hongming
在主节点生成新 token

/usr/local/bin/kubeadm token create

node01 节点替换掉旧的token

vim /etc/kubernetes/kubeadm-client.conf
替换旧 token

重新在控制机执行 ./add-node.sh

仍然报错，如下：

TASK [kubernetes/kubeadm : Join to cluster] ********************************************************************************************************************************************
Thursday 28 November 2019  15:28:06 +0800 (0:00:01.002)       0:11:16.197 *****
skipping: [master01]
skipping: [master02]
skipping: [master03]
fatal: [node01]: FAILED! => {
    "changed": true,
    "cmd": [
        "timeout",
        "-k",
        "120s",
        "120s",
        "/usr/local/bin/kubeadm",
        "join",
        "--config",
        "/etc/kubernetes/kubeadm-client.conf",
        "--ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests"
    ],
    "delta": "0:01:00.323301",
    "end": "2019-11-28 15:29:07.570282",
    "rc": 1,
    "start": "2019-11-28 15:28:07.246981"
}

STDOUT:

[preflight] Running pre-flight checks


STDERR:

error execution phase preflight: couldn't validate the identity of the API Server: abort connecting to API servers after timeout of 1m0s


MSG:

non-zero return code

fatal: [node02]: FAILED! => {
    "changed": true,
    "cmd": [
        "timeout",
        "-k",
        "120s",
        "120s",
        "/usr/local/bin/kubeadm",
        "join",
        "--config",
        "/etc/kubernetes/kubeadm-client.conf",
        "--ignore-preflight-errors=DirAvailable--etc-kubernetes-manifests"
    ],
    "delta": "0:01:00.763477",
    "end": "2019-11-28 15:29:08.158381",
    "rc": 1,
    "start": "2019-11-28 15:28:07.394904"
}

STDOUT:

[preflight] Running pre-flight checks


STDERR:

error execution phase preflight: couldn't validate the identity of the API Server: abort connecting to API servers after timeout of 1m0s


MSG:

non-zero return code


TASK [kubernetes/kubeadm : Join to cluster with ignores] *******************************************************************************************************************************
Thursday 28 November 2019  15:29:08 +0800 (0:01:01.274)       0:12:17.471 *****
fatal: [node01]: FAILED! => {
    "changed": true,
    "cmd": [
        "timeout",
        "-k",
        "120s",
        "120s",
        "/usr/local/bin/kubeadm",
        "join",
        "--config",
        "/etc/kubernetes/kubeadm-client.conf",
        "--ignore-preflight-errors=all"
    ],
    "delta": "0:01:00.319199",
    "end": "2019-11-28 15:30:08.734662",
    "rc": 1,
    "start": "2019-11-28 15:29:08.415463"
}

STDOUT:

[preflight] Running pre-flight checks


STDERR:

error execution phase preflight: couldn't validate the identity of the API Server: abort connecting to API servers after timeout of 1m0s


MSG:

non-zero return code

fatal: [node02]: FAILED! => {
    "changed": true,
    "cmd": [
        "timeout",
        "-k",
        "120s",
        "120s",
        "/usr/local/bin/kubeadm",
        "join",
        "--config",
        "/etc/kubernetes/kubeadm-client.conf",
        "--ignore-preflight-errors=all"
    ],
    "delta": "0:01:00.658588",
    "end": "2019-11-28 15:30:09.236997",
    "rc": 1,
    "start": "2019-11-28 15:29:08.578409"
}

STDOUT:

[preflight] Running pre-flight checks


STDERR:

error execution phase preflight: couldn't validate the identity of the API Server: abort connecting to API servers after timeout of 1m0s


MSG:

non-zero return code


TASK [kubernetes/kubeadm : Display kubeadm join stderr if any] *************************************************************************************************************************
Thursday 28 November 2019  15:30:09 +0800 (0:01:01.085)       0:13:18.557 *****
skipping: [master01]
skipping: [master02]
ok: [node01] => {}

MSG:

Joined with warnings
[u"error execution phase preflight: couldn't validate the identity of the API Server: abort connecting to API servers after timeout of 1m0s"]


skipping: [master03]
ok: [node02] => {}

MSG:

Joined with warnings
[u"error execution phase preflight: couldn't validate the identity of the API Server: abort connecting to API servers after timeout of 1m0s"]



TASK [kubernetes/kubeadm : Update server field in kubelet kubeconfig] ******************************************************************************************************************
Thursday 28 November 2019  15:30:09 +0800 (0:00:00.159)       0:13:18.716 *****
skipping: [master01]
skipping: [master02]
skipping: [master03]

hongming

今天使用 nginx 搭建 tcp LB, 然后重新部署，还是同样的报错；

按照群里兄弟给的建议，control panel 上新建 token 添加 –print-join-command

然后执行： /usr/local/bin/kubeadm join 10.10.14.108:6443 –token 5sy6fz.1q03vb6g1j8qk83w –discovery-token-ca-cert-hash sha256:c847fb4d59bb735f2d373d6d87ac31f454b542f34660ccbd1b5615be76b1c6c2 -v=10

worker_processes auto;

events {
    worker_connections  1024;
}

error_log /var/log/nginx_error.log info;

stream {
    upstream k8s{
        hash $remote_addr consistent;
        server 10.10.14.108:6443 weight=5 max_fails=1 fail_timeout=10s;
        server 10.10.14.109:6443 weight=5 max_fails=1 fail_timeout=10s;
        server 10.10.14.110:6443 weight=5 max_fails=1 fail_timeout=10s;
    }

    server {
        listen 16443;
        proxy_connect_timeout 1s;
        proxy_timeout 3s;
        proxy_pass k8s;
    }
}