ubuntu18.04多节点使用 kk安装k8s+kubesphere3.0，安装 后页面打开正常 ，页面查看服务正常 ，但是主机上使用kubectl get报错Unable to connect to the server: EOF

root@master:/home/bglab# kubectl version
Client Version: version.Info{Major:“1”, Minor:“17”, GitVersion:“v1.17.9”, GitCommit:“4fb7ed12476d57b8437ada90b4f93b17ffaeed99”, GitTreeState:“clean”, BuildDate:“2020-07-15T16:18:16Z”, GoVersion:“go1.13.9”, Compiler:“gc”, Platform:“linux/amd64”}
Unable to connect to the server: EOF

root@master:/home/bglab# kubectl get nodes
Unable to connect to the server: EOF

  • wangchengli 主机不能使用kubectl修改$HOME/.kube/config文件中的server: https://lb.kubesphere.local:6443，将lb.kubesphere.local修改为master的IP，然后kubectl就可以正常使用了，
    界面打不开再用kubectl describe去排查具体问题

    journalctl -u kubelet |tail -100
    Sep 17 09:17:52 master kubelet[17025]: I0917 09:17:52.735837 17025 reconciler.go:156] Reconciler: start to sync state
    Sep 17 09:17:53 master kubelet[17025]: I0917 09:17:53.248524 17025 operation_generator.go:587] MountVolume.MountDevice succeeded for volume “pvc-48e509f2-4e00-40fc-9c94-e32bf6f1c349” (UniqueName: “kubernetes.io/local-volume/pvc-48e509f2-4e00-40fc-9c94-e32bf6f1c349”) pod “openldap-0” (UID: “9a7a3bad-8846-46b2-bedc-ef5297dec2ae”) device mount path “/var/openebs/local/pvc-48e509f2-4e00-40fc-9c94-e32bf6f1c349″
    Sep 17 11:25:26 master kubelet[17025]: W0917 11:25:26.302302 17025 prober.go:108] No ref for container “docker://3d9895475031c6cdd247bfa233c91fa260e7b0ee7dc9092933a67456e8cf1cd5″ (kube-apiserver-master_kube-system(a4969f6f281289b18fabecf8192094e4):kube-apiserver)
    Sep 17 11:25:46 master kubelet[17025]: W0917 11:25:46.304534 17025 prober.go:108] No ref for container “docker://3d9895475031c6cdd247bfa233c91fa260e7b0ee7dc9092933a67456e8cf1cd5” (kube-apiserver-master_kube-system(a4969f6f281289b18fabecf8192094e4):kube-apiserver)
    Sep 17 11:25:56 master kubelet[17025]: W0917 11:25:56.304669 17025 prober.go:108] No ref for container “docker://3d9895475031c6cdd247bfa233c91fa260e7b0ee7dc9092933a67456e8cf1cd5″ (kube-apiserver-master_kube-system(a4969f6f281289b18fabecf8192094e4):kube-apiserver)
    Sep 17 11:26:06 master kubelet[17025]: W0917 11:26:06.305515 17025 prober.go:108] No ref for container “docker://3d9895475031c6cdd247bfa233c91fa260e7b0ee7dc9092933a67456e8cf1cd5″ (kube-apiserver-master_kube-system(a4969f6f281289b18fabecf8192094e4):kube-apiserver)
    Sep 17 11:26:16 master kubelet[17025]: W0917 11:26:16.302577 17025 prober.go:108] No ref for container “docker://3d9895475031c6cdd247bfa233c91fa260e7b0ee7dc9092933a67456e8cf1cd5″ (kube-apiserver-master_kube-system(a4969f6f281289b18fabecf8192094e4):kube-apiserver)
    Sep 17 11:26:21 master kubelet[17025]: W0917 11:26:21.927450 17025 prober.go:108] No ref for container “docker://98084d6edd2ef3be44057aeead216039e2dd09452d1ad5fdf3d4a153c5d44ece” (kube-scheduler-master_kube-system(ebfd2fd6ad6578e9c4f9371af10daeea):kube-scheduler)
    Sep 17 11:26:29 master kubelet[17025]: W0917 11:26:29.067782 17025 pod_container_deletor.go:75] Container “1b142990caec3bf603fbd65ddddefeaa71c8e980b8c698e6bfaecbd2f91c937f” not found in pod’s containers
    Sep 17 11:26:32 master kubelet[17025]: E0917 11:26:32.161670 17025 remote_runtime.go:295] ContainerStatus “bc4fb092b4c0f384f57c2ff3097a91dc999423081660f547e1e7829cd983d3bd” from runtime service failed: rpc error: code = Unknown desc = Error: No such container: bc4fb092b4c0f384f57c2ff3097a91dc999423081660f547e1e7829cd983d3bd
    Sep 17 11:26:32 master kubelet[17025]: E0917 11:26:32.161755 17025 kuberuntime_manager.go:955] getPodContainerStatuses for pod “kube-scheduler-master_kube-system(ebfd2fd6ad6578e9c4f9371af10daeea)” failed: rpc error: code = Unknown desc = Error: No such container: bc4fb092b4c0f384f57c2ff3097a91dc999423081660f547e1e7829cd983d3bd
    Sep 17 11:26:56 master kubelet[17025]: W0917 11:26:56.300921 17025 prober.go:108] No ref for container “docker://3d9895475031c6cdd247bfa233c91fa260e7b0ee7dc9092933a67456e8cf1cd5” (kube-apiserver-master_kube-system(a4969f6f281289b18fabecf8192094e4):kube-apiserver)
    Sep 17 11:44:36 master kubelet[17025]: W0917 11:44:36.310091 17025 prober.go:108] No ref for container “docker://3d9895475031c6cdd247bfa233c91fa260e7b0ee7dc9092933a67456e8cf1cd5” (kube-apiserver-master_kube-system(a4969f6f281289b18fabecf8192094e4):kube-apiserver)
    Sep 17 11:45:26 master kubelet[17025]: W0917 11:45:26.308931 17025 prober.go:108] No ref for container “docker://3d9895475031c6cdd247bfa233c91fa260e7b0ee7dc9092933a67456e8cf1cd5” (kube-apiserver-master_kube-system(a4969f6f281289b18fabecf8192094e4):kube-apiserver)
    Sep 17 11:45:56 master kubelet[17025]: W0917 11:45:56.304229 17025 prober.go:108] No ref for container “docker://3d9895475031c6cdd247bfa233c91fa260e7b0ee7dc9092933a67456e8cf1cd5” (kube-apiserver-master_kube-system(a4969f6f281289b18fabecf8192094e4):kube-apiserver)

    取日志时间是下午3点，以上 最新时间点是11:45

    journalctl -u docker |tail -100
    Sep 17 09:12:41 master dockerd[5917]: time=“2020-09-17T09:12:41.591608177+08:00″ level=warning msg=“Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.”
    Sep 17 09:13:02 master dockerd[5917]: time=“2020-09-17T09:13:02.197464375+08:00″ level=warning msg=“Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.”
    Sep 17 09:13:03 master dockerd[5917]: time=“2020-09-17T09:13:03.741194368+08:00″ level=info msg=“ignoring event” module=libcontainerd namespace=moby topic=/tasks/delete type=“events.TaskDelete”
    Sep 17 09:13:13 master dockerd[5917]: time=“2020-09-17T09:13:13.674923285+08:00” level=warning msg=“Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.”
    Sep 17 09:13:24 master dockerd[5917]: time=“2020-09-17T09:13:24.513895265+08:00” level=info msg=“ignoring event” module=libcontainerd namespace=moby topic=/tasks/delete type=“    events.TaskDelete”
    Sep 17 09:13:47 master dockerd[5917]: time=“2020-09-17T09:13:47.590228919+08:00″ level=warning msg=“Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.”
    Sep 17 09:14:12 master dockerd[5917]: time=“2020-09-17T09:14:12.237895876+08:00” level=info msg=“ignoring event” module=libcontainerd namespace=moby topic=/tasks/delete type=“events.TaskDelete”
    Sep 17 09:14:28 master dockerd[5917]: time=“2020-09-17T09:14:28.576451913+08:00” level=warning msg=“Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.”
    Sep 17 11:26:19 master dockerd[5917]: time=“2020-09-17T11:26:19.122899679+08:00” level=info msg=“ignoring event” module=libcontainerd namespace=moby topic=/tasks/delete type=“    events.TaskDelete”
    bglab@master:$

    取日志时间是下午3点，以上 最新时间点是11:26

    1、资源：
    bglab@master:~$ free -m
    total used free shared buff/cache available
    Mem: 16040 1327 11095 2 3617 14624
    Swap: 0 0 0
    bglab@master:~$ df -h
    Filesystem Size Used Avail Use% Mounted on
    udev 7.9G 0 7.9G 0% /dev
    tmpfs 1.6G 2.1M 1.6G 1% /run
    /dev/sda2 147G 9.4G 130G 7% /
    tmpfs 7.9G 0 7.9G 0% /dev/shm
    tmpfs 5.0M 0 5.0M 0% /run/lock
    tmpfs 7.9G 0 7.9G 0% /sys/fs/cgroup
    /dev/loop0 90M 90M 0 100% /snap/core/8268
    /dev/loop1 98M 98M 0 100% /snap/core/9993
    tmpfs 1.6G 0 1.6G 0% /run/user/1000
    bglab@master:~$ top
    top - 15:08:54 up 23:45, 2 users, load average: 1.25, 1.84, 1.67
    Tasks: 247 total, 1 running, 164 sleeping, 0 stopped, 0 zombie
    %Cpu(s): 3.8 us, 3.0 sy, 0.0 ni, 85.9 id, 7.0 wa, 0.0 hi, 0.3 si, 0.0 st
    KiB Mem : 16425168 total, 11361208 free, 1359136 used, 3704824 buff/cache
    KiB Swap: 0 total, 0 free, 0 used. 14973288 avail Mem

    PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

    8449 root 20 0 773592 610124 70016 S 9.3 3.7 37:07.89 kube-apiserver

    17025 root 20 0 1706720 114736 63636 S 7.3 0.7 24:33.56 kubelet

    7571 root 20 0 10.085g 83140 25984 S 3.6 0.5 15:40.51 etcd

    5917 root 20 0 2635188 106368 47848 S 3.0 0.6 12:19.77 dockerd

    11895 root 20 0 1337824 44676 33052 S 2.0 0.3 9:10.25 calico-node

    11483 root 20 0 211316 118932 58880 S 1.7 0.7 9:17.48 kube-controller

    2、防火墙：
    # ufw status
    Status: inactive

    @Feynman 对于只有一个master节点的k8s集群，config-sample.yaml文件中的lb.kubesphere.local:6443地址需要替换成master本机地址吗

      csz711 单节点不需要替换本机地址。
      机器内核版本贴下。

          Forest-L
          root@master:/etc/kubernetes# cat /proc/version
          Linux version 4.15.0-76-generic (buildd@lcy01-amd64-029) (gcc version 7.4.0 (Ubuntu 7.4.0-1ubuntu118.04.1)) #86-Ubuntu SMP Fri Jan 17 17:24:28 UTC 2020

          另外是否跟网络插件是calico有关系呢？我想将网络插件从calico更改为flannel也没有成功，问题描述见https://kubesphere.com.cn/forum/d/2222-kubesphere3-0-network-plugin-calico-flannel

              4 天 后
              1 个月 后

              csz711 现在什么进展？成功了吗？我遇到了同样的问题，甚至现在页面都打不开。

                  wangchengli 还没解决，我的kubesphere界面上的kubectl是可以用的，暂时在用着界面上的kubectl

                    在后台敲命令加上–v=9看看到底用的哪个文件 有可能是多次装集群有残留的旧文件

                    14 天 后

                    wangchengli 主机不能使用kubectl修改$HOME/.kube/config文件中的server: https://lb.kubesphere.local:6443，将lb.kubesphere.local修改为master的IP，然后kubectl就可以正常使用了，
                    界面打不开再用kubectl describe去排查具体问题