离线安装文档疑问
今天尝试了用registry和harbor做离线仓库, 发现了两个新问题
1.拉镜像时都会出现x509证书错误,在/etc/containerd/config.toml指定ca和dockerhub.kubekey.local的证书后才完成部署，这是不是kk的bug?
2.kk create cluster必须放在要部署的集群的某个节点上执行吗？，放在非集群节点会有镜像推送失败的问题
- 已编辑
- 如果是你自己部署的containerd或者docker，kk不会对其config进行重新配置（有覆盖用户自定义配置的风险），所以既然用户选择自行部署容器运行时，那么也需要自行对其配置。
- 不是必须的，你推送失败了应该是因为非集群节点没有镜像仓库证书
- 组件版本目前支持的比较少，不用修改。只需改k8s版本和镜像列表即可。官方文档给出的例子即是最佳实践
--- apiVersion: kubekey.kubesphere.io/v1alpha2 kind: Manifest metadata: name: sample spec: arches: - amd64 operatingSystems: - arch: amd64 type: linux id: centos version: "7" repository: iso: localPath: url: https://github.com/kubesphere/kubekey/releases/download/v2.2.1/centos7-rpms-amd64.iso - arch: amd64 type: linux id: ubuntu version: "20.04" repository: iso: localPath: url: https://github.com/kubesphere/kubekey/releases/download/v2.2.1/ubuntu-20.04-debs-amd64.iso kubernetesDistributions: - type: kubernetes version: v1.22.10 components: helm: version: v3.6.3 cni: version: v0.9.1 etcd: version: v3.4.13 ## For now, if your cluster container runtime is containerd, KubeKey will add a docker 20.10.8 container runtime in the below list. ## The reason is KubeKey creates a cluster with containerd by installing a docker first and making kubelet connect the socket file of containerd which docker contained. containerRuntimes: - type: docker version: 20.10.8 crictl: version: v1.22.0 docker-registry: version: "2" harbor: version: v2.4.1 docker-compose: version: v2.2.2 images: - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-apiserver:v1.22.10 - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controller-manager:v1.22.10 - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-proxy:v1.22.10 - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-scheduler:v1.22.10 - registry.cn-beijing.aliyuncs.com/kubesphereio/pause:3.5 - registry.cn-beijing.aliyuncs.com/kubesphereio/coredns:1.8.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/cni:v3.20.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controllers:v3.20.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/node:v3.20.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/pod2daemon-flexvol:v3.20.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/typha:v3.20.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/flannel:v0.12.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/provisioner-localpv:2.10.1 - registry.cn-beijing.aliyuncs.com/kubesphereio/linux-utils:2.10.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/haproxy:2.3 - registry.cn-beijing.aliyuncs.com/kubesphereio/nfs-subdir-external-provisioner:v4.0.2 - registry.cn-beijing.aliyuncs.com/kubesphereio/k8s-dns-node-cache:1.15.12 - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-installer:v3.3.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-apiserver:v3.3.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-console:v3.3.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-controller-manager:v3.3.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/kubectl:v1.20.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/kubectl:v1.21.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/kubectl:v1.22.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/kubefed:v0.8.1 - registry.cn-beijing.aliyuncs.com/kubesphereio/tower:v0.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/minio:RELEASE.2019-08-07T01-59-21Z - registry.cn-beijing.aliyuncs.com/kubesphereio/mc:RELEASE.2019-08-07T23-14-43Z - registry.cn-beijing.aliyuncs.com/kubesphereio/snapshot-controller:v4.0.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/nginx-ingress-controller:v1.1.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/defaultbackend-amd64:1.4 - registry.cn-beijing.aliyuncs.com/kubesphereio/metrics-server:v0.4.2 - registry.cn-beijing.aliyuncs.com/kubesphereio/redis:5.0.14-alpine - registry.cn-beijing.aliyuncs.com/kubesphereio/haproxy:2.0.25-alpine - registry.cn-beijing.aliyuncs.com/kubesphereio/alpine:3.14 - registry.cn-beijing.aliyuncs.com/kubesphereio/openldap:1.3.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/netshoot:v1.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/cloudcore:v1.9.2 - registry.cn-beijing.aliyuncs.com/kubesphereio/iptables-manager:v1.9.2 - registry.cn-beijing.aliyuncs.com/kubesphereio/edgeservice:v0.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/openpitrix-jobs:v3.2.1 - registry.cn-beijing.aliyuncs.com/kubesphereio/devops-apiserver:v3.3.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/devops-controller:v3.3.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/devops-tools:v3.3.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/ks-jenkins:v3.3.0-2.319.1 - registry.cn-beijing.aliyuncs.com/kubesphereio/inbound-agent:4.10-2 - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-base:v3.2.2 - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-nodejs:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.1-jdk11 - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-python:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.16 - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.17 - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.18 - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-base:v3.2.2-podman - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-nodejs:v3.2.0-podman - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.0-podman - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.1-jdk11-podman - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-python:v3.2.0-podman - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.0-podman - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.16-podman - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.17-podman - registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.18-podman - registry.cn-beijing.aliyuncs.com/kubesphereio/s2ioperator:v3.2.1 - registry.cn-beijing.aliyuncs.com/kubesphereio/s2irun:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/s2i-binary:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java11-centos7:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java11-runtime:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java8-centos7:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java8-runtime:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/java-11-centos7:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/java-8-centos7:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/java-8-runtime:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/java-11-runtime:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/nodejs-8-centos7:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/nodejs-6-centos7:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/nodejs-4-centos7:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/python-36-centos7:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/python-35-centos7:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/python-34-centos7:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/python-27-centos7:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/argocd:v2.3.3 - registry.cn-beijing.aliyuncs.com/kubesphereio/argocd-applicationset:v0.4.1 - registry.cn-beijing.aliyuncs.com/kubesphereio/dex:v2.30.2 - registry.cn-beijing.aliyuncs.com/kubesphereio/redis:6.2.6-alpine - registry.cn-beijing.aliyuncs.com/kubesphereio/configmap-reload:v0.5.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/prometheus:v2.34.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/prometheus-config-reloader:v0.55.1 - registry.cn-beijing.aliyuncs.com/kubesphereio/prometheus-operator:v0.55.1 - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-rbac-proxy:v0.11.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-state-metrics:v2.3.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/node-exporter:v1.3.1 - registry.cn-beijing.aliyuncs.com/kubesphereio/alertmanager:v0.23.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/thanos:v0.25.2 - registry.cn-beijing.aliyuncs.com/kubesphereio/grafana:8.3.3 - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-rbac-proxy:v0.8.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/notification-manager-operator:v1.4.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/notification-manager:v1.4.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/notification-tenant-sidecar:v3.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/elasticsearch-curator:v5.7.6 - registry.cn-beijing.aliyuncs.com/kubesphereio/elasticsearch-oss:6.8.22 - registry.cn-beijing.aliyuncs.com/kubesphereio/fluentbit-operator:v0.13.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/docker:19.03 - registry.cn-beijing.aliyuncs.com/kubesphereio/fluent-bit:v1.8.11 - registry.cn-beijing.aliyuncs.com/kubesphereio/log-sidecar-injector:1.1 - registry.cn-beijing.aliyuncs.com/kubesphereio/filebeat:6.7.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-events-operator:v0.4.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-events-exporter:v0.4.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-events-ruler:v0.4.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-auditing-operator:v0.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/kube-auditing-webhook:v0.2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/pilot:1.11.1 - registry.cn-beijing.aliyuncs.com/kubesphereio/proxyv2:1.11.1 - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-operator:1.27 - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-agent:1.27 - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-collector:1.27 - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-query:1.27 - registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-es-index-cleaner:1.27 - registry.cn-beijing.aliyuncs.com/kubesphereio/kiali-operator:v1.38.1 - registry.cn-beijing.aliyuncs.com/kubesphereio/kiali:v1.38 - registry.cn-beijing.aliyuncs.com/kubesphereio/busybox:1.31.1 - registry.cn-beijing.aliyuncs.com/kubesphereio/nginx:1.14-alpine - registry.cn-beijing.aliyuncs.com/kubesphereio/wget:1.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/hello:plain-text - registry.cn-beijing.aliyuncs.com/kubesphereio/wordpress:4.8-apache - registry.cn-beijing.aliyuncs.com/kubesphereio/hpa-example:latest - registry.cn-beijing.aliyuncs.com/kubesphereio/java:openjdk-8-jre-alpine - registry.cn-beijing.aliyuncs.com/kubesphereio/fluentd:v1.4.2-2.0 - registry.cn-beijing.aliyuncs.com/kubesphereio/perl:latest - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-productpage-v1:1.16.2 - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-reviews-v1:1.16.2 - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-reviews-v2:1.16.2 - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-details-v1:1.16.2 - registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-ratings-v1:1.16.3 - registry.cn-beijing.aliyuncs.com/kubesphereio/scope:1.13.0
安装官网的离线文档安装报错：
could not fetch a Kubernetes version from the internet: unable to get URL “https://dl.k8s.io/release/stable-1.txt”: Get “https://dl.k8s.io/release/stable-1.txt”: dial tcp: lookup dl.k8s.io on 8.8.8.8:53: dial udp 8.8.8.8:53: connect: network is unreachable
W0804 09:38:33.138340 50405 version.go:104] falling back to the local client version: v1.22.10
cannot use “0.0.0.0” as the bind address for the API Server
怎么它还是要找互联网
1 个月 后
willqyK零S
现在新的制品离线安装方式问题很大：
1、组件版本混乱，不知道哪些版本和哪些版本适配
2、自动安装的harbor仓库证书在各个节点缺失，默认没有在/etc/docker/certs.d，要么containerd证书路径空白，导致登陆和push都会报x509
3、制作的离线制品版本不是实际安装时需要的，导致依然去公网拉镜像
- willqyK零S
2、https://kubesphere.io/zh/docs/v3.3/installing-on-linux/introduction/air-gapped-installation/ 这个文档有个问题，certsPath 这个参数不能少，不然containerd中ca配置是空的
registry:
auths:
"dockerhub.kubekey.local":
username: "xxx"
password: "***"
skipTLSVerify: false # Allow contacting registries over HTTPS with failed TLS verification.
plainHTTP: false # Allow contacting registries over HTTP.
certsPath: "/etc/docker/certs.d/dockerhub.kubekey.local" # Use certificates at path (*.crt, *.cert, *.key) to connect to the registry.