Kubersphere 3.0 Harbor仓库连接配置

freemankevin · 2020年11月28日

整体梳理了下全过程，感谢官网各位老师的支持和帮助。

折腾了好久，最关键的还是harbor官网那边给的方案让我重新检查自己的各个部署环节。

版本情况：
harbor2.0
ks3.0
k8s17.9
centos7.8
docker18.06

CA工具
cfssl

1.harbor使用域名（这是我们这边的需求，用IP也可以，尝试报错过不再测试）
1）配置

hostname: harbor.dockerregistry.com

2）hosts
/etc/hosts

3）要使配置完全更新（我应该是在这个地方翻车的，因为还要搞其它的东西，有机会再复现一下）
./prepare（官网做法）

之前是安装了其他插件所以直接附带参数
./prepare –…

2.harbor使用http
1）配置

# http related config
http:

  port: 80

# https related config
#https:
  # https port for harbor, default is 443
#  port: 443
  # The path of cert and key files for nginx
#  certificate: /data/harbor/ssl/harbor.dockerregistry.com.pem
#  private_key: /data/harbor/ssl/harbor.dockerregistry.com-key.pem

3.docker服务和客户端要都做非http可信：insecure-registries
1）配置

[root@cicd-jgh harbor]# cat /etc/docker/daemon.json
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "registry-mirrors": ["https://pfei7wep.mirror.aliyuncs.com"],
  "insecure-registries": ["harbor.dockerregistry.com"],
  "log-opts":{ "max-size" :"10m","max-file":"10"},
  "log-driver": "json-file"
}
[root@cicd-jgh harbor]#

4.ks面板的harbor密钥使用域名方式

5.corends添加hosts映射，即那个域名和ip的关系映射

有其它的dns服务应该也要加（这里是看其他帖子来的，我这边暂时没有）
注意这个yaml位置的空格

kubectl -n kube-system get  cm coredns -o yaml

kubectl -n kube-system edit cm coredns

...
        hosts{
           ...
           10.0.1.185  harbor.dockerregistry.com
           fallthrough
        }
...


[root@master1 ~]# kubectl delete --force -n kube-system  pod  coredns-7f9c544f75-vqxxr
warning: --force is ignored because --grace-period is not 0.
pod "coredns-7f9c544f75-vqxxr" deleted
[root@master1 ~]# kubectl delete --force -n kube-system  pod  coredns-7f9c544f75-h9qhr
warning: --force is ignored because --grace-period is not 0.
pod "coredns-7f9c544f75-h9qhr" deleted




[root@master1 ~]# kubectl get pod -A |grep core
kube-system                    coredns-7f9c544f75-sf7lh                                          1/1     Running     0          109s
kube-system                    coredns-7f9c544f75-vspcs                                          1/1     Running     0          2m5s
[root@master1 ~]#
[root@master1 ~]#
[root@master1 ~]#

#  10.96.0.10  是上面修改coredns看到的forward时的dns-IP 
[root@master1 ~]# nslookup node1 -n 10.96.0.10
*** Invalid option: n
Server:		10.96.0.10
Address:	10.96.0.10#53

Name:	node1
Address: 10.0.1.224
** server can't find node1: NXDOMAIN

[root@master1 ~]# nslookup node2 -n 10.96.0.10
*** Invalid option: n
Server:		10.96.0.10
Address:	10.96.0.10#53

Name:	node2
Address: 10.0.1.225
** server can't find node2: NXDOMAIN

[root@master1 ~]# nslookup harbor.dockerregistry.com -n 10.96.0.10
*** Invalid option: n
Server:		10.96.0.10
Address:	10.96.0.10#53

Name:	harbor.dockerregistry.com
Address: 10.0.1.185
** server can't find harbor.dockerregistry.com: NXDOMAIN

6.最后确认下，harbor状态正常，docker客户端登陆正常，docker客户端使用harbor的一键拉取命令正常

服务端

[root@cicd-jgh harbor]# docker-compose ps
      Name                     Command                  State                 Ports
---------------------------------------------------------------------------------------------
harbor-core         /harbor/entrypoint.sh            Up (healthy)
harbor-db           /docker-entrypoint.sh            Up (healthy)   5432/tcp
harbor-jobservice   /harbor/entrypoint.sh            Up (healthy)
harbor-log          /bin/sh -c /usr/local/bin/ ...   Up (healthy)   127.0.0.1:1514->10514/tcp
harbor-portal       nginx -g daemon off;             Up (healthy)   8080/tcp
nginx               nginx -g daemon off;             Up (healthy)   0.0.0.0:80->8080/tcp
redis               redis-server /etc/redis.conf     Up (healthy)   6379/tcp
registry            /home/harbor/entrypoint.sh       Up (healthy)   5000/tcp
registryctl         /home/harbor/start.sh            Up (healthy)
[root@cicd-jgh harbor]#
[root@cicd-jgh harbor]#
[root@cicd-jgh harbor]#
[root@cicd-jgh harbor]# docker login harbor.dockerregistry.com
Username (admin): admin
Password:
Login Succeeded

客户端

[root@master1 ~]# docker login harbor.dockerregistry.com
Authenticating with existing credentials...
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
[root@master1 ~]#
[root@node5 ~]# docker pull harbor.dockerregistry.com/app/mysql@sha256:9e902247f80eca2e0fc4179f48a91afbe3b2a61122dd5e7575b5f0549aab97d0
sha256:9e902247f80eca2e0fc4179f48a91afbe3b2a61122dd5e7575b5f0549aab97d0: Pulling from app/mysql
d121f8d1c412: Pull complete
f3cebc0b4691: Pull complete
1862755a0b37: Pull complete
489b44f3dbb4: Pull complete
690874f836db: Pull complete
baa8be383ffb: Pull complete
55356608b4ac: Pull complete
277d8f888368: Pull complete
21f2da6feb67: Extracting [==============================>                    ]  66.85MB/108.3MB
2c98f818bcb9: Download complete
031b0a770162: Download complete

7.注意找寻镜像文件应加对应标签

8.最后附上重要的几份文档，不一定完全对，结合着看（可嫩需要科学上网）

1）添加 Harbor 镜像仓库
https://v2-1.docs.kubesphere.io/docs/zh-CN/configuration/image-registry/#%E6%B7%BB%E5%8A%A0-harbor-%E9%95%9C%E5%83%8F%E4%BB%93%E5%BA%93

2）生成证书颁发机构证书
https://goharbor.io/docs/1.10/install-config/configure-https/

3）对HTTPS连接进行故障排除
https://goharbor.io/docs/1.10/install-config/troubleshoot-installation/#https

以上，感谢

PS:
以上是基于域名的，改成实际IP反而会报异常，换回域名就正常

Kubersphere 3.0 Harbor仓库连接配置

freemankevinK零S

freemankevinK零S