零、参考资料及注意

参考资料

https://github.com/kubesphere/porter/blob/master/doc/install-porter-on-kubesphere.md
https://github.com/kubesphere/porter/blob/master/doc/zh/layer2.md
https://github.com/kubesphere/porter/blob/master/doc/use-porter-in-layer-2-mode.md
https://www.bilibili.com/video/BV17K4y177YG?from=search&seid=17868051001232522796

⚠️可在安装一次后再回来看注意事项

1、不要使用ks console 自带应用商店的porter，版本过老
2、重新安装应用porter后，需要手动删除残留，kubectl get crds | grep network.kubesphere.io 看还有没有bgpconfs,bgppeers,eips残留，有的话先删掉，然后再安装
3、porter及EIP配置完成后修改平台项目网关注释，企业空间–》项目—》项目设置—》高级设置–》网关并修改访问方式为LB
lb.kubesphere.io/v1alpha1:porter
protocol.porter.kubesphere.io/v1alpha1:layer2

一、环境说明

基础vms:vmwares
k8s:1.17.1
docker:1.13
ks:v3.0.0
节点网段：172.31.0.0/22
porter：0.2.1 [0.4.1]
porter配置方式：layer2

二、porter安装

安装方式选择

使用ks console 安装

install

https://github.com/kubesphere/porter/blob/master/doc/install-porter-on-kubesphere.md

delete

安装教程删除后，需要清除残留bgpconfs,bgppeers,eips,否则后续EIP无法找到CRD

[root@namenodemaster ~]# kubectl get crds | grep network.kubesphere.io
bgpconfs.network.kubesphere.io                    2021-02-07T09:30:15Z
bgppeers.network.kubesphere.io                    2021-02-07T09:30:15Z
eips.network.kubesphere.io                        2021-02-07T09:30:15Z
namespacenetworkpolicies.network.kubesphere.io    2021-01-28T09:55:13Z
[root@namenodemaster ~]# kubectl delete crds bgpconfs.network.kubesphere.io bgppeers.network.kubesphere.io eips.network.kubesphere.io

三、配置porter-layer2

由于私有环境不支持BGP所以使用layer2

https://github.com/kubesphere/porter/blob/master/doc/zh/layer2.md
https://github.com/kubesphere/porter/blob/master/doc/use-porter-in-layer-2-mode.md

Enable strictARP for kube-proxy

1、kubectl edit configmap kube-proxy -n kube-system
ipvs:
  strictARP: true
2、Run the following command to restart kube-proxy:
kubectl rollout restart daemonset kube-proxy -n kube-system

配置EIP

• 创建layer2 eip–⚠️必须和节点同一网段且未使用的、网卡名称注意修改，如果多个网卡参考上面第二个链接

apiVersion: network.kubesphere.io/v1alpha2
kind: Eip
metadata:
  name: porter-layer2-eip
spec:
  address: 172.31.2.200-172.31.2.210
  interface: ens32
  protocol: layer2

• 创建工作负载

apiVersion: apps/v1
kind: Deployment
metadata:
  name: porter-layer2
spec:
  replicas: 2
  selector:
    matchLabels:
      app: porter-layer2
  template:
    metadata:
      labels:
        app: porter-layer2
    spec:
      containers:
        - image: luksa/kubia
          name: kubia
          ports:
            - containerPort: 8080

• 创建service


kind: Service
apiVersion: v1
metadata:
  name: porter-layer2-svc
  annotations:
    lb.kubesphere.io/v1alpha1: porter
    protocol.porter.kubesphere.io/v1alpha1: layer2
spec:
  selector:
    app: porter-layer2
  type: LoadBalancer
  ports:
    - name: http
      port: 80
      targetPort: 8080
  externalTrafficPolicy: Cluster

验证layer2

[namenodemaster]# k get svc porter-layer2-svc
NAME                TYPE           CLUSTER-IP    EXTERNAL-IP    PORT(S)        AGE
porter-layer2-svc   LoadBalancer   10.1.37.153   172.31.2.200   80:31060/TCP   16h
mac#ping 172.31.2.201
PING 172.31.2.201 (172.31.2.201): 56 data bytes
92 bytes from 172.31.0.234: Time to live exceeded
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 5400 fe20   0 0000  01  01 fc9c 172.31.99.228  172.31.2.201
 
 [root@namenodemaster porter]# k get no -o wide
NAME             STATUS   ROLES    AGE     VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION          CONTAINER-RUNTIME
k8s-node2        Ready    <none>   6d18h   v1.17.1   172.31.0.233   <none>        CentOS Linux 7 (Core)   3.10.0-957.el7.x86_64   docker://1.13.1
k8s-node3        Ready    <none>   6d18h   v1.17.1   172.31.0.234   <none>        CentOS Linux 7 (Core)   3.10.0-957.el7.x86_64   docker://1.13.1
namenodemaster   Ready    master   15d     v1.17.1   172.31.1.37    <none>        CentOS Linux 7 (Core)   3.10.0-957.el7.x86_64   docker://1.13.1
yarnserver       Ready    <none>   15d     v1.17.1   172.31.1.38    <none>        CentOS Linux 7 (Core)   3.10.0-957.el7.x86_64   docker://1.13.1

四、ks 外网访问网关设置

1、给网关添加注释
lb.kubesphere.io/v1alpha1: porter
protocol.porter.kubesphere.io/v1alpha1: layer2
2、正常保存后外网地址会出现任意一个node的ip，如果没有说明有问题

五、创建应用路由验证LB

1、新增应用路由，前提是有svc
2、可以绑定域名到网关地址或者内网dns解析后访问验证