ks-installer镜像构建文件

weiliang-ms · 2021年4月12日

场景需要，需要对一些镜像tag进行更改，
通过构建ks-installer release 3.0下的Dockerfile,构建kubesphere/ks-installer:v3.0.0镜像(未更改任何内容)

使用时，会卡在以下步骤

[root@node1 ~]# kubectl logs -f ks-installer-b5fccd5f4-65bw6 -n kubesphere-system
2021-04-12T15:42:02+08:00 INFO     : shell-operator latest
2021-04-12T15:42:02+08:00 INFO     : HTTP SERVER Listening on 0.0.0.0:9115
2021-04-12T15:42:02+08:00 INFO     : Use temporary dir: /tmp/shell-operator
2021-04-12T15:42:02+08:00 INFO     : Initialize hooks manager ...
2021-04-12T15:42:02+08:00 INFO     : Search and load hooks ...
2021-04-12T15:42:02+08:00 INFO     : Initializing schedule manager ...
2021-04-12T15:42:02+08:00 INFO     : KUBE Init Kubernetes client
2021-04-12T15:42:02+08:00 INFO     : KUBE-INIT Kubernetes client is configured successfully
2021-04-12T15:42:02+08:00 INFO     : MAIN: run main loop
2021-04-12T15:42:02+08:00 INFO     : MAIN: add onStartup tasks
2021-04-12T15:42:02+08:00 INFO     : QUEUE add all HookRun@OnStartup
2021-04-12T15:42:02+08:00 INFO     : MSTOR Create new metric shell_operator_live_ticks
2021-04-12T15:42:02+08:00 INFO     : Running schedule manager ...
2021-04-12T15:42:02+08:00 INFO     : MSTOR Create new metric shell_operator_tasks_queue_length

直接使用kubesphere/ks-installer:v3.0.0 不会出现上述问题。

请问是我执行的构建命令有问题吗？

docker build -t harbor.wl.com/kubesphere/ks-installer:v3.0.0 .

Rick · 2021年4月12日

卡了多久呢？会不会是网络问题，上面没看到什么问题。

weiliang-ms · 2021年4月12日

Rick

INFO[05:38:43 EDT] Installation is complete.

Please check the result using the command:

       kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f

[root@localhost ~]# kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f
2021-04-12T17:33:51+08:00 INFO     : shell-operator latest
2021-04-12T17:33:51+08:00 INFO     : Use temporary dir: /tmp/shell-operator
2021-04-12T17:33:51+08:00 INFO     : Initialize hooks manager ...
2021-04-12T17:33:51+08:00 INFO     : Search and load hooks ...
2021-04-12T17:33:51+08:00 INFO     : HTTP SERVER Listening on 0.0.0.0:9115
2021-04-12T17:33:51+08:00 INFO     : Initializing schedule manager ...
2021-04-12T17:33:51+08:00 INFO     : KUBE Init Kubernetes client
2021-04-12T17:33:51+08:00 INFO     : KUBE-INIT Kubernetes client is configured successfully
2021-04-12T17:33:51+08:00 INFO     : MAIN: run main loop
2021-04-12T17:33:51+08:00 INFO     : MAIN: add onStartup tasks
2021-04-12T17:33:51+08:00 INFO     : QUEUE add all HookRun@OnStartup
2021-04-12T17:33:51+08:00 INFO     : Running schedule manager ...
2021-04-12T17:33:51+08:00 INFO     : MSTOR Create new metric shell_operator_live_ticks
2021-04-12T17:33:51+08:00 INFO     : MSTOR Create new metric shell_operator_tasks_queue_length

目前三分钟，毫无反应，如果使用未修改的镜像，不会出现。

weiliang-ms · 2021年4月12日

weiliang-ms

更改文件为roles/download/defaults/main.yml

yuswift · 2021年4月12日

你这是已经构建好了再去用构建好的镜像装集群会卡住是吗

weiliang-ms · 2021年4月12日

yuswift
不是，重新装（用自己打包的ks-installer安装）也会卡住，build源码是3.0 的

weiliang-ms · 2021年4月12日

请问，docker hub上的3.0镜像也是基于这个3.0构建的吧？

Rick · 2021年4月12日

kubectl get cc -n kubesphere-system ks-installer -oyaml 这个命令的输出是什么

weiliang-ms · 2021年4月12日

Rick

apiVersion: installer.kubesphere.io/v1alpha1
kind: ClusterConfiguration
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"installer.kubesphere.io/v1alpha1","kind":"ClusterConfiguration","metadata":{"annotations":{},"labels":{"version":"v3.0.0"},"name":"ks-installer","namespace":"kubesphere-system"},"spec":{"alerting":{"enabled":true},"auditing":{"enabled":true},"authentication":{"jwtSecret":""},"common":{"es":{"elasticsearchDataVolumeSize":"100Gi","elasticsearchMasterVolumeSize":"4Gi","elkPrefix":"logstash","logMaxAge":30},"etcdVolumeSize":"20Gi","minioVolumeSize":"20Gi","mysqlVolumeSize":"20Gi","openldapVolumeSize":"2Gi","redisVolumSize":"2Gi"},"console":{"enableMultiLogin":false,"port":7777},"devops":{"enabled":true,"jenkinsJavaOpts_MaxRAM":"2g","jenkinsJavaOpts_Xms":"512m","jenkinsJavaOpts_Xmx":"512m","jenkinsMemoryLim":"2Gi","jenkinsMemoryReq":"1500Mi","jenkinsVolumeSize":"8Gi"},"etcd":{"endpointIps":"192.168.1.111,192.168.1.112,192.168.1.113","monitoring":true,"port":2379,"tlsEnable":true},"events":{"enabled":true,"ruler":{"enabled":true,"replicas":2}},"local_registry":"harbor.wl.com","logging":{"enabled":true,"logsidecarReplicas":2},"metrics_server":{"enabled":true},"monitoring":{"prometheusMemoryRequest":"400Mi","prometheusVolumeSize":"20Gi"},"multicluster":{"clusterRole":"none"},"networkpolicy":{"enabled":true},"notification":{"enabled":true},"openpitrix":{"enabled":true},"persistence":{"storageClass":""},"servicemesh":{"enabled":true}}}
  creationTimestamp: "2021-04-12T09:33:34Z"
  generation: 1
  labels:
    version: v3.0.0
  managedFields:
  - apiVersion: installer.kubesphere.io/v1alpha1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:kubectl.kubernetes.io/last-applied-configuration: {}
        f:labels:
          .: {}
          f:version: {}
      f:spec:
        .: {}
        f:alerting:
          .: {}
          f:enabled: {}
        f:auditing:
          .: {}
          f:enabled: {}
        f:authentication:
          .: {}
          f:jwtSecret: {}
        f:common:
          .: {}
          f:es:
            .: {}
            f:elasticsearchDataVolumeSize: {}
            f:elasticsearchMasterVolumeSize: {}
            f:elkPrefix: {}
            f:logMaxAge: {}
          f:etcdVolumeSize: {}
          f:minioVolumeSize: {}
          f:mysqlVolumeSize: {}
          f:openldapVolumeSize: {}
          f:redisVolumSize: {}
        f:console:
          .: {}
          f:enableMultiLogin: {}
          f:port: {}
        f:devops:
          .: {}
          f:enabled: {}
          f:jenkinsJavaOpts_MaxRAM: {}
          f:jenkinsJavaOpts_Xms: {}
          f:jenkinsJavaOpts_Xmx: {}
          f:jenkinsMemoryLim: {}
          f:jenkinsMemoryReq: {}
          f:jenkinsVolumeSize: {}
        f:etcd:
          .: {}
          f:endpointIps: {}
          f:monitoring: {}
          f:port: {}
          f:tlsEnable: {}
        f:events:
          .: {}
          f:enabled: {}
          f:ruler:
            .: {}
            f:enabled: {}
            f:replicas: {}
        f:local_registry: {}
        f:logging:
          .: {}
          f:enabled: {}
          f:logsidecarReplicas: {}
        f:metrics_server:
          .: {}
          f:enabled: {}
        f:monitoring:
          .: {}
          f:prometheusMemoryRequest: {}
          f:prometheusVolumeSize: {}
        f:multicluster:
          .: {}
          f:clusterRole: {}
        f:networkpolicy:
          .: {}
          f:enabled: {}
        f:notification:
          .: {}
          f:enabled: {}
        f:openpitrix:
          .: {}
          f:enabled: {}
        f:persistence:
          .: {}
          f:storageClass: {}
        f:servicemesh:
          .: {}
          f:enabled: {}
    manager: kubectl
    operation: Update
    time: "2021-04-12T09:33:34Z"
  name: ks-installer
  namespace: kubesphere-system
  resourceVersion: "1513"
  selfLink: /apis/installer.kubesphere.io/v1alpha1/namespaces/kubesphere-system/clusterconfigurations/ks-installer
  uid: 2270424d-a11e-4daf-a69c-28c97bc66c49
spec:
  alerting:
    enabled: true
  auditing:
    enabled: true
  authentication:
    jwtSecret: ""
  common:
    es:
      elasticsearchDataVolumeSize: 100Gi
      elasticsearchMasterVolumeSize: 4Gi
      elkPrefix: logstash
      logMaxAge: 30
    etcdVolumeSize: 20Gi
    minioVolumeSize: 20Gi
    mysqlVolumeSize: 20Gi
    openldapVolumeSize: 2Gi
    redisVolumSize: 2Gi
  console:
    enableMultiLogin: false
    port: 7777
  devops:
    enabled: true
    jenkinsJavaOpts_MaxRAM: 2g
    jenkinsJavaOpts_Xms: 512m
    jenkinsJavaOpts_Xmx: 512m
    jenkinsMemoryLim: 2Gi
    jenkinsMemoryReq: 1500Mi
    jenkinsVolumeSize: 8Gi
  etcd:
    endpointIps: 192.168.1.111,192.168.1.112,192.168.1.113
    monitoring: true
    port: 2379
    tlsEnable: true
  events:
    enabled: true
    ruler:
      enabled: true
      replicas: 2
  local_registry: harbor.wl.com
  logging:
    enabled: true
    logsidecarReplicas: 2
  metrics_server:
    enabled: true
  monitoring:
    prometheusMemoryRequest: 400Mi
    prometheusVolumeSize: 20Gi
  multicluster:
    clusterRole: none
  networkpolicy:
    enabled: true
  notification:
    enabled: true
  openpitrix:
    enabled: true
  persistence:
    storageClass: ""
  servicemesh:
    enabled: true

weiliang-ms · 2021年4月12日

上次的安装进程，依然停留在以下输出

[root@node1 ~]# kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f
2021-04-12T17:33:51+08:00 INFO     : shell-operator latest
2021-04-12T17:33:51+08:00 INFO     : Use temporary dir: /tmp/shell-operator
2021-04-12T17:33:51+08:00 INFO     : Initialize hooks manager ...
2021-04-12T17:33:51+08:00 INFO     : Search and load hooks ...
2021-04-12T17:33:51+08:00 INFO     : HTTP SERVER Listening on 0.0.0.0:9115
2021-04-12T17:33:51+08:00 INFO     : Initializing schedule manager ...
2021-04-12T17:33:51+08:00 INFO     : KUBE Init Kubernetes client
2021-04-12T17:33:51+08:00 INFO     : KUBE-INIT Kubernetes client is configured successfully
2021-04-12T17:33:51+08:00 INFO     : MAIN: run main loop
2021-04-12T17:33:51+08:00 INFO     : MAIN: add onStartup tasks
2021-04-12T17:33:51+08:00 INFO     : QUEUE add all HookRun@OnStartup
2021-04-12T17:33:51+08:00 INFO     : Running schedule manager ...
2021-04-12T17:33:51+08:00 INFO     : MSTOR Create new metric shell_operator_live_ticks
2021-04-12T17:33:51+08:00 INFO     : MSTOR Create new metric shell_operator_tasks_queue_length

weiliang-ms · 2021年4月21日

Rick 已解决。

原因为：controller/** 脚本需要授权执行权限。在拉取解压源代码后，脚本文件不具有执行权限，导致shell-operator无法执行hook脚本内容

XXWE · 2021年8月27日

weiliang-ms 请问你是怎么解决的啊，我也遇到了这个问题

Cauchy · 2021年8月27日

XWE

chmod +x  -R  controller/

ks-installer镜像构建文件

weiliang-msK零S

RickK零S

weiliang-msK零S

weiliang-msK零S

yuswiftK零S

weiliang-msK零S

weiliang-msK零S

RickK零S

weiliang-msK零S

weiliang-msK零S

weiliang-msK零S

XXWE

CauchyK零SK壹S