• DevOps
  • devops 自定义jenkins agent node docker build 异常

devops 自定义jenkins agent node 版本18.13.0 打包镜像时报错 docker: not found script returned exit code 127

操作系统信息
Ubuntu22.0.4,4C/8G

Kubernetes版本信息
1.22.12

容器运行时

docekr版本:20.10.9

KubeSphere版本信息
v3.3.1

问题是什么

devops 自定义jenkins agent node 版本18.13.0 docker build时报错 docker: not found script returned exit code 127 我手动build 可以打成镜像 仔细检查过dockerfile的目录结构也没发现什么问题

刚又找了一下原因好像是没找到docker命令 但是这是为什么

[upl-image-preview url=

]

    这个 node18 的 image 是自己构建的吗,那可以 构建这个 image 时把 docker 装进去。

    我们用的node版本比较高内置agent,npm build的时候前端会报错所以自定义了node18,请问我在构建image的时候怎么把docker也装进去有详细文档么?我去试了一下按照nodejs的那种写法装docker没有成功

                - name: "node18"
                namespace: "kubesphere-devops-worker"
    
                label: "node18"
    
                nodeUsageMode: "EXCLUSIVE"
    
                idleMinutes: 0
    
                inheritFrom: "nodejs"
    
                containers:
    
                - name: "nodejs"
    
                  image: "101.43.101.99:7788/node/node:18.13.0"
    
                  command: "cat"
    
                  args: ""
    
                  ttyEnabled: true
    
                  privileged: false
    
                  resourceRequestCpu: "100m"
    
                  resourceLimitCpu: "4000m"
    
                  resourceRequestMemory: "100Mi"
    
                  resourceLimitMemory: "8192Mi"
    
                - name: "jnlp"
    
                  image: "registry.cn-beijing.aliyuncs.com/kubesphereio/inbound-agent:4.10-2"
    
                  args: "^${computer.jnlpmac} ^${computer.name}"
    
                  resourceRequestCpu: "50m"
    
                  resourceLimitCpu: "500m"
    
                  resourceRequestMemory: "400Mi"
    
                  resourceLimitMemory: "1536Mi"
    
                workspaceVolume:
    
                  emptyDirWorkspaceVolume:
    
                    memory: false
    
                volumes:
    
                - hostPathVolume:
    
                    hostPath: "/var/run/docker.sock"
    
                    mountPath: "/var/run/docker.sock"
    
                - hostPathVolume:
    
                    hostPath: "/var/data/jenkins_nodejs_yarn_cache"
    
                    mountPath: "/root/.yarn"
    
                - hostPathVolume:
    
                    hostPath: "/var/data/jenkins_nodejs_npm_cache"
    
                    mountPath: "/root/.npm"
    
                - hostPathVolume:
    
                    hostPath: "/var/data/jenkins_sonar_cache"
    
                    mountPath: "/root/.sonar/cache"
    
                yaml: |
    
                  spec:
    
                    affinity:
    
                      nodeAffinity:
    
                        preferredDuringSchedulingIgnoredDuringExecution:
    
                        - weight: 1
    
                          preference:
    
                            matchExpressions:
    
                            - key: node-role.kubernetes.io/worker
    
                              operator: In
    
                              values:
    
                              - ci
    
                    tolerations:
    
                    - key: "node.kubernetes.io/ci"
    
                      operator: "Exists"
    
                      effect: "NoSchedule"
    
                    - key: "node.kubernetes.io/ci"
    
                      operator: "Exists"
    
                      effect: "PreferNoSchedule"
    
                    containers:
    
                    - name: "node18"
    
                      resources:
    
                        requests:
    
                          ephemeral-storage: "1Gi"
    
                        limits:
    
                          ephemeral-storage: "10Gi"
    
                    securityContext:
    
                      fsGroup: 1000

    chilianyi 这个 链接 里有 安装 docker 的命令,可以参考下。

    或者就是 构建 image 时,from 已有的 base image,比如:kubesphere/builder-base:v3.2.2

    谢谢那个问题解决了 但是k8s构建镜像的时候 报错envsubst: not found + kubectl apply -f -是为什么 我手动用kubectl命令是能构建的

    [upl-image-preview url=

    ]

    以下是我构建node18镜像时候的脚本

    Dockerfile:

    FROM node:18.13.0

    LABEL maintainer=“Bibin Wilson bibinwilsonn@gmail.com

    COPY ./test.sh /data/nodes/test.sh

    RUN /data/nodes/test.sh

    CMD [“docker”,“version”]

    shell:

    curl -f https://download.docker.com/linux/static/stable/x86_64/docker-18.09.9.tgz | tar xvz && \

    mv docker/docker /usr/bin/ && \

    rm -rf docker

    curl -f https://get.helm.sh/helm-v2.11.0-linux-amd64.tar.gz | tar xzv && \

    mv linux-amd64/helm /usr/bin/ && \

    mv linux-amd64/tiller /usr/bin/ && \

    rm -rf linux-amd64

    curl -f https://get.helm.sh/helm-v3.5.0-linux-amd64.tar.gz | tar xzv && \

    mv linux-amd64/helm /usr/bin/helm3 && \

    rm -rf linux-amd64

    curl -f -LO https://storage.googleapis.com/kubernetes-release/release/v1.26.1/bin/linux/amd64/kubectl && \

    chmod +x kubectl && \

    mv kubectl /usr/bin/

    curl -fL https://github.com/kubesphere-sigs/ks/releases/download/v0.0.64/ks-linux-amd64.tar.gz | tar xzv && \

    mv ks /usr/bin/

    curl -fL https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize%2Fv4.5.3/kustomize_v4.5.3_linux_amd64.tar.gz | tar xzv && \

    mv kustomize /usr/bin/

      chilianyi 基于nodejs 去构建的 需要基于base 么 如果基于base的话那我还可以使用nodejs的功能么

      chilianyi 我刚试了一下基于base自定义node18的镜像还是不

      [upl-image-preview url=

      ]

      构建镜像脚本:

      # docker run -it kubespheredev/builder-base:v3.1.0 envsubst -V
      envsubst (GNU gettext-runtime) 0.19.8.1
      Copyright (C) 2003-2007 Free Software Foundation, Inc.
      License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
      Written by Bruno Haible.

      我试了下,原生带的。 你用这个 配置,casc 对应 configmap 里找到对应的 image id,进去看下,有 envsubst 这个 命令吗

        qweqe417 我试了下这个 image,是没有装 envsubst

        docker run --network=host -it node:18.13.0 bash
        
        root@node1:/# envsubst
        bash: envsubst: command not found
        root@node1:/# apt-get update
        Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
        Get:2 http://deb.debian.org/debian-security bullseye-security InRelease [48.4 kB]
        Get:3 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
        Get:4 http://deb.debian.org/debian bullseye/main amd64 Packages [8183 kB]
        Get:5 http://deb.debian.org/debian-security bullseye-security/main amd64 Packages [228 kB]
        Get:6 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [14.6 kB]
        Fetched 8634 kB in 10s (833 kB/s)
        Reading package lists... Done
        root@node1:/# apt install gettext-base
        Reading package lists... Done
        Building dependency tree... Done
        Reading state information... Done
        The following NEW packages will be installed:
          gettext-base
        0 upgraded, 1 newly installed, 0 to remove and 28 not upgraded.
        Need to get 175 kB of archives.
        After this operation, 675 kB of additional disk space will be used.
        Get:1 http://deb.debian.org/debian bullseye/main amd64 gettext-base amd64 0.21-4 [175 kB]
        Fetched 175 kB in 1s (128 kB/s)
        debconf: delaying package configuration, since apt-utils is not installed
        Selecting previously unselected package gettext-base.
        (Reading database ... 22790 files and directories currently installed.)
        Preparing to unpack .../gettext-base_0.21-4_amd64.deb ...
        Unpacking gettext-base (0.21-4) ...
        Setting up gettext-base (0.21-4) ...
        root@node1:/# env
        env       envsubst
        root@node1:/# envsubst -V
        envsubst (GNU gettext-runtime) 0.21
        Copyright (C) 2003-2020 Free Software Foundation, Inc.
        License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>
        This is free software: you are free to change and redistribute it.
        There is NO WARRANTY, to the extent permitted by law.
        Written by Bruno Haible.

        你可以试试在 Dockerfile 中,装上这个 gettext-base

          chilianyi 谢谢指点 解决了,不过有个建议 如果您是官方人员建议把自定义agent 的那个文档补充完整一些,按照你们现在那个文档来,自定义镜像那块好多不确定的点,我一路踩了好多坑。

            1 年 后

            yongpanyan 是否要基于 base 镜像,可以自己决定;base 镜像安装了一些命令,如果不基于 base 镜像,自己需要的命令需要在自定义的镜像里安装;

            1 个月 后

            商业产品与合作咨询