在jenkins执行这步骤的时候报错了kubernetesDeploy configs: ‘deploy3.yaml’, kubeconfigId: ‘fccc3a6a-5707-4dc0-b692-9591b027a4c0′

错误日志:
[Pipeline] kubernetesDeploy
Starting Kubernetes deployment
ERROR: ERROR: java.lang.RuntimeException: io.kubernetes.client.openapi.ApiException: okhttp3.internal.http2.ConnectionShutdownException
hudson.remoting.ProxyException: java.lang.RuntimeException: io.kubernetes.client.openapi.ApiException: okhttp3.internal.http2.ConnectionShutdownException
at com.microsoft.jenkins.kubernetes.wrapper.ResourceManager.handleApiExceptionExceptNotFound(ResourceManager.java:180)

。。。。。

我感觉是凭证的问题,凭证是复制.kube/config的内容。
以下是我创建的yaml一个非常简单的案例:
kind: Service
apiVersion: v1
metadata:
name: web
spec:
type: NodePort
selector:
app: tpro
ports:
- protocol: TCP
port: 8080
targetPort: 8080
nodePort: 30666

完整的日志如下:
Started by GitLab push by Administrator
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] podTemplate
[Pipeline] {
[Pipeline] node
Still waiting to schedule task
‘Jenkins’ doesn’t have label ‘jenkins-agent’
Created Pod: ns-jk/jenkins-agent-lgmj7-b3kpf
[Normal][ns-jk/jenkins-agent-lgmj7-b3kpf][Scheduled] Successfully assigned ns-jk/jenkins-agent-lgmj7-b3kpf to jk02
[Normal][ns-jk/jenkins-agent-lgmj7-b3kpf][Pulled] Container image “jk03.harbor.com/jenkinsci/jenkins-slave-jdk:1.8″ already present on machine
[Normal][ns-jk/jenkins-agent-lgmj7-b3kpf][Created] Created container jnlp
[Normal][ns-jk/jenkins-agent-lgmj7-b3kpf][Started] Started container jnlp

Agent jenkins-agent-lgmj7-b3kpf is provisioned from template jenkins-agent-lgmj7

apiVersion: “v1″
kind: “Pod”
metadata:
annotations:
buildUrl: “http://192.168.60.175:30006/job/gitlab-pipe/120/
runUrl: “job/gitlab-pipe/120/”
labels:
jenkins: “slave”
jenkins/label: “jenkins-agent”
name: “jenkins-agent-lgmj7-b3kpf”
spec:
containers:

  • env:
    • name: “JENKINS_SECRET”
      value: “********”
    • name: “JENKINS_AGENT_NAME”
      value: “jenkins-agent-lgmj7-b3kpf”
    • name: “JENKINS_NAME”
      value: “jenkins-agent-lgmj7-b3kpf”
    • name: “JENKINS_AGENT_WORKDIR”
      value: “/home/jenkins/agent”
    • name: “JENKINS_URL”
      value: “http://192.168.60.175:30006/
      image: “jk03.harbor.com/jenkinsci/jenkins-slave-jdk:1.8″
      imagePullPolicy: “IfNotPresent”
      name: “jnlp”
      resources:
      limits: {}
      requests: {}
      tty: false
      volumeMounts:
    • mountPath: “/var/run/docker.sock”
      name: “volume-0″
      readOnly: false
    • mountPath: “/usr/bin/docker”
      name: “volume-1”
      readOnly: false
    • mountPath: “/home/jenkins/agent”
      name: “workspace-volume”
      readOnly: false
      nodeSelector:
      kubernetes.io/os: “linux”
      restartPolicy: “Never”
      volumes:
  • hostPath:
    path: “/var/run/docker.sock”
    name: “volume-0″
  • hostPath:
    path: “/usr/bin/docker”
    name: “volume-1″
  • emptyDir:
    medium: ""
    name: “workspace-volume”

Running on jenkins-agent-lgmj7-b3kpf in /home/jenkins/agent/workspace/gitlab-pipe
[Pipeline] {
[Pipeline] stage
[Pipeline] { (拉取代码)
[Pipeline] checkout
using credential 93c21173-b58e-4a92-93bc-a465a96b62d2
Cloning the remote Git repository
Cloning repository http://192.168.60.176:10080/root/tpro.git

git init /home/jenkins/agent/workspace/gitlab-pipe # timeout=10
Fetching upstream changes from http://192.168.60.176:10080/root/tpro.git
git –version # timeout=10
using GIT_ASKPASS to set credentials gitlab-auth
git fetch –tags –progress http://192.168.60.176:10080/root/tpro.git +refs/heads/:refs/remotes/origin/ # timeout=10
skipping resolution of commit remotes/origin/master, since it originates from another repository
git config remote.origin.url http://192.168.60.176:10080/root/tpro.git # timeout=10
git config –add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10
git config remote.origin.url http://192.168.60.176:10080/root/tpro.git # timeout=10
Fetching upstream changes from http://192.168.60.176:10080/root/tpro.git
using GIT_ASKPASS to set credentials gitlab-auth
git fetch –tags –progress http://192.168.60.176:10080/root/tpro.git +refs/heads/:refs/remotes/origin/ # timeout=10
git rev-parse refs/remotes/origin/master{commit} # timeout=10
Checking out Revision 01f685e4279472d4acdcf87f97ca4a9c22623377 (refs/remotes/origin/master)
git rev-parse refs/remotes/origin/origin/master{commit} # timeout=10
git config core.sparsecheckout # timeout=10
git checkout -f 01f685e4279472d4acdcf87f97ca4a9c22623377 # timeout=10
Commit message: “Fri Jul 31 16:07:07 CST 2020″
git rev-list –no-walk 36f6e5a82be0c5ea8206d2297ef0d9fecd74c108 # timeout=10
[Pipeline] sh

  • ls
    Demo.iml
    Dockerfile
    deploy.yaml
    deploy2.yaml
    deploy3.yaml
    out
    pod-slave.log
    pom.xml
    server.xml
    src
    target
    web
    [Pipeline] }
    [Pipeline] // stage
    [Pipeline] stage
    [Pipeline] { (代码编译)
    [Pipeline] sh
  • mvn clean package -Dmaven.test.skip=true
    [INFO] Scanning for projects…
    [INFO]

    [INFO] ————————————————————————
    [INFO] Building Demo Maven Webapp 1.0-SNAPSHOT
    [INFO] ————————————————————————
    。。。。。。。。。。。。。。。。。。
    [INFO] Packaging webapp
    [INFO] Assembling webapp [Demo] in [/home/jenkins/agent/workspace/gitlab-pipe/target/Demo]
    [INFO] Processing war project
    [INFO] Copying webapp resources [/home/jenkins/agent/workspace/gitlab-pipe/src/main/webapp]
    [INFO] Webapp assembled in [17 msecs]
    [INFO] Building war: /home/jenkins/agent/workspace/gitlab-pipe/target/Demo.war
    [INFO] ————————————————————————
    [INFO] BUILD SUCCESS
    [INFO] ————————————————————————
    [INFO] Total time: 1:06.075s
    [INFO] Finished at: Fri Jul 31 08:10:38 UTC 2020
    [INFO] Final Memory: 16M/107M
    [INFO] ————————————————————————
    [Pipeline] sh
  • ls
    Demo.iml
    Dockerfile
    deploy.yaml
    deploy2.yaml
    deploy3.yaml
    out
    pod-slave.log
    pom.xml
    server.xml
    src
    target
    web
    [Pipeline] }
    [Pipeline] // stage
    [Pipeline] stage
    [Pipeline] { (构建镜像)
    [Pipeline] withCredentials
    Masking supported pattern matches of $username or $password
    [Pipeline] {
    [Pipeline] sh
  • echo '
    FROM jk03.harbor.com/tools/tomcat:1.8
    LABEL maintainer sunxq
    COPY server.xml /opt/tomcat/conf/
    RUN rm -rf /opt/tomcat/webapps/*
    ADD target/Demo.war /opt/tomcat/webapps/Demo.war
    '
  • docker build -t jk03.harbor.com/jenkinsci/tomcat:120 .
    Sending build context to Docker daemon 203.5MB

Step ⅕ : FROM jk03.harbor.com/tools/tomcat:1.8
—> 98801f0967f1
Step ⅖ : LABEL maintainer sunxq
—> Using cache
—> 0672ceb6818c
Step ⅗ : COPY server.xml /opt/tomcat/conf/
—> Using cache
—> ba89af3f319e
Step ⅘ : RUN rm -rf /opt/tomcat/webapps/*
—> Using cache
—> 856b27eda474
Step 5/5 : ADD target/Demo.war /opt/tomcat/webapps/Demo.war
—> 7ea256dc3260
Successfully built 7ea256dc3260
Successfully tagged jk03.harbor.com/jenkinsci/tomcat:120

Login Succeeded

  • docker push jk03.harbor.com/jenkinsci/tomcat:120
    The push refers to repository [jk03.harbor.com/jenkinsci/tomcat]
    fddf9f986fd6: Preparing
    8c437460b125: Preparing
    7edc61713ad5: Preparing
    1a9bb4cac20e: Preparing
    3300dc670908: Preparing
    dcf4fb859e21: Preparing
    837b88776346: Preparing
    09db62a6cf73: Preparing
    edf3aa290fb3: Preparing
    dcf4fb859e21: Waiting
    837b88776346: Waiting
    09db62a6cf73: Waiting
    edf3aa290fb3: Waiting
    8c437460b125: Layer already exists
    3300dc670908: Layer already exists
    7edc61713ad5: Layer already exists
    1a9bb4cac20e: Layer already exists
    09db62a6cf73: Layer already exists
    dcf4fb859e21: Layer already exists
    edf3aa290fb3: Layer already exists
    837b88776346: Layer already exists
    fddf9f986fd6: Pushed
    120: digest: sha256:8c259a0947ef82a2732d4f2aba0c9cb7cd87579c57abba49b3fb57bbbb9a972d size: 2201
    [Pipeline] }
    [Pipeline] // withCredentials
    [Pipeline] }
    [Pipeline] // stage
    [Pipeline] stage
    [Pipeline] { (部署到K8s)
    [Pipeline] sh
  • sed -i ‘s#$NODE_PORT#30666#’ deploy3.yaml
    [Pipeline] kubernetesDeploy
    Starting Kubernetes deployment
    ERROR: ERROR: java.lang.RuntimeException: io.kubernetes.client.openapi.ApiException: okhttp3.internal.http2.ConnectionShutdownException
    hudson.remoting.ProxyException: java.lang.RuntimeException: io.kubernetes.client.openapi.ApiException: okhttp3.internal.http2.ConnectionShutdownException
    at com.microsoft.jenkins.kubernetes.wrapper.ResourceManager.handleApiExceptionExceptNotFound(ResourceManager.java:180)
    at com.microsoft.jenkins.kubernetes.wrapper.V1ResourceManager$ServiceUpdater.getCurrentResource(V1ResourceManager.java:391)
    at com.microsoft.jenkins.kubernetes.wrapper.V1ResourceManager$ServiceUpdater.getCurrentResource(V1ResourceManager.java:379)
    at com.microsoft.jenkins.kubernetes.wrapper.ResourceManager$ResourceUpdater.createOrApply(ResourceManager.java:93)
    at com.microsoft.jenkins.kubernetes.wrapper.KubernetesClientWrapper.handleResource(KubernetesClientWrapper.java:289)
    at com.microsoft.jenkins.kubernetes.wrapper.KubernetesClientWrapper.apply(KubernetesClientWrapper.java:256)
    at com.microsoft.jenkins.kubernetes.command.DeploymentCommand$DeploymentTask.doCall(DeploymentCommand.java:172)
    at com.microsoft.jenkins.kubernetes.command.DeploymentCommand$DeploymentTask.call(DeploymentCommand.java:124)
    at com.microsoft.jenkins.kubernetes.command.DeploymentCommand$DeploymentTask.call(DeploymentCommand.java:106)
    at hudson.remoting.UserRequest.perform(UserRequest.java:211)
    at hudson.remoting.UserRequest.perform(UserRequest.java:54)
    at hudson.remoting.Request$2.run(Request.java:369)
    at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:117)
    at java.lang.Thread.run(Thread.java:748)
    Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from 10.244.1.⅒.244.1.1:51257
    at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
    at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
    at hudson.remoting.Channel.call(Channel.java:998)
    at hudson.FilePath.act(FilePath.java:1159)
    at com.microsoft.jenkins.kubernetes.command.DeploymentCommand.execute(DeploymentCommand.java:68)
    at com.microsoft.jenkins.kubernetes.command.DeploymentCommand.execute(DeploymentCommand.java:45)
    at com.microsoft.jenkins.azurecommons.command.CommandService.runCommand(CommandService.java:88)
    at com.microsoft.jenkins.azurecommons.command.CommandService.execute(CommandService.java:96)
    at com.microsoft.jenkins.azurecommons.command.CommandService.executeCommands(CommandService.java:75)
    at com.microsoft.jenkins.azurecommons.command.BaseCommandContext.executeCommands(BaseCommandContext.java:77)
    at com.microsoft.jenkins.kubernetes.KubernetesDeploy.perform(KubernetesDeploy.java:42)
    at com.microsoft.jenkins.azurecommons.command.SimpleBuildStepExecution.run(SimpleBuildStepExecution.java:54)
    at com.microsoft.jenkins.azurecommons.command.SimpleBuildStepExecution.run(SimpleBuildStepExecution.java:35)
    at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    … 1 more
    Caused by: hudson.remoting.ProxyException: io.kubernetes.client.openapi.ApiException: okhttp3.internal.http2.ConnectionShutdownException
    at io.kubernetes.client.openapi.ApiClient.execute(ApiClient.java:898)
    at io.kubernetes.client.openapi.apis.CoreV1Api.readNamespacedServiceWithHttpInfo(CoreV1Api.java:26889)
    at io.kubernetes.client.openapi.apis.CoreV1Api.readNamespacedService(CoreV1Api.java:26865)
    at com.microsoft.jenkins.kubernetes.wrapper.V1ResourceManager$ServiceUpdater.getCurrentResource(V1ResourceManager.java:388)
    … 16 more
    Caused by: hudson.remoting.ProxyException: okhttp3.internal.http2.ConnectionShutdownException
    at okhttp3.internal.http2.Http2Connection.newStream(Http2Connection.java:242)
    at okhttp3.internal.http2.Http2Connection.newStream(Http2Connection.java:225)
    at okhttp3.internal.http2.Http2ExchangeCodec.writeRequestHeaders(Http2ExchangeCodec.java:116)
    at okhttp3.internal.connection.Exchange.writeRequestHeaders(Exchange.java:72)
    at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:43)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
    at io.kubernetes.client.openapi.ApiClient$2.intercept(ApiClient.java:1232)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:43)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
    at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:94)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
    at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:88)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
    at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:221)
    at okhttp3.RealCall.execute(RealCall.java:81)
    at io.kubernetes.client.openapi.ApiClient.execute(ApiClient.java:894)
    … 19 more
    Loading configuration: /home/jenkins/agent/workspace/gitlab-pipe/deploy3.yaml
    Api call failed with code 0, detailed message: null
    [Pipeline] }
    [Pipeline] // stage
    [Pipeline] }
    [Pipeline] // node
    [Pipeline] }
    [Pipeline] // podTemplate
    [Pipeline] End of Pipeline
    ERROR: Kubernetes deployment ended with HasError
    Finished: FAILURE