• DevOps
  • devops Run xxx not found in organization jenkins and pipeline xxx

创建部署问题时,请参考下面模板,你提供的信息越多,越容易及时获得解答。如果未按模板创建问题,管理员有权关闭问题。
确保帖子格式清晰易读,用 markdown code block 语法格式化代码块。
你只花一分钟创建的问题,不能指望别人花上半个小时给你解答。

操作系统信息
例如:虚拟机/物理机,Centos7.5/Ubuntu18.04,4C/8G

Kubernetes版本信息

Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.12", GitCommit:"b058e1760c79f46a834ba59bd7a3486ecf28237d", GitTreeState:"clean", BuildDate:"2022-07-13T14:59:18Z", GoVersion:"go1.16.15", Compiler:"gc", Platform:"linux/amd64"}

Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.12", GitCommit:"b058e1760c79f46a834ba59bd7a3486ecf28237d", GitTreeState:"clean", BuildDate:"2022-07-13T14:53:39Z", GoVersion:"go1.16.15", Compiler:"gc", Platform:"linux/amd64"}

容器运行时
docker version

Client: Docker Engine - Community
 Version:           20.10.21
 API version:       1.41
 Go version:        go1.18.7
 Git commit:        baeda1f
 Built:             Tue Oct 25 18:04:24 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.21
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.7
  Git commit:       3056208
  Built:            Tue Oct 25 18:02:38 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.10
  GitCommit:        770bd0108c32f3fb5c73ae1264f7e503fe7b2661
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

KubeSphere版本信息

v3.4.1

问题描述

一、运行流水线没有反应

二、停止流水线提示Run x not found in organization jenkins and pipeline xxxxx

三、查看devops-jenkins的日志提示Built-In Node went off-line before the task’s worker thread started. No termination trace available.

 2024-07-16 08:45:36.196+0000 [id=657]	WARNING	hudson.model.Executor#resetWorkUnit: Executor #-1 for Built-In Node grabbed hudson.model.queue.WorkUnit@6dc0f274[work=wechat-prod4mrf2 » bsnc-secondary-validation-pc » release-bsnc] from queue but Built-In Node went off-line before the task's worker thread started. No termination trace available.

四、查看devops-controller 报了很多pipelinerun-controller “msg”="get pipelinerun data(id: 2) error with not exit, retry

 I0716 08:51:08.545961       1 pipelinerun_controller.go:136] pipelinerun-controller "msg"="get pipelinerun data(id: 3) error with not exit, retry." "Pipeline"="bsnc-secondary-validation-pc" "PipelineRun"={"Namespace":"wechat-prod4mrf2","Name":"bsnc-secondary-validation-pc-n465n"} "namespace"="wechat-prod4mrf2" 

五、查看devops-apiserver 有返回404

 I0716 08:45:42.399041       1 apiserver.go:325] 10.233.97.82 - "PUT /kapis/devops.kubesphere.io/v1alpha2/devops/wechat-prod4mrf2/pipelines/bsnc-secondary-validation-pc/branches/release-bsnc/runs/3/stop/?blocking=true&timeOutInSecs=10 HTTP/1.1" 500 119 35ms

 E0716 08:47:19.263873       1 pure_request.go:92] {

   "message" : "Run 3 not found in organization jenkins and pipeline release-bsnc",

   "code" : 404,

   "errors" : [ ]

 }

 E0716 08:47:19.263899       1 pipeline.go:472] {

   "message" : "Run 3 not found in organization jenkins and pipeline release-bsnc",

   "code" : 404,

   "errors" : [ ]

 }

 E0716 08:47:19.263909       1 devops.go:685] {

   "message" : "Run 3 not found in organization jenkins and pipeline release-bsnc",

   "code" : 404,

   "errors" : [ ]

 }

 E0716 08:47:19.263916       1 devops.go:906] {

   "message" : "Run 3 not found in organization jenkins and pipeline release-bsnc",

   "code" : 404,

   "errors" : [ ]

 }

 I0716 08:47:19.263939       1 apiserver.go:325] 10.233.99.55 - "PUT /kapis/devops.kubesphere.io/v1alpha2/devops/wechat-prod4mrf2/pipelines/bsnc-secondary-validation-pc/branches/release-bsnc/runs/3/stop/?blocking=true&timeOutInSecs=10 HTTP/1.1" 500 119 12ms

我不知道哪里有问题,尝试重启了devops-apiserver 、devops-controller、devops-jenkins这三个服务,但还是没什么用,最后重启了devops 节点,还是没什么用。最后我试了调整ks-installer 中devops 的内容,试图通过ks-installer 重新安装来解决,但遗憾的是,最后这一板斧还是没用,流水线还是无法正常使用。求各位大佬支支招,还有什么地方可以排查一下。祝各位好心大佬日进百金,年入百万,精神抖擞,长命百岁!

emmm,有可能是我磁盘空间不足了,我做了个骚操作,重新挂个盘给devops-jenkins,然后,重新创建项目和流水线,新项目的流水线正常了,但原来项目的流水线GG,慎重 !!!!