创建部署问题时,请参考下面模板,你提供的信息越多,越容易及时获得解答。如果未按模板创建问题,管理员有权关闭问题。
确保帖子格式清晰易读,用 markdown code block 语法格式化代码块。
你只花一分钟创建的问题,不能指望别人花上半个小时给你解答。
操作系统信息
例如:虚拟机/物理机,Centos7.5/Ubuntu18.04,4C/8G
Kubernetes版本信息
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.12", GitCommit:"b058e1760c79f46a834ba59bd7a3486ecf28237d", GitTreeState:"clean", BuildDate:"2022-07-13T14:59:18Z", GoVersion:"go1.16.15", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.12", GitCommit:"b058e1760c79f46a834ba59bd7a3486ecf28237d", GitTreeState:"clean", BuildDate:"2022-07-13T14:53:39Z", GoVersion:"go1.16.15", Compiler:"gc", Platform:"linux/amd64"}
容器运行时
docker version
Client: Docker Engine - Community
Version: 20.10.21
API version: 1.41
Go version: go1.18.7
Git commit: baeda1f
Built: Tue Oct 25 18:04:24 2022
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.21
API version: 1.41 (minimum version 1.12)
Go version: go1.18.7
Git commit: 3056208
Built: Tue Oct 25 18:02:38 2022
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.10
GitCommit: 770bd0108c32f3fb5c73ae1264f7e503fe7b2661
runc:
Version: 1.1.4
GitCommit: v1.1.4-0-g5fd4c4d
docker-init:
Version: 0.19.0
GitCommit: de40ad0
KubeSphere版本信息
v3.4.1
问题描述
一、运行流水线没有反应
二、停止流水线提示Run x not found in organization jenkins and pipeline xxxxx
三、查看devops-jenkins的日志提示Built-In Node went off-line before the task’s worker thread started. No termination trace available.
2024-07-16 08:45:36.196+0000 [id=657] WARNING hudson.model.Executor#resetWorkUnit: Executor #-1 for Built-In Node grabbed hudson.model.queue.WorkUnit@6dc0f274[work=wechat-prod4mrf2 » bsnc-secondary-validation-pc » release-bsnc] from queue but Built-In Node went off-line before the task's worker thread started. No termination trace available.
四、查看devops-controller 报了很多pipelinerun-controller “msg”="get pipelinerun data(id: 2) error with not exit, retry
I0716 08:51:08.545961 1 pipelinerun_controller.go:136] pipelinerun-controller "msg"="get pipelinerun data(id: 3) error with not exit, retry." "Pipeline"="bsnc-secondary-validation-pc" "PipelineRun"={"Namespace":"wechat-prod4mrf2","Name":"bsnc-secondary-validation-pc-n465n"} "namespace"="wechat-prod4mrf2"
五、查看devops-apiserver 有返回404
I0716 08:45:42.399041 1 apiserver.go:325] 10.233.97.82 - "PUT /kapis/devops.kubesphere.io/v1alpha2/devops/wechat-prod4mrf2/pipelines/bsnc-secondary-validation-pc/branches/release-bsnc/runs/3/stop/?blocking=true&timeOutInSecs=10 HTTP/1.1" 500 119 35ms
E0716 08:47:19.263873 1 pure_request.go:92] {
"message" : "Run 3 not found in organization jenkins and pipeline release-bsnc",
"code" : 404,
"errors" : [ ]
}
E0716 08:47:19.263899 1 pipeline.go:472] {
"message" : "Run 3 not found in organization jenkins and pipeline release-bsnc",
"code" : 404,
"errors" : [ ]
}
E0716 08:47:19.263909 1 devops.go:685] {
"message" : "Run 3 not found in organization jenkins and pipeline release-bsnc",
"code" : 404,
"errors" : [ ]
}
E0716 08:47:19.263916 1 devops.go:906] {
"message" : "Run 3 not found in organization jenkins and pipeline release-bsnc",
"code" : 404,
"errors" : [ ]
}
I0716 08:47:19.263939 1 apiserver.go:325] 10.233.99.55 - "PUT /kapis/devops.kubesphere.io/v1alpha2/devops/wechat-prod4mrf2/pipelines/bsnc-secondary-validation-pc/branches/release-bsnc/runs/3/stop/?blocking=true&timeOutInSecs=10 HTTP/1.1" 500 119 12ms
我不知道哪里有问题,尝试重启了devops-apiserver 、devops-controller、devops-jenkins这三个服务,但还是没什么用,最后重启了devops 节点,还是没什么用。最后我试了调整ks-installer 中devops 的内容,试图通过ks-installer 重新安装来解决,但遗憾的是,最后这一板斧还是没用,流水线还是无法正常使用。求各位大佬支支招,还有什么地方可以排查一下。祝各位好心大佬日进百金,年入百万,精神抖擞,长命百岁!