流水线部署devops-java-sample/Jenkinsfile-online下的deploy to dev动作报错

xuanying

在用devops工程测试devops-java-sample/Jenkinsfile-online时，在做到deploy to dev动作时出错
stage('deploy to dev') { when{ branch 'master' } steps { input(id: 'deploy-to-dev', message: 'deploy to dev?') kubernetesDeploy(configs: 'deploy/dev-ol/**', enableConfigSubstitution: true, kubeconfigId: "$KUBECONFIG_CREDENTIAL_ID") } }流水线在做到这个动作时报错。我的运行环境时aws的eks
报错如下：
Starting Kubernetes deployment Loading configuration: /home/jenkins/agent/workspace/java-testl2rnf_5454_master/deploy/dev-ol/devops-sample.yaml ERROR: ERROR: java.lang.RuntimeException: io.kubernetes.client.openapi.ApiException: Unauthorized hudson.remoting.ProxyException: java.lang.RuntimeException: io.kubernetes.client.openapi.ApiException: Unauthorized at com.microsoft.jenkins.kubernetes.wrapper.ResourceManager.handleApiExceptionExceptNotFound(ResourceManager.java:180) at com.microsoft.jenkins.kubernetes.wrapper.V1ResourceManager$DeploymentUpdater.getCurrentResource(V1ResourceManager.java:213) at com.microsoft.jenkins.kubernetes.wrapper.V1ResourceManager$DeploymentUpdater.getCurrentResource(V1ResourceManager.java:201) at com.microsoft.jenkins.kubernetes.wrapper.ResourceManager$ResourceUpdater.createOrApply(ResourceManager.java:93) at com.microsoft.jenkins.kubernetes.wrapper.KubernetesClientWrapper.handleResource(KubernetesClientWrapper.java:289) at com.microsoft.jenkins.kubernetes.wrapper.KubernetesClientWrapper.apply(KubernetesClientWrapper.java:256) at com.microsoft.jenkins.kubernetes.command.DeploymentCommand$DeploymentTask.doCall(DeploymentCommand.java:172) at com.microsoft.jenkins.kubernetes.command.DeploymentCommand$DeploymentTask.call(DeploymentCommand.java:124) at com.microsoft.jenkins.kubernetes.command.DeploymentCommand$DeploymentTask.call(DeploymentCommand.java:106) at hudson.remoting.UserRequest.perform(UserRequest.java:212) at hudson.remoting.UserRequest.perform(UserRequest.java:54) at hudson.remoting.Request$2.run(Request.java:369) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93) at java.lang.Thread.run(Thread.java:748) Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from ip-172-20-0-204.ap-east-1.compute.internal/172.20.0.204:39526 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800) at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357) at hudson.remoting.Channel.call(Channel.java:1001) at hudson.FilePath.act(FilePath.java:1160) at com.microsoft.jenkins.kubernetes.command.DeploymentCommand.execute(DeploymentCommand.java:68) at com.microsoft.jenkins.kubernetes.command.DeploymentCommand.execute(DeploymentCommand.java:45) at com.microsoft.jenkins.azurecommons.command.CommandService.runCommand(CommandService.java:88) at com.microsoft.jenkins.azurecommons.command.CommandService.execute(CommandService.java:96) at com.microsoft.jenkins.azurecommons.command.CommandService.executeCommands(CommandService.java:75) at com.microsoft.jenkins.azurecommons.command.BaseCommandContext.executeCommands(BaseCommandContext.java:77) at com.microsoft.jenkins.kubernetes.KubernetesDeploy.perform(KubernetesDeploy.java:42) at com.microsoft.jenkins.azurecommons.command.SimpleBuildStepExecution.run(SimpleBuildStepExecution.java:54) at com.microsoft.jenkins.azurecommons.command.SimpleBuildStepExecution.run(SimpleBuildStepExecution.java:35) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 more Caused by: hudson.remoting.ProxyException: io.kubernetes.client.openapi.ApiException: Unauthorized at io.kubernetes.client.openapi.ApiClient.handleResponse(ApiClient.java:979) at io.kubernetes.client.openapi.ApiClient.execute(ApiClient.java:895) at io.kubernetes.client.openapi.apis.AppsV1Api.readNamespacedDeploymentWithHttpInfo(AppsV1Api.java:7299) at io.kubernetes.client.openapi.apis.AppsV1Api.readNamespacedDeployment(AppsV1Api.java:7275) at com.microsoft.jenkins.kubernetes.wrapper.V1ResourceManager$DeploymentUpdater.getCurrentResource(V1ResourceManager.java:210) ... 16 more Kubernetes deployment ended with HasError
望指导一下

hongming

xuanying 可以看看这个 issue , eks 环境不支持客户端证书认证，可以用 serviceaccount token 代替，3.1 也支持 serviceaccount 管理

kubesphere/kubesphere#3197

xuanying

hongming https://kubesphere.io/zh/docs/faq/devops/create-devops-kubeconfig-on-aws/ 按照这个文档来替换了kubeconfig之后，deploy to dev 报错：Starting Kubernetes deployment
ERROR: ERROR: java.lang.RuntimeException: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Empty input
hudson.remoting.ProxyException: java.lang.RuntimeException: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Empty input

hongming

xuanying 应该是你 kubeconfig 的格式不对，你可以贴出来看看

xuanying

hongming 请帮忙看看：

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EVXhNREEyTWpNd01Wb1hEVE14TURVd09EQTJNak13TVZvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTXJ3CnBIa2hkd3VzWTVTMG5RdGNmazhxNXpBMXdQbnE1UlVIUlZZY0dOSW9SYVpHWVFEV0Rja21UTWIxdDFpYUdRRFoKSld5N1UxWVczOHZPR0lFSUJIczMwVlV2NjlNbHNsN2tMN21HN3FsMllodFpwOGd0MEJJWjh5YS9qZTkyVFc1awpxMGJjZkZEK0Z3b0ZFeXp4Y2pPVnZkei9yTTlXZndJL3htbG9lRWZ4N04xdFBpa0EveVNSNFJmMDZLUGVDeTV0CjIwUFZzaW94ckZGWkszNmhrVkJwSHlabThmbU42d1gwR1VYVFUyTThsU2duMjFKVXFYL21JOE44ek5LdUVZREMKWGg4YWVuK2JGSnlxY0o3OFNqMjdZTlZodmJOSUd6L2VONWllbGNPQ1FzUTYxa3VjRGpEeGFWWGlsRVM4NnBmNwpoM2VIWEN6RzY2SUhPNFlkNUIwQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZKQ1p0WTMwMHZ3NzR6amFyVG5Sc3U5SmZGem9NQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFDeHlySXdGc1F5dVpRM3QvZjVESUpHZUltazhMS05XTzEycFZVdTI3cjk5UVlDL2xvbgpaMmVnNzRvRWhiWnJnSHNZNWc3RzlUYUdDb2FQYkZJWUZhd3JkeVdpUS9kY3lXNStVM2V5NzF6d3JMLzJTZVlBCmNaOTI0NFVYUDkzMG1JazJrN2FCZlg3MEE0NkFTRzR5NEVIQVNZRFkzdUlWOURwYm5ld0UzWktWV3pEalZtRncKL3RNK3d2QUNjL1BpdU1UUm5OczE2NVg5cnlFSzQ2d0h5UEY2T3Izc2JzM2lueXlzSFRjQUNEWUp2Y0w2Y3lSTgplT1M1dlUyZHJxMFdmSnZ1eks5dGF6QXB5S1EvK3JEam5XVm94Nk9LZWNKMHBudVd4elVJbDN6SU9NcmFzSS9oCm1aeUFjTXNmTjZtclprMWtESS8yOHY5YXV6eTZ4QmhCVUdpWAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
    server: https://10.100.0.1:443
  name: local
contexts:
- context:
    cluster: local
    namespace: default
    user: admin
  name: admin@local
current-context: admin@local
kind: Config
preferences: {}
users:
- name: admin
  user:
    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURLRENDQWhDZ0F3SUJBZ0lVV3JCNjJPelIwTW1VUVVOczUyaVVIZUdqd0Vnd0RRWUpLb1pJaHZjTkFRRUwKQlFBd0ZURVRNQkVHQTFVRUF4TUthM1ZpWlhKdVpYUmxjekFlRncweU1UQTFNakF3T0RVd01EQmFGdzB5TWpBMQpNakF3T0RVd01EQmFNQkF4RGpBTUJnTlZCQU1UQldGa2JXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DCkFROEFNSUlCQ2dLQ0FRRUFwV1JKSDRzZXVTSzJETHJ5YWxNNGFwZVRQQUllem5oMEZFL0xWZWIrSzdqaUVRV1AKaWRFZHJzZnJaKzk5L3JWRXB5ODl6RW5SOGhXa2RIOW5lT0N4dDl0dUs0TEdTQUVZc2twWXJiNHR6WGlBOFE3bgpOTlNJVWdpdXVOZ0V5bzRtbEs4NFg0SG0xVVdGV2c2dTJMMXA4ZkxqN0kzeC9mQi9CVkJHZ0VMTzA3SEZEWHNpCk1lc0VSa2RnTHRlZXd6cjV4clhGczRVUjBnd0FLeFM1UFM5Qml3SWgxcURSOGJybE5vWHpKVThFekNOb0NKUW8KZHA4K21xaTcwOUdiMFQySE1udHRoTTJKUjVaelNqbUlSdmFaL2FEVGZyMDRmZ3U3MDJPNmVwbGFsSkk3MWU2NgpSeExsbXQ3ZEsra3Y4VndobVpHT3oxRU9ZeFlEOURrTkxycjM5d0lEQVFBQm8zVXdjekFPQmdOVkhROEJBZjhFCkJBTUNCYUF3RXdZRFZSMGxCQXd3Q2dZSUt3WUJCUVVIQXdJd0RBWURWUjBUQVFIL0JBSXdBREFkQmdOVkhRNEUKRmdRVU52ME9KQ29scWVwTWQzaE9adVFpOTlpY2IwRXdId1lEVlIwakJCZ3dGb0FVa0ptMWpmVFMvRHZqT05xdApPZEd5NzBsOFhPZ3dEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUJBQVpNeWV1SkhDczNwK3MvczNaUGxMeVVQSi9YCkFVREhQQ0RmUUNqeUhGbC90WnJLMzIrY1gweUNna1JqMlA3anNkZUVBRHovQmtrZnp6Q2NjM2VCVnFXelZ4MHgKK1NtTnRYSmhWL0RUS3BLQlNScEFTOUVWQm9vUldwQ09sZ0UxaEZhMEF3WUlETEFlTUM5Z2NnVzU3NUdMVDlWdApZb3NBU3BSOUtpdFhKQmtEWTVyay9CSjAybXZ2ZUdrZ2lhUFhTaGN4VmZKY3IySjY2MUoxSUlRYjNkTjBzL1NiCnBlZVRHN29PenRiS2hteXE5NGJaNWRhdCsrVHJoZFVDUlNYMlZseXkzZHFYa0lhMjg1MEdaQWZER1JDeFRlejYKVXN6OGg2NVVtQTZQWUQ4YUc4V243LzdLWVpHaGxGN0RhdkFyM3VvVU92aCs0SEFZNFNtdDZTcGx0cVk9Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
    client-key-data: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUVvd0lCQUFLQ0FRRUFwV1JKSDRzZXVTSzJETHJ5YWxNNGFwZVRQQUllem5oMEZFL0xWZWIrSzdqaUVRV1AKaWRFZHJzZnJaKzk5L3JWRXB5ODl6RW5SOGhXa2RIOW5lT0N4dDl0dUs0TEdTQUVZc2twWXJiNHR6WGlBOFE3bgpOTlNJVWdpdXVOZ0V5bzRtbEs4NFg0SG0xVVdGV2c2dTJMMXA4ZkxqN0kzeC9mQi9CVkJHZ0VMTzA3SEZEWHNpCk1lc0VSa2RnTHRlZXd6cjV4clhGczRVUjBnd0FLeFM1UFM5Qml3SWgxcURSOGJybE5vWHpKVThFekNOb0NKUW8KZHA4K21xaTcwOUdiMFQySE1udHRoTTJKUjVaelNqbUlSdmFaL2FEVGZyMDRmZ3U3MDJPNmVwbGFsSkk3MWU2NgpSeExsbXQ3ZEsra3Y4VndobVpHT3oxRU9ZeFlEOURrTkxycjM5d0lEQVFBQkFvSUJBUUNYWUIxcFNIeTdPQ0RsCnIrbVdGVXNxakViUEFxWE5sZ093WDRRY3dJbmIzS0RVc1lFNXczTG1lTVpsMzVZekhSZVQ2VElubTEwVHRHbEwKMHh3U1Q1MTVTTlloN3RTcUxJNjlDRHozOEh1ZGR2Q1ZNVzIrTmRQN09VVXBtMVZ2U3dIV3pqVExmV3FYd3NTVgpzNkNtY1Y3WmV0VFB1dTlFMit4MklZNFZjSkE2d2g2RGRLaXJMWVA0MGswTnp4bWk3U1FkTUsxVVZqY3ZTSURjCmYwTFNHdm5JVnFyVXpGN0dGUzFFcWswWlVBcUNpQ25JbE90Z1d4eUlwZGY4TGlQRGpvdTh6VmNzemRkWDJrVmMKbFoxakZJcXRoZ1NOL2tpMW9OT3NQM3BnWDg3QjRpUzBEZWxOanUzcWdEUUs2aDNnQTNqeHNUekxtNlBsVEt6TwpuRWt4R3hlSkFvR0JBTkROeUM1K016TkVkNjdYbUVvTkZ1N0k2WnpnTFNRTjlBL2ZjQWtKYTkxSW1zQWZCMGRCCm44TGduTnhGMVBhQTlNais5RHd4d2ZwVEtROUN1NGZXN0FjNU1xcVIvU2EzQWVTaURqUWZmT3Z1eGlGanpBc1IKN0ZxS3hkRDUyb0NhMWlxejB0NjNpM1VMMUl4RmJtNkdGeEJBVjBZaGlQdTZtellNamFpNEhtMERBb0dCQU1yRwpnZmdSbWh0cm1peVNMRnpRNmdSSDUrR2lFSEs2d3RDMFpMN2d1NHNpVmwxdUpQMzl2ajRIejJveEsvOUVzQlBPCnlmM1dzeWZ6UFo4cVh1amFLc2prcVZrUTdxMkNqYTVJVVM5V0owT1ZNUFVyckVDa0Z0YTFyWEMwb1JsN20wVWkKN3lWWGVVb21laEZxUjRWRmMxN0JHa3hoTnNlSkJNMGwrTGY1dHhUOUFuODNkbkcxNTgreHpuMzc4MGlBWXZtMApjNm1hQWZIN29ia3lmaWRFUHp4bml1R1FKcTNJUDZZd0hYbU5DUCtJZXJpS3F5NDB6d0szZTE5c3JrMGh4L1RwCjlrZC9CMGczQUdXQVZZdHdUT3podm9yaVU1L2ZVT04zS1dWaUhQdnRMVE8ySkEzZkd1SHdOVFJBRGpGSEtsYjEKK2lFMGY3UGpLYTRRclVUemkvU0pBb0dCQUxraTNPOWRRdExlUDNtS1RDN0NINmVnN3ptMitrV0Y5dkVPZ3gwegptb2ZCNWhDUHkvNktlZk93dGROQzZlUzdGZmcveVhTVlhpV1BsQ2VLUy9WVm05S24wMmFhYXhESUxndDl3NEh3CnF4Q3RXbkcrV2hyNnRrOTZ1TXc2WHd3aEt2Ykltb2dCZ2FBTjJVMmxOMythbFdkaDcxemxOVnFnNDExYTJlYy8KZThVUkFvR0JBS1NTZVpZbDg0RCthYktZS1k0anZSS3hvd1VrNGVmWWU4NXZsSDBNR1NoUHBCL2MzYnpRc2RhRApyRUlhOVc5SitJRHlRcDNIcys0SU9tWmZwcmpTS0pHMFZpNFRJdEZORElGTHl4aE1PZUtxRWx5eUoxcmV2KzBVCndkcEVEMExZcG1KNTV0UTZnenl5QnV2cm9IU1FiVHFxSGltM096SWE1OWJjRmtvVzhQcTAKLS0tLS1FTkQgUFJJVkFURSBLRVktLS0tLQo=

johnniang

xuanying 这个是 demo-kubeconfig 凭证么?

hongming

xuanying

需要按照这个教程重新生成一个 kubeconfig
https://kubesphere.io/zh/docs/faq/devops/create-devops-kubeconfig-on-aws/

需要包含 user token 而不是 client-certificate-data

users:
- name: admin
   token: ****************

如果是 3.1 环境可以直接创建一个服务账户，把 kubeconfig copy 出来使用，注意给权限

xuanying

johnniang 是的