First of all, I have Kubernetes cluster behind the proxy environment.
I have three servers, master, node1, and node2.
I installed Jenkins using below command.
Create jenkine-project namespace and then
helm install --name jenkins -f jenkins-values.yaml stable/jenkins --namespace jenkins-project
jenkins-values.yaml is
Master:
Name: jenkins-master
Image: "jenkins/jenkins"
ImageTag: "lts"
ImagePullPolicy: "Always"
# ImagePullSecret: jenkins
Component: "jenkins-master"
UseSecurity: true
AdminUser: admin
AdminPassword: 1qaz2wsx
Cpu: "200m"
Memory: "512Mi"
# Environment variables that get added to the init container (useful for e.g. http_proxy)
InitContainerEnv:
- name: http_proxy
value: "http://168.219.yyy.zzz:8080"
- name: https_proxy
value: "http://168.219.yyy.zzz:8080"
- name: no_proxy
value: "localhost,127.0.0.1,10.251.141.*,"
ContainerEnv:
- name: http_proxy
value: "http://168.219.yyy.zzz:8080"
- name: https_proxy
value: "http://168.219.yyy.zzz:8080"
JavaOpts: >-
-Dhttp.proxyHost=168.219.yyy.zzz
-Dhttp.proxyPort=8080
-Dhttps.proxyHost=168.219.yyy.zzz
-Dhttps.proxyPort=8080
# Set min/max heap here if needed with:
# JavaOpts: "-Xms512m -Xmx512m"
# JenkinsOpts: ""
# JenkinsUriPrefix: "/jenkins"
# Set RunAsUser to 1000 to let Jenkins run as non-root user 'jenkins' which exists in 'jenkins/jenkins' docker image.
# When setting RunAsUser to a different value than 0 also set FsGroup to the same value:
# RunAsUser: <defaults to 0>
# FsGroup: <will be omitted in deployment if RunAsUser is 0>
ServicePort: 8080
# For minikube, set this to NodePort, elsewhere use LoadBalancer
# Use ClusterIP if your setup includes ingress controller
ServiceType: LoadBalancer
# Master Service annotations
ServiceAnnotations: {}
# service.beta.kubernetes.io/aws-load-balancer-backend-protocol: https
# Used to create Ingress record (should used with ServiceType: ClusterIP)
# HostName: jenkins.cluster.local
# NodePort: <to set explicitly, choose port between 30000-32767
ContainerPort: 8080
# Enable Kubernetes Liveness and Readiness Probes
HealthProbes: true
HealthProbesTimeout: 60
SlaveListenerPort: 50000
# Kubernetes service type for the JNLP slave service
# SETTING THIS TO "LoadBalancer" IS A HUGE SECURITY RISK: https://github.com/kubernetes/charts/issues/1341
SlaveListenerServiceType: ClusterIP
SlaveListenerServiceAnnotations: {}
LoadBalancerSourceRanges:
- 0.0.0.0/0
# Optionally assign a known public LB IP
# LoadBalancerIP: 1.2.3.4
# Optionally configure a JMX port
# requires additional JavaOpts, ie
# JavaOpts: >
# -Dcom.sun.management.jmxremote.port=4000
# -Dcom.sun.management.jmxremote.authenticate=false
# -Dcom.sun.management.jmxremote.ssl=false
# JMXPort: 4000
# List of plugins to be install during Jenkins master start
InstallPlugins:
- kubernetes:1.4
- workflow-aggregator:2.5
- workflow-job:2.17
- credentials-binding:1.16
- p4:1.8.7
- blueocean:1.4.2
# Used to approve a list of groovy functions in pipelines used the script-security plugin. Can be viewed under /scriptApproval
ScriptApproval:
- "method groovy.json.JsonSlurperClassic parseText java.lang.String"
- "new groovy.json.JsonSlurperClassic"
- "staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods leftShift java.util.Map java.util.Map"
- "staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods split java.lang.String"
# List of groovy init scripts to be executed during Jenkins master start
InitScripts:
# - |
# print 'adding global pipeline libraries, register properties, bootstrap jobs...'
# Kubernetes secret that contains a 'credentials.xml' for Jenkins
# CredentialsXmlSecret: jenkins-credentials
# Kubernetes secret that contains files to be put in the Jenkins 'secrets' directory,
# useful to manage encryption keys used for credentials.xml for instance (such as
# master.key and hudson.util.Secret)
# SecretsFilesSecret: jenkins-secrets
# Jenkins XML job configs to provision
# Jobs: |-
# test: |-
# <<xml here>>
CustomConfigMap: false
# Node labels and tolerations for pod assignment
# ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector
# ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#taints-and-tolerations-beta-feature
NodeSelector: {}
Tolerations: {}
Ingress:
Annotations:
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
TLS:
# - secretName: jenkins.cluster.local
# hosts:
# - jenkins.cluster.local
Agent:
Enabled: true
Image: jenkins/jnlp-slave
ImageTag: 3.10-1
# ImagePullSecret: jenkins
Component: "jenkins-slave"
Privileged: false
Cpu: "200m"
Memory: "256Mi"
# You may want to change this to true while testing a new image
AlwaysPullImage: false
# You can define the volumes that you want to mount for this container
# Allowed types are: ConfigMap, EmptyDir, HostPath, Nfs, Pod, Secret
# Configure the attributes as they appear in the corresponding Java class for that type
# https://github.com/jenkinsci/kubernetes-plugin/tree/master/src/main/java/org/csanchez/jenkins/plugins/kubernetes/volumes
volumes:
- type: HostPath
secretName: /var/run/docker.sock
mountPath: /var/run/docker.sock
NodeSelector: {}
# Key Value selectors. Ex:
# jenkins-agent: v1
Persistence:
Enabled: true
## A manually managed Persistent Volume and Claim
## Requires Persistence.Enabled: true
## If defined, PVC must be created manually before volume will be bound
# ExistingClaim: pvc-jenkins-master
## jenkins data Persistent Volume Storage Class
## If defined, storageClassName: <storageClass>
## If set to "-", storageClassName: "", which disables dynamic provisioning
## If undefined (the default) or set to null, no storageClassName spec is
## set, choosing the default provisioner. (gp2 on AWS, standard on
## GKE, AWS & OpenStack)
##
StorageClass: "jenkins-pv"
Annotations: {}
AccessMode: ReadWriteOnce
Size: 20Gi
volumes:
# - name: nothing
# emptyDir: {}
mounts:
# - mountPath: /var/nothing
# name: nothing
# readOnly: true
NetworkPolicy:
# Enable creation of NetworkPolicy resources.
Enabled: false
# For Kubernetes v1.4, v1.5 and v1.6, use 'extensions/v1beta1'
# For Kubernetes v1.7, use 'networking.k8s.io/v1'
ApiVersion: networking.k8s.io/v1
## Install Default RBAC roles and bindings
rbac:
install: true
serviceAccountName: default
# RBAC api version (currently either v1beta1 or v1alpha1)
apiVersion: v1beta1
# Cluster role reference
roleRef: cluster-admin
And pod jenkins-694674f4bd-zqfpq
is created.
I run kubectl logs jenkins-694674f4bd-zqfpq -n jenkins-project
command here is problem.
# kubectl logs jenkins-694674f4bd-zqfpq -n jenkins-project
Error from server: Get https://10.251.141.74:10250/containerLogs/jenkins-project/jenkins-694674f4bd-zqfpq/jenkins: read tcp 10.251.141.xxx:34630->168.219.yyy.zzz:8080: read: connection reset by peer
In this error message, 10.251.141.xxx is the master server's IP address and 168.219.yyy.zzz:8080 is proxy address.
And (I guess) because of this problem plugin will not be installed normally.
What is the problem and how can I fix this?
As I understood, you have a cluster behind a proxy, so it looks like that:
You | Proxy| All Kubernetes nodes and master
When you call kubectl logs
command, kubectl
is connecting to the API server and then the API server is getting logs of your pod from the node.
As I see from the output of the command, the API server is trying to connect to the node through the proxy server instead of direct connection, that's why I think you have a bit incorrect setup of proxy settings on your master.
Try to add all cluster internal IP ranges to exception using no_proxy
on the master and on nodes, I think it should help.