Jenkins on Kubernetes node is complaining its plug-ins need newer version of Jenkins, but don't want to lose data

6/12/2019

Jenkins (on a Kubernetes node) is complaining it requires a newer version of Jenkins to run some of my plug-ins.

SEVERE: Failed Loading plugin Matrix Authorization Strategy Plugin v2.4.2 (matrix-auth) java.io.IOException: Matrix Authorization Strategy Plugin v2.4.2 failed to load. - You must update Jenkins from v2.121.2 to v2.138.3 or later to run this plugin.

The same log file also complains farther down that it can't read my config file... I'm hoping this is just because of the version issue above, but I'm including it here in case it is a sign of deeper issues:

SEVERE: Failed Loading global config java.io.IOException: Unable to read /var/jenkins_home/config.xml

I'd either like to disable the plug-ins that are causing the issue so I can see the Jenkins UI and manage the plug-ins from there, or I'd like to update Jenkins in a way that DOES NOT DELETE MY USER DATA AND JOB CONFIG DATA.

So far, I tried disabling ALL the plug-ins by adding .disabled files to the Jenkins plug-ins folder. That got rid of most of the errors, but it still complained about the plug-in above. So I removed the .disabled file for that, and now it's complaining about Jenkins not being a new enough version again (the error above).

Note: this installation of Jenkins is using a persistent storage volume, mounted with EFS. So that will probably help alleviate some of the restrictions around upgrading Jenkins, if that's what we need to do.

Finally, whatever we do with the plug-ins and Jenkins version, I need to make sure the change is going to persist if Kubernetes re-starts the node in the future. Unfortunately, I am pretty unfamiliar with Kubernetes, and I haven't discovered yet where these changes need to be made. I'm guessing the file that controls the Kubernetes deployment configuration?

This project is using Helm, in case that matters. But again, I hardly know anything about Helm, so I don't know what files you might need to see to make this question solvable. Please comment so I know what to include here to help provide the needed information.

-- David
jenkins
kubernetes
kubernetes-helm
persistent-volumes

1 Answer

6/21/2019

We faced the same problem with our cluster, and we have a basic explanation about that, but not sure about it (The following fix works)

That error come with the fact that you have installed Jenkins via Helm, and their plugins through the Jenkins UI. It works if you decide to never reboot the pod, but if one day, jenkins have to make his initialization again, you will face that error. Jenkins try to load plugins from the JENKINS_PLUGINS_DIR, which is empty, so the pod die.

To fix the current error, you should specify your plugin in the master.installPLugins parameter. If you followed a normal install, just go on your cluster and

helm get values jenkins_release_name

So you may have something like that:

master:
  enableRawHtmlMarkupFormatter: true
  installPlugins:
  - kubernetes:1.16.0
  - workflow-job:2.32

By default, some values are "embedded" by helm to be sure that jenkins works, see here for more details: Github Helm Charts Jenkins

So, just copy it in a file with the same syntax and add your plugins with their versions. After, you have just to use the helm upgrade command with your file on your release:

helm upgrade [RELEASE] [CHART] -f your_file.yaml

Good luck !

-- Manyia
Source: StackOverflow