I work in a large enterprise that manages several k8s clusters from dev to production.

We often run "kubectl" commands in our Jenkins builds, either for "rollout" or getting configmap data for diagnostics, or other uses.

Each "kubectl" call gets a base set of parameters, like this:

kubectl ... --namespace ... --server=... --insecure-skip-tls-verify=true --token=...

In recent weeks, we've been seeing many of these commands fail with the following:

error: more than one authentication method found for tier-two-user; found [token basicAuth], only one is allowed

Sometimes these go away when the same build is respawned. Sometimes it's more persistent. We've had teams of people looking at this problem for a while now, and there's no apparent resolution.

We have no idea what is adding basic auth credentials.

I would appreciate any bit of applicable information you can provide.

Update:

% kubectl version
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.3", GitCommit:"2bba0127d85d5a46ab4b778548be28623b32d0b0", GitTreeState:"clean", BuildDate:"2018-05-21T09:17:39Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"8+", GitVersion:"v1.8.5+coreos.0", GitCommit:"b8e596026feda7b97f4337b115d1a9a250afa8ac", GitTreeState:"clean", BuildDate:"2017-12-12T11:01:08Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}

Concerning which actual command fails, it's either been "rollout" or "get". Otherwise, all the options specified above are being provided.

Update:

I just tried adding some debug output to the build, and I think I might be seeing a clue, but I don't know.

I thought it would be interesting to cat out the ~/.kube/config file before the command runs.

This is basically what it shows:

apiVersion: v1
clusters:
- cluster:
...
contexts:
- context:
    cluster: target
    user: tier-two-user
  name: aaf-edge-upgrade
...
users:
...
- name: tier-two-user
  user:
    password: ...
    username: ...

Note that the settings for user "tier-two-user" is the last block in the file.

I find it curious that the error message refers to incorrect settings for the same user name, stating that it has both a password setting and a token. It almost seems like the configuration it's trying to use is the contents of the config file concatenated with the command line arguments, which includes a "--token" option.

Is it possible that the config file in question needs a newline after the first non-blank line?