K
Q

How can I copy a string from a vendor app in one Kubernetes pod to a vendor app in another pod?

1/7/2020

I have a set of vendor-provided containers that work together to provide a service, let's call them A, B and C - the Kubernetes cluster runs one of each, in separate pods.

  • Container A is a frontend Nginx container that mostly forwards stuff to container B. The frontend allows you to signup an account, which is really an API call forwarded to container B.
  • Container B is the backend - it has a proprietary in-memory database, and stores the accounts there. When an account is created, a security token is also created and stored. (Users can retrieve that token through API calls via A.)
  • Container C is a bridge to some stuff outside the cluster. But crucially, you configure it with an environment variable containing a valid security token from a user account stored in container B. C uses that token to talk to B through A.

Ideally I would like to deploy this all in one go - meaning I somehow need to create an account in container B (by calling A), then get that value into the environment variable for C.

Some thoughts:

  • I can't use an init container to SQL insert a new user because the database isn't exposed.
  • I can build containers on top of the vendor containers to edit config/scripts etc, but replacing binaries is probably out of scope.
  • I can just about script the backend API through A to B. However, it's challenging as it makes use of XSRF tokens that persist between requests, etc. Any suggestions for the simplest tools/libraries to use to achieve that are welcome.
  • If I do script that account creation, I would then need to get that token into the Deployment for container C - I could use a ConfigMap, but then I would need to call the Kube API to modify the ConfigMap from within the cluster, and that doesn't seem like a great idea to me.

To me, the only viable solution is to put an initContainer on C that will query B (via A) for the security token, then write that to a shared volume. Then I would build on top of container C to read from the shared volume, set the environment variable internal to the container and start the vendor's process. But then I have to manage the secret for the user account.

Are there any improvements on that approach, or anything completely different I haven't considered?

-- Oli
kubernetes
vendor

Similar Questions

Preemptive Cloud Run on GKE
docker start failed after adding daemon.json file
Helm installs charts but doesn't see them
How to isolate data of one persistent volume claim from another
How to create template in the helm chart?
public image pull fails -- without installing docker desktop and login

1 Answer

1/7/2020

Performing API operations from inside a container is not an antipattern. Block startup of your process on C until the initContainer runs and updates the Secret containing the token. Don't use a ConfigMap for secrets; the Secret object exists for these purposes, and you can pull a Secret into a PodSpec - as env vars, or a volume mount - the same way you pull a ConfigMap (with some minor syntax variation).

The only trouble I can see you potentially running into is that you'll be dealing with multiple replicas, so you might want to randomise some of the Secret name. Create it in the initContainer, pass the randomised name across in the filesystem since Pods share a filesystem, then consume it in the main container and delete the Secret after the env var or mount gets set up. Secrets and ConfigMaps can disappear after a Pod starts up without affecting their presence inside the Pod.

You'll also probably want some way of cleaning up the user accounts, as you're essentially creating a new user every time a replica starts and won't have an opportunity to delete it. A CronJob might be the way forward for that - list the user accounts matching your naming convention, then restart the deployment and delete the accounts in the list you fetched before the restart. That way you won't be pulling the rug out on any active replicas.

-- dave.io
Source: StackOverflow