K
Q

Question

Kubernetes: Modeling Jobs/Cron tasks for Postgres + Tomcat application

8/1/2019

I work on an open source system that is comprised of a Postgres database and a tomcat server. I have docker images for each component. We currently use docker-compose to test the application.

I am attempting to model this application with kubernetes.

Here is my first attempt.

apiVersion: v1
kind: Pod
metadata:
  name: dspace-pod
spec:
  volumes:
  - name: "pgdata-vol"
    emptyDir: {}
  - name: "assetstore"
    emptyDir: {}
  - name: my-local-config-map
    configMap:
      name: local-config-map
  containers:
  - image: dspace/dspace:dspace-6_x
    name: dspace
    ports:
    - containerPort: 8080
      name: http
      protocol: TCP
    volumeMounts:
    - mountPath: "/dspace/assetstore"
      name: "assetstore"
    - mountPath: "/dspace/config/local.cfg"
      name: "my-local-config-map"
      subPath: local.cfg
  #
  - image: dspace/dspace-postgres-pgcrypto
    name: dspacedb
    ports:
    - containerPort: 5432
      name: http
      protocol: TCP
    volumeMounts:
    - mountPath: "/pgdata"
      name: "pgdata-vol"
    env:
    - name: PGDATA
      value: /pgdata

I have a configMap that is setting the hostname to the name of the pod.

apiVersion: v1
kind: ConfigMap
metadata:
  creationTimestamp: 2016-02-18T19:14:38Z
  name: local-config-map
  namespace: default
data:
  local.cfg: |-
    dspace.dir = /dspace
    db.url = jdbc:postgresql://dspace-pod:5432/dspace
    dspace.hostname = dspace-pod
    dspace.baseUrl = http://dspace-pod:8080
    solr.server=http://dspace-pod:8080/solr

This application has a number of tasks that are run from the command line.

I have created a 3rd Docker image that contains the jars that are needed on the command line.

I am interested in modeling these command line tasks as Jobs in Kubernetes. Assuming that is a appropriate way to handle these tasks, how do I specify that a job should run within a Pod that is already running?

Here is my first attempt at defining a job.

apiVersion: batch/v1
kind: Job
#https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/
metadata:
  name: dspace-create-admin
spec:
  template:
    spec:
      volumes:
      - name: "assetstore"
        emptyDir: {}
      - name: my-local-config-map
        configMap:
          name: local-config-map
      containers:
      - name: dspace-cli
        image: dspace/dspace-cli:dspace-6_x
        command: [
          "/dspace/bin/dspace",
          "create-administrator",
          "-e", "test@test.edu",
          "-f", "test",
          "-l", "admin",
          "-p", "admin",
          "-c", "en"
        ]
        volumeMounts:
        - mountPath: "/dspace/assetstore"
          name: "assetstore"
        - mountPath: "/dspace/config/local.cfg"
          name: "my-local-config-map"
          subPath: local.cfg
      restartPolicy: Never

-- terrywb

docker

kubernetes

kubernetes-pod

1 Answer

8/2/2019

The following configuration has allowed me to start my services (tomcat and postgres) as I hoped.

apiVersion: v1
kind: ConfigMap
metadata:
  creationTimestamp: 2016-02-18T19:14:38Z
  name: local-config-map
  namespace: default
data:
  # example of a simple property defined using --from-literal
  #example.property.1: hello
  #example.property.2: world
  # example of a complex property defined using --from-file
  local.cfg: |-
    dspace.dir = /dspace
    db.url = jdbc:postgresql://dspacedb-service:5432/dspace
    dspace.hostname = dspace-service
    dspace.baseUrl = http://dspace-service:8080
    solr.server=http://dspace-service:8080/solr
---
apiVersion: v1
kind: Service
metadata:
  name: dspacedb-service
  labels:
    app: dspacedb-app
spec:
  type: NodePort
  selector:
    app: dspacedb-app
  ports:
  - protocol: TCP
    port: 5432
  #  targetPort: 5432
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: dspacedb-deploy
  labels:
    app: dspacedb-app
spec:
  selector:
    matchLabels:
      app: dspacedb-app
  template:
    metadata:
      labels:
        app: dspacedb-app
    spec:
      volumes:
      - name: "pgdata-vol"
        emptyDir: {}
      containers:
      - image: dspace/dspace-postgres-pgcrypto
        name: dspacedb
        ports:
        - containerPort: 5432
          name: http
          protocol: TCP
        volumeMounts:
        - mountPath: "/pgdata"
          name: "pgdata-vol"
        env:
        - name: PGDATA
          value: /pgdata
---
apiVersion: v1
kind: Service
metadata:
  name: dspace-service
  labels:
    app: dspace-app
spec:
  type: NodePort
  selector:
    app: dspace-app
  ports:
  - protocol: TCP
    port: 8080
    targetPort: 8080
    name: http
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: dspace-deploy
  labels:
    app: dspace-app
spec:
  selector:
    matchLabels:
      app: dspace-app
  template:
    metadata:
      labels:
        app: dspace-app
    spec:
      volumes:
      - name: "assetstore"
        emptyDir: {}
      - name: my-local-config-map
        configMap:
          name: local-config-map
      containers:
      - image: dspace/dspace:dspace-6_x-jdk8-test
        name: dspace
        ports:
        - containerPort: 8080
          name: http
          protocol: TCP
        volumeMounts:
        - mountPath: "/dspace/assetstore"
          name: "assetstore"
        - mountPath: "/dspace/config/local.cfg"
          name: "my-local-config-map"
          subPath: local.cfg

After applying the configuration above, I have the following results.

$ kubectl get services -o wide
NAME               TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE       SELECTOR
dspace-service     NodePort    10.104.224.245   <none>        8080:32459/TCP   3s        app=dspace-app
dspacedb-service   NodePort    10.96.212.9      <none>        5432:30947/TCP   3s        app=dspacedb-app
kubernetes         ClusterIP   10.96.0.1        <none>        443/TCP          22h       <none>

$ kubectl get pods
NAME                               READY     STATUS      RESTARTS   AGE
dspace-deploy-c59b77bb8-mr47k      1/1       Running     0          10m
dspacedb-deploy-58dd85f5b9-6v2lf   1/1       Running     0          10

I was pleased to see that the service name can be used for port forwarding.

$ kubectl port-forward service/dspace-service 8080:8080
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080

I am also able to run the following job using the defined service names in the configMap.

apiVersion: batch/v1
kind: Job
metadata:
  name: dspace-create-admin
spec:
  template:
    spec:
      volumes:
      - name: "assetstore"
        emptyDir: {}
      - name: my-local-config-map
        configMap:
          name: local-config-map
      containers:
      - name: dspace-cli
        image: dspace/dspace-cli:dspace-6_x
        command: [
          "/dspace/bin/dspace",
          "create-administrator",
          "-e", "test@test.edu",
          "-f", "test",
          "-l", "admin",
          "-p", "admin",
          "-c", "en"
        ]
        volumeMounts:
        - mountPath: "/dspace/assetstore"
          name: "assetstore"
        - mountPath: "/dspace/config/local.cfg"
          name: "my-local-config-map"
          subPath: local.cfg
      restartPolicy: Never

Results

$ kubectl get pods
NAME                               READY     STATUS      RESTARTS   AGE
dspace-create-admin-kl6wd          0/1       Completed   0          5m
dspace-deploy-c59b77bb8-mr47k      1/1       Running     0          10m
dspacedb-deploy-58dd85f5b9-6v2lf   1/1       Running     0          10m

I still have some work to do persisting the volumes.

-- terrywb

Source: StackOverflow

KQ

Kubernetes: Modeling Jobs/Cron tasks for Postgres + Tomcat application

Similar Questions

1 Answer

K
Q