I have created a cron job which runs every 60 min. In the job's container I have mounted emptyDir volume as detailed-logs. In my container I am writing a csv file at path detailed-logs\logs.csv.
I am trying to copy this file from pod to local machine using kubectl cp podname:detailed-logs\logs.csv \k8slogs\logs.csv but it throws the error:
path "detailed-logs\logs.csv" not found (no such file or directory).
Once job runs successfully, pod created by job goes to completed state, is this can be a issue?
The file you are referring to is not going to persist once your pod completes running. What you can do is make a backup of the file when the cron job is running. The two solutions I can suggest are either attach a persistent volume to the job pod, or to upload the file somewhere while running the job.
USE A PERSISTENT VOLUME
Here you can create a PV through a quick readWriteOnce Persistent Volume Claim:
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: my-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5GiThen you can mount it onto the pod using the following:
...
volumeMounts:
- name: persistent-storage
mountPath: /detailed-logs
volumes:
- name: persistent-storage
persistentVolumeClaim:
claimName: my-pvc
...UPLOAD FILE
The way I do it is run the job in a container that has aws-cli installed, and then store my file on AWS S3, you can choose another platform:
apiVersion: v1
kind: ConfigMap
metadata:
name: backup-sh
data:
backup.sh: |-
#!/bin/bash
aws s3 cp /myText.txt s3://bucketName/
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: s3-backup
spec:
schedule: "0 0 * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: aws-kubectl
image: expert360/kubectl-awscli:v1.11.2
env:
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: s3-creds
key: access-key-id
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: s3-creds
key: secret-access-key
command:
- /bin/sh
- -c
args: ["sh /backup.sh"]
volumeMounts:
- name: backup-sh
mountPath: /backup.sh
readOnly: true
subPath: backup.sh
volumes:
- name: backup-sh
configMap:
name: backup-sh
restartPolicy: Never