I am trying to use Snakemake on AWS EKS, running shell commands in singularity containers. If I understood correctly, Snakemake is then run as a container itself by kubernetes, so I would run a singularity container inside a Pod. I have a ubuntu image in the same folder of the Snakefile. How do I tell Singularity the path where the image is stored?
I have tried to run the workflow specifying an image in the DockerHub and it works perfectly. However I cannot make it work with images stored in my EC2 instance, if I specify --kubernetes flag (if I run it without --kubernetes, it runs perfectly also with local images). I tried specifying the path of the image on the EC2 instance, but it is not working.
My Snakefile looks something like that:
rule test:
singularity:
"ubuntu.simg"
output:
"out.txt"
shell:
"cat /etc/os-release > {output}"
and I run Snakemake with the command:
snakemake --kubernetes --default-remote-provider S3 --default-remote-prefix S3-bucket-name --use-singularity
Desired output is the file "out.txt" created in the specified s3 bucket with the info about the os release of the "ubuntu.sigm" container. The actual result is a failed job. If I inspect with
kubectl logs snakejob-c25eaf1f-6ad4-505e-94a0-646543a59d33
I get the error:
ERROR : Image path ubuntu.simg doesn't exist: No such file or directory
ABORT : Retval = 255
Anyone who had the same issue?