Not able to connect rabbitmq from kubernetes cron jobs

10/26/2018

I am using rabbitmq at a remote (cloudamqp.com) and I create a cron job on Kubernetes. On my local machine, my job is working fine and the Kubernetes cronJob schedules perfectly well but the Job redirects the rabbitmq connection URL to 127.0.0.1:5672 and I get an error.

pika.exceptions.ConnectionClosed: Connection to 127.0.0.1:5672 failed: [Errno 111] Connection refused

I check logs of cron job and my connection URL is perfectly fine but when pika is trying to connect to the host it automatically redirects to 127.0.0.1:5672 as we know the cron pod is not running any rabbitmq server so it refuses the connection.

CronJob.yml

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: scrape-news
spec:
  concurrencyPolicy: Forbid
  jobTemplate:
    spec:
      template:
        metadata:
          labels:
            app: scrape-news
        spec:
          containers:
          - name: scrape-news
            image: SCRAPER_IMAGE
            imagePullPolicy: Always
          restartPolicy: Never
  schedule: '* * * * *'
  successfulJobsHistoryLimit: 3

RabbitMQ Connection

    print(env.RABBIT_URL)
    self.params = pika.URLParameters(env.RABBIT_URL)
    self.connection = pika.BlockingConnection(parameters=self.params)
    self.channel = self.connection.channel() # start a channel

Connection URL is exact same and works on my local setup.

-- Zero
cloudamqp
kubernetes
rabbitmq

1 Answer

10/29/2018

Based on your CronJob spec you are not passing the environment variable RABBIT_URL.

Your code looks as if it is expecting this variable to be set, which it is not, and which is likely why it is defaulting to localhost.

self.params = pika.URLParameters(env.RABBIT_URL)

You probably want something like this:

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: scrape-news
spec:
  concurrencyPolicy: Forbid
  jobTemplate:
    spec:
      template:
        metadata:
          labels:
            app: scrape-news
        spec:
          containers:
          - name: scrape-news
            image: SCRAPER_IMAGE
            imagePullPolicy: Always
            env:
              - name: RABBIT_URL
                value: cloudamqp.com
          restartPolicy: Never
  schedule: '* * * * *'
  successfulJobsHistoryLimit: 3
-- yomateo
Source: StackOverflow