How do I configure my helm chart to pickup a rabbitmq community plugin?

1/11/2020

GOAL: To use a rabbitmq community plugin from https://www.rabbitmq.com/community-plugins.html in my rabbitmq-ha helm chart.

Any plugins that do not ship with the server will need to be installed. Plugins are distributed as .ez archives (which are zip files with metadata). The files must be copied to one of the plugins directories specified by $RABBITMQ_PLUGINS_DIR.

PROBLEM: How does the rabbitmq community plugin's .ev file get injected into the chart for use?


BACKGROUND:

For my example I am trying to install the latest version of the community plugin rabbitmq_delayed_message_exchange into the latest stable version of rabbitmq-ha.

The name of the plugin is identified by adding this to my values.yaml:

amqp-rabbitmq:
  extraPlugins: |
    rabbitmq_delayed_message_exchange,

And when I

helm install example .  --debug

it generates

# Source: example/charts/amqp-rabbitmq/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: example-amqp-rabbitmq
  namespace: default
  labels:
    app: amqp-rabbitmq
    chart: amqp-rabbitmq-1.38.2
    release: example
    heritage: Helm
data:
  enabled_plugins: |
    [
      rabbitmq_delayed_message_exchange,
      rabbitmq_consistent_hash_exchange,
      rabbitmq_management,
      rabbitmq_peer_discovery_k8s
    ].

  rabbitmq.conf: |
    ## RabbitMQ configuration
    ## Ref: https://github.com/rabbitmq/rabbitmq-server/blob/master/docs/rabbitmq.conf.example

    ## Authentification

    ## Clustering
    cluster_formation.peer_discovery_backend  = rabbit_peer_discovery_k8s
    cluster_formation.k8s.host = kubernetes.default.svc.cluster.local
    cluster_formation.k8s.address_type = hostname
    cluster_formation.node_cleanup.interval = 10
    # Set to false if automatic cleanup of absent nodes is desired.
    # This can be dangerous, see http://www.rabbitmq.com/cluster-formation.html#node-health-checks-and-cleanup.
    cluster_formation.node_cleanup.only_log_warning = true
    cluster_partition_handling = autoheal
    ## The default "guest" user is only permitted to access the server
    ## via a loopback interface (e.g. localhost)
    loopback_users.guest = false

    management.load_definitions = /etc/definitions/definitions.json

    ## Memory-based Flow Control threshold
    vm_memory_high_watermark.absolute = 256MB

    ## Auth HTTP Backend Plugin

    ## LDAP Plugin

    ## MQTT Plugin

    ## Web MQTT Plugin

    ## STOMP Plugin

    ## Web STOMP Plugin

    ## Prometheus Plugin

    ## AMQPS support

and creates 3 k8s_amqp-rabbitmq_example-amqp-rabbitmq-n_default_... containers.

The log in each container shows

2020-01-11 01:21:16.826 [info] <0.8.0> Server startup complete; 6 plugins started.

* rabbitmq_management

* rabbitmq_web_dispatch

* rabbitmq_consistent_hash_exchange

* rabbitmq_peer_discovery_k8s

* rabbitmq_management_agent

* rabbitmq_peer_discovery_common

2020-01-11 01:23:06.954 [info] <0.529.0> node 'rabbit@example-amqp-rabbitmq-1.example-amqp-rabbitmq-discovery.default.svc.cluster.local' up

2020-01-11 01:23:12.776 [info] <0.529.0> rabbit on node 'rabbit@example-amqp-rabbitmq-1.example-amqp-rabbitmq-discovery.default.svc.cluster.local' up

I can successfully log into rabbitmq, create exchanges, queues, etc.

However, the desired rabbitmq_delayed_message_exchange plugin is no where to be found.

  • It is not mentioned in the log file anywhere.
  • It does not show up in the rabbit@example-amqp-rabbitmq-n.example-amqp-rabbitmq-discovery.default.svc.cluster.local-plugins-expand folder with the other plugins.

When my application tries to use the helm installed rabbitmq-ha to create the delayed exchange it gets:

'Error: Connection closed: 503 (COMMAND-INVALID) with message "COMMAND_INVALID - unknown exchange type 'x-delayed-message'"

It works with the non kubernetes instances of rabbitmq where I could easily install the plugin.


I have spent several days searching the web for a clear example, hints, and clues as to how to avail the rabbitmq-ha chart of a community plugin and not progressed.

Any ideas out there? What am I missing?

-- peasant
kubernetes-helm
rabbitmq

2 Answers

1/11/2020

I think rabbitmq-ha default helm chart use the official rabbitMQ image, so it only contains core plugins.

I check the chart config, it just set the /etc/rabbitmq/enabled_plugins conf file, plugin file will not be downloaded automatically.

There are two solutions:

  • Build your custom rabbitmq image, ship the rabbitmq_delayed_message_exchange file with it.
  • Download the plugin file in initContainer, add it to your chart values extraInitContainers
-- menya
Source: StackOverflow

4/24/2020

Here is an example to install and enable two RabbitMQ community plugins:

  • rabbitmq_delayed_message_exchange
  • rabbitmq_message_timestamp

Step 1: Get the Helm chart:

$ helm pull --untar bitnami/rabbitmq

Step 2: Update file statefulset.yaml

...
# Install community plugins
pushd /opt/bitnami/rabbitmq/plugins/
curl -LO https://github.com/rabbitmq/rabbitmq-delayed-message-exchange/releases/download/v3.8.0/rabbitmq_delayed_message_exchange-3.8.0.ez
curl -LO https://github.com/rabbitmq/rabbitmq-message-timestamp/releases/download/v3.8.0/rabbitmq_message_timestamp-3.8.0.ez
popd
# End of installing community plugins
exec rabbitmq-server
...

Step 3: Install RabbitMQ with the two plugins enabled

$ cd rabbitmq
$ helm install rmq . --set rabbitmq.extraPlugins="rabbitmq_delayed_message_exchange rabbitmq_message_timestamp"

Now you can verify that the two plugins have been installed

$ kubectl get pod
NAME             READY   STATUS    RESTARTS   AGE
rmq-rabbitmq-0   1/1     Running   0          3m32s

$ kubectl exec -it rmq-rabbitmq-0 bash
I have no name!@rmq-rabbitmq-0:/$ rabbitmq-plugins list
Listing plugins with pattern ".*" ...
 Configured: E = explicitly enabled; e = implicitly enabled
 | Status: * = running on rabbit@rmq-rabbitmq-0.rmq-rabbitmq-headless.default.svc.cluster.local
 |/
[  ] rabbitmq_amqp1_0                  3.8.3
[  ] rabbitmq_auth_backend_cache       3.8.3
[  ] rabbitmq_auth_backend_http        3.8.3
[  ] rabbitmq_auth_backend_ldap        3.8.3
[  ] rabbitmq_auth_backend_oauth2      3.8.3
[  ] rabbitmq_auth_mechanism_ssl       3.8.3
[  ] rabbitmq_consistent_hash_exchange 3.8.3
[E*] rabbitmq_delayed_message_exchange 3.8.0
[  ] rabbitmq_event_exchange           3.8.3
[  ] rabbitmq_federation               3.8.3
[  ] rabbitmq_federation_management    3.8.3
[  ] rabbitmq_jms_topic_exchange       3.8.3
[E*] rabbitmq_management               3.8.3
[e*] rabbitmq_management_agent         3.8.3
[E*] rabbitmq_message_timestamp        3.8.0
[  ] rabbitmq_mqtt                     3.8.3
...

Note: My Helm version is v3.1.1

$ helm version
version.BuildInfo{Version:"v3.1.1", GitCommit:"afe70585407b420d0097d07b21c47dc511525ac8", GitTreeState:"clean", GoVersion:"go1.13.8"}
-- Yuci
Source: StackOverflow