GOAL: To use a rabbitmq community plugin from https://www.rabbitmq.com/community-plugins.html in my rabbitmq-ha helm chart.
Any plugins that do not ship with the server will need to be installed. Plugins are distributed as .ez archives (which are zip files with metadata). The files must be copied to one of the plugins directories specified by $RABBITMQ_PLUGINS_DIR.
PROBLEM: How does the rabbitmq community plugin's .ev file get injected into the chart for use?
BACKGROUND:
For my example I am trying to install the latest version of the community plugin rabbitmq_delayed_message_exchange into the latest stable version of rabbitmq-ha.
The name of the plugin is identified by adding this to my values.yaml:
amqp-rabbitmq:
extraPlugins: |
rabbitmq_delayed_message_exchange,
And when I
helm install example . --debug
it generates
# Source: example/charts/amqp-rabbitmq/templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: example-amqp-rabbitmq
namespace: default
labels:
app: amqp-rabbitmq
chart: amqp-rabbitmq-1.38.2
release: example
heritage: Helm
data:
enabled_plugins: |
[
rabbitmq_delayed_message_exchange,
rabbitmq_consistent_hash_exchange,
rabbitmq_management,
rabbitmq_peer_discovery_k8s
].
rabbitmq.conf: |
## RabbitMQ configuration
## Ref: https://github.com/rabbitmq/rabbitmq-server/blob/master/docs/rabbitmq.conf.example
## Authentification
## Clustering
cluster_formation.peer_discovery_backend = rabbit_peer_discovery_k8s
cluster_formation.k8s.host = kubernetes.default.svc.cluster.local
cluster_formation.k8s.address_type = hostname
cluster_formation.node_cleanup.interval = 10
# Set to false if automatic cleanup of absent nodes is desired.
# This can be dangerous, see http://www.rabbitmq.com/cluster-formation.html#node-health-checks-and-cleanup.
cluster_formation.node_cleanup.only_log_warning = true
cluster_partition_handling = autoheal
## The default "guest" user is only permitted to access the server
## via a loopback interface (e.g. localhost)
loopback_users.guest = false
management.load_definitions = /etc/definitions/definitions.json
## Memory-based Flow Control threshold
vm_memory_high_watermark.absolute = 256MB
## Auth HTTP Backend Plugin
## LDAP Plugin
## MQTT Plugin
## Web MQTT Plugin
## STOMP Plugin
## Web STOMP Plugin
## Prometheus Plugin
## AMQPS support
and creates 3 k8s_amqp-rabbitmq_example-amqp-rabbitmq-n_default_... containers.
The log in each container shows
2020-01-11 01:21:16.826 [info] <0.8.0> Server startup complete; 6 plugins started.
* rabbitmq_management
* rabbitmq_web_dispatch
* rabbitmq_consistent_hash_exchange
* rabbitmq_peer_discovery_k8s
* rabbitmq_management_agent
* rabbitmq_peer_discovery_common
2020-01-11 01:23:06.954 [info] <0.529.0> node 'rabbit@example-amqp-rabbitmq-1.example-amqp-rabbitmq-discovery.default.svc.cluster.local' up
2020-01-11 01:23:12.776 [info] <0.529.0> rabbit on node 'rabbit@example-amqp-rabbitmq-1.example-amqp-rabbitmq-discovery.default.svc.cluster.local' up
I can successfully log into rabbitmq, create exchanges, queues, etc.
However, the desired rabbitmq_delayed_message_exchange plugin is no where to be found.
When my application tries to use the helm installed rabbitmq-ha to create the delayed exchange it gets:
'Error: Connection closed: 503 (COMMAND-INVALID) with message "COMMAND_INVALID - unknown exchange type 'x-delayed-message'"
It works with the non kubernetes instances of rabbitmq where I could easily install the plugin.
I have spent several days searching the web for a clear example, hints, and clues as to how to avail the rabbitmq-ha chart of a community plugin and not progressed.
Any ideas out there? What am I missing?
I think rabbitmq-ha
default helm chart use the official rabbitMQ image, so it only contains core plugins.
I check the chart config, it just set the /etc/rabbitmq/enabled_plugins
conf file, plugin file will not be downloaded automatically.
There are two solutions:
rabbitmq_delayed_message_exchange
file with it.extraInitContainers
Here is an example to install and enable two RabbitMQ community plugins:
$ helm pull --untar bitnami/rabbitmq
statefulset.yaml
...
# Install community plugins
pushd /opt/bitnami/rabbitmq/plugins/
curl -LO https://github.com/rabbitmq/rabbitmq-delayed-message-exchange/releases/download/v3.8.0/rabbitmq_delayed_message_exchange-3.8.0.ez
curl -LO https://github.com/rabbitmq/rabbitmq-message-timestamp/releases/download/v3.8.0/rabbitmq_message_timestamp-3.8.0.ez
popd
# End of installing community plugins
exec rabbitmq-server
...
$ cd rabbitmq
$ helm install rmq . --set rabbitmq.extraPlugins="rabbitmq_delayed_message_exchange rabbitmq_message_timestamp"
$ kubectl get pod
NAME READY STATUS RESTARTS AGE
rmq-rabbitmq-0 1/1 Running 0 3m32s
$ kubectl exec -it rmq-rabbitmq-0 bash
I have no name!@rmq-rabbitmq-0:/$ rabbitmq-plugins list
Listing plugins with pattern ".*" ...
Configured: E = explicitly enabled; e = implicitly enabled
| Status: * = running on rabbit@rmq-rabbitmq-0.rmq-rabbitmq-headless.default.svc.cluster.local
|/
[ ] rabbitmq_amqp1_0 3.8.3
[ ] rabbitmq_auth_backend_cache 3.8.3
[ ] rabbitmq_auth_backend_http 3.8.3
[ ] rabbitmq_auth_backend_ldap 3.8.3
[ ] rabbitmq_auth_backend_oauth2 3.8.3
[ ] rabbitmq_auth_mechanism_ssl 3.8.3
[ ] rabbitmq_consistent_hash_exchange 3.8.3
[E*] rabbitmq_delayed_message_exchange 3.8.0
[ ] rabbitmq_event_exchange 3.8.3
[ ] rabbitmq_federation 3.8.3
[ ] rabbitmq_federation_management 3.8.3
[ ] rabbitmq_jms_topic_exchange 3.8.3
[E*] rabbitmq_management 3.8.3
[e*] rabbitmq_management_agent 3.8.3
[E*] rabbitmq_message_timestamp 3.8.0
[ ] rabbitmq_mqtt 3.8.3
...
Note: My Helm version is v3.1.1
$ helm version
version.BuildInfo{Version:"v3.1.1", GitCommit:"afe70585407b420d0097d07b21c47dc511525ac8", GitTreeState:"clean", GoVersion:"go1.13.8"}