How to setup kubemqctl to connect to local kubemq cluster running in docker?

2/6/2020

I need to run kubemq cluster on my desktop for development. I have installed kubemq docker container as described here https://docs.kubemq.io/installation/docker.html. But I can't figure out how to connect to it using kubemqctl utility? Is it possible in general? It shows only kubernetes clusters from kubectl config. And I don't see the way how to pass connection information. Thanks.

update #1

after following installation instructions, I see only those cluster which is working in kubernetes cluster and listed in my kubectl config.

Getting KubeMQ Cluster List...
Current Kubernetes cluster context connection: gke_xxxxx_us-central1-a_xxxx
NAME            DESIRED  RUNNING  READY  IMAGE                 AGE          SERVICES
env-dev/kubemq  3        3        3      kubemq/kubemq:latest  1792h54m57s  ClusterIP 10.0.2.211:8080,50000,9090,5228

when I try to switch conext it shows only that cluster again

Current Kubernetes cluster context connection: gke_xxxxx_us-central1-a_xxxx ? Select kubernetes cluster context  [Use arrows to move, type to filter, ? for more help]
> gke_xxxxx_us-central1-a_xxxx

My .kubemqctl.yaml content is:

autointegrated: false
currentnamespace: kubemq
currentstatefulset: kubemq-cluster
host: localhost
grpcport: 50000
restport: 9090
apiport: 8080
issecured: false
certfile: ""
kubeconfigpath: ""
connectiontype: grpc
defaulttoken: XXXXXX-ed0c-4077-9a74-b53805ee3214

update #2

I am able to connect from my go code, to my locally running kubemq cluster

archi@eagle:~/prj/kubemq-go/examples/rpc/command$ docker container ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS                                                                      NAMES
3cd60e4373e4        kubemq/kubemq       "./kubemq-run"      46 hours ago        Up 46 hours         0.0.0.0:8080->8080/tcp, 0.0.0.0:9090->9090/tcp, 0.0.0.0:50000->50000/tcp   modest_wescoff

But I cannot figure out how it is possible to connect to it using kubemq utility, because it does looking once to my kubectl config which cannot contain my local kubemq cluster, running in docker container, not kubernetes cluster. To list my kubemq clusters I use command:

kubemqctl cluster get

the output of the command is shown above (in update #1 section of my question)

update #3

As @mario said. I am able to attach to my queries channel and see all messages!

$kubemqctl queries attach userservice_channel
Adding 'userservice_channel' to attach list
[queries]  [userservice_channel]  kubemq-3cd60e4373e4 Server connected.
[queries]  [userservice_channel]  { "Kind": "request", "ID": "user_token_validate", "Channel": "userservice_channel", "ReplyChannel": "_INBOX.OYnfIQX2k7V9hTmevxHApp.Iqfr3HUw", "Metadata": "some-metadata", "Body": "\n\ufffd\u0001eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyIjoxLCJyb2xlIjoic3VwZXJ1c2Vy....... 
-- Testobile Testossimo
docker
kubernetes
message-queue

1 Answer

2/6/2020

I assume you've already installed kubemqctl command following this instruction but you probably haven't configured it yet.

Once you run kubemqctl help command, it will show you all available options including:

Usage:
  kubemqctl [command]

Available Commands:
  cluster      Executes KubeMQ cluster management commands
  commands     Execute KubeMQ 'commands' RPC commands
  config       Run Kubemqctl configuration wizard command

First you need to run kubemqctl config which will start the configuration wizard, guiding you step by step through the basic configuration:

In second step you should select (using arrow keys) your Local docker container:

? Set Default KubeMQ Token (press Enter for default): 
? Select KubeMQ install location:  [Use arrows to move, type to filter, ? for more help]
  Kubernetes cluster
  MicroK8s
  K3s
  Minikube
  Other Kubernetes distribution
> Local docker container

Other options you can probably left at their defaults.

I assume you already checked that your KubeMQ is up and running and its API is available with:

curl --location --request GET "http://localhost:8080/health" \
  --header "Content-Type: application/json"

If so, you should be able to interact with it using your kubemqctl command line tool.

Please let me know if it helps.


Update:

kubemqctl config is merely a wizard tool which creates your .kubemqctl.yaml file based on which you can run kubemqctl commands against your kubemq instance. My .kubemqctl.yaml looks exactly the same as yours (only with different token).

As I already mentioned in my comment, single docker container isn't considered a cluster at all. From kubernetes perspective it is merely a container runtime and it is totally normal that kubemqctl similarly to kubectl will not list it as a cluster and won't be able to perform on it any cluster-related operation such as kubemqctl cluster get. You can easily verify it by changing your .kube/config name:

mv .kube/config .kube/config_temporarily_disabled

If I run kubemqctl cluster get now, it shows me the following output:

Getting KubeMQ Cluster List...
Error: Kubernetes Config File: Stat <path_to_the_kube_config_file>: No Such File Or Directory

which means that kubemqctl similarly to kubectl always looks for kubeconfig file when performing any cluster-related operation. This group of kubemqctl commands is simply designed only for this purpose and is not intended to be used with your kubemq instance deployed as a single docker container following this instruction.

Important: single docker container IS NOT a cluster

It is not a cluster both in common sense and it is not considered as such by kubemqctl utility.

Note that you didn't even create it with:

kubemqctl cluster create -t <YOUR_KUBEMQ_TOKEN>

which is used to create a cluster in all kubernetes-based solutions described in the documentation: Kubernetes, MicroK8s and K3s but you won't find it in Docker section. Basically if you didn't create your kubemq instance using kubemqctl cluster create command, you also won't be able to list it using kubemqctl cluster get as this is not a cluster.

However you can still successfully run other kubemqctl commands and their sub-commands against your standalone kubemq instance running on a single docker container. You can actually run most of commands listed below (excluding cluster):

$ kubemqctl 
Usage:
  kubemqctl [command]

Available Commands:
  cluster      Executes KubeMQ cluster management commands
  commands     Execute KubeMQ 'commands' RPC commands
  config       Run Kubemqctl configuration wizard command
  events       Execute KubeMQ 'events' Pub/Sub commands
  events_store Execute KubeMQ 'events_store' Pub/Sub commands
  help         Help about any command
  queries      Execute KubeMQ 'queries' RPC based commands
  queues       Execute KubeMQ 'queues' commands

Flags:
  -h, --help      help for kubemqctl
      --version   version for kubemqctl

Use "kubemqctl [command] --help" for more information about a command.

As I already mentioned your .kubemqctl.yaml looks totally fine. It is properly configured to run commands against your kubemq running in docker. When I run docker ps on my machine I can see properly deployed kubemq container:

c9adac88484f        kubemq/kubemq          "./kubemq-run"           3 hours ago         Up 3 hours          0.0.0.0:8080->8080/tcp, 0.0.0.0:9090->9090/tcp, 0.0.0.0:50000->50000/tcp   sleepy_ganguly

As you can see in the output above (or in the output added to your question as it is almost the same), it properly maps required ports exposed by container to machine's localhost address. You can also check it with netstat command:

$ sudo netstat -ntlp | egrep "8080|9090|50000"
tcp6       0      0 :::8080                 :::*                    LISTEN      21431/docker-proxy
tcp6       0      0 :::50000                :::*                    LISTEN      21394/docker-proxy
tcp6       0      0 :::9090                 :::*                    LISTEN      21419/docker-proxy

It's basically enough to be able to use such kubemq instance. Let's run a few example commands.

Running simple:

kubemqctl queues

will show us its help page which specifies how it should be used and gives some useful examples. Let's pick kubemqctl queues list as first:

$ kubemqctl queues list
CHANNELS:
NAME  CLIENTS  MESSAGES  BYTES  FIRST_SEQUENCE  LAST_SEQUENCE
q1    1        12        2621   1               12

TOTAL CHANNELS:  1

CLIENTS:
CLIENT_ID                             CHANNEL  ACTIVE  LAST_SENT  PENDING  STALLED
#####################################  q1       false   2          1        true

TOTAL CLIENTS:  1

Let's pick another one:

kubemqctl queues send

Again, when run without required flags and parameters, it shows us some helpful usage examples:

Error: Missing Arguments, Must Be 2 Arguments, Channel And A Message
Try: 
    # Send message to a queue channel channel
    kubemqctl queue send q1 some-message

    # Send message to a queue channel with metadata
    kubemqctl queue send q1 some-message --metadata some-metadata

    # Send 5 messages to a queues channel with metadata
    kubemqctl queue send q1 some-message --metadata some-metadata -m 5

    # Send message to a queue channel with a message expiration of 5 seconds
    kubemqctl queue send q1 some-message -e 5

    # Send message to a queue channel with a message delay of 5 seconds
    kubemqctl queue send q1 some-message -d 5

    # Send message to a queue channel with a message policy of max receive 5 times and dead-letter queue 'dead-letter'
    kubemqctl queue send q1 some-message -r 5 -q dead-letter

Let's run one of them (slightly modified):

kubemqctl queue send queue2 some-message --metadata some-metadata

You should see similar output:

[Channel: queue2] [client id: ##############################] -> {id: ############################, metadata: some-metadata, body: some-message - (0), sent at: #### }

Now if you list our available queues with kubemqctl queues list command, you'll see among others (q1 was previously created by me) our recently created queue named queue2:

CHANNELS:
NAME    CLIENTS  MESSAGES  BYTES  FIRST_SEQUENCE  LAST_SEQUENCE
q1      1        12        2621   1               12
queue2  0        1         232    1               1

TOTAL CHANNELS:  2
...

It was just an example to show you that it can be run against kubemq instance running on a single docker container but you can run the same way other kubemqctl commands such as commands, events or queries. kubemqctl has really good context help with many usage examples so you should find easily what you need.

I hope I was able to clear up all the ambiguities.

-- mario
Source: StackOverflow