I am trying to run the akka-sample-cluster-kubernetes-scala as it is recommended to deploy an Akka cluster on to minikube using akka-management-cluster-bootstrap. After run every step recommended on the README file I can see the pods running on my kubectl output:
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
appka-8b88f7bdd-485nx 1/1 Running 0 48m
appka-8b88f7bdd-4blrv 1/1 Running 0 48m
appka-8b88f7bdd-7qlc9 1/1 Running 0 48mWhen I execute the ./scripts/test.sh it seems to fail on the last step:
"No 3 MemberUp log events found"And I cannot connect to the given address said on the README file. The error:
$ curl http://127.0.0.1:8558/cluster/members/
curl: (7) Failed to connect to 127.0.0.1 port 8558: Connection refusedFrom now on I describe how I try to find the reason that I cannot use the sample akka + kubernetes project. I am trying to find the cause of the error above mentioned. I suppose I have to execute sbt run, even it is not mentioned on the sample project. And them I get the following error with respect to the ${REQUIRED_CONTACT_POINT_NR} variable at application.conf:
error Exception in thread "main" com.typesafe.config.ConfigException$UnresolvedSubstitution: application.conf @ jar:file:/home/felipe/workspace-idea/akka-sample-cluster-kubernetes-scala/target/bg-jobs/sbt_12a05599/job-1/target/d9ddd12d/64fe375d/akka-sample-cluster-kubernetes_2.13-0.0.0-70-49d6a855-20210104-1057.jar!/application.conf: 19: Could not resolve substitution to a value: ${REQUIRED_CONTACT_POINT_NR}
#management-config
akka.management {
cluster.bootstrap {
contact-point-discovery {
# pick the discovery method you'd like to use:
discovery-method = kubernetes-api
required-contact-point-nr = ${REQUIRED_CONTACT_POINT_NR}
}
}
}
#management-configSo, I suppose that it is not getting the configuration from the kubernetes/akka-cluster.yml file: name: REQUIRED_CONTACT_POINT_NR. Changing it to required-contact-point-nr = 3 or 4 I get the error:
[error] SLF4J: A number (4) of logging calls during the initialization phase have been intercepted and are
[error] SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
[error] SLF4J: See also http://www.slf4j.org/codes.html#replay
...
[info] [2021-01-04 11:00:57,373] [INFO] [akka.remote.RemoteActorRefProvider$RemotingTerminator] [] [appka-akka.actor.default-dispatcher-3] - Shutting down remote daemon. MDC: {akkaAddress=akka://appka@127.0.0.1:25520, sourceThread=appka-akka.remote.default-remote-dispatcher-9, akkaSource=akka://appka@127.0.0.1:25520/system/remoting-terminator, sourceActorSystem=appka, akkaTimestamp=10:00:57.373UTC}
[info] [2021-01-04 11:00:57,376] [INFO] [akka.remote.RemoteActorRefProvider$RemotingTerminator] [] [appka-akka.actor.default-dispatcher-3] - Remote daemon shut down; proceeding with flushing remote transports. MDC: {akkaAddress=akka://appka@127.0.0.1:25520, sourceThread=appka-akka.remote.default-remote-dispatcher-9, akkaSource=akka://appka@127.0.0.1:25520/system/remoting-terminator, sourceActorSystem=appka, akkaTimestamp=10:00:57.375UTC}
[info] [2021-01-04 11:00:57,414] [INFO] [akka.remote.RemoteActorRefProvider$RemotingTerminator] [] [appka-akka.actor.default-dispatcher-3] - Remoting shut down. MDC: {akkaAddress=akka://appka@127.0.0.1:25520, sourceThread=appka-akka.remote.default-remote-dispatcher-9, akkaSource=akka://appka@127.0.0.1:25520/system/remoting-terminator, sourceActorSystem=appka, akkaTimestamp=10:00:57.414UTC}
[error] Nonzero exit code returned from runner: 255
[error] (Compile / run) Nonzero exit code returned from runner: 255
[error] Total time: 6 s, completed Jan 4, 2021 11:00:57 AMThe answer and the comments on my questions are right. I don't need to run sbt run to visualize the output at the web browser. What I was missing is to port-forward the current port of the cluster nodes to the output port. This is not specified on the akka-sample-cluster-kubernetes-scala, but I believe that is because they run directly on the Google Kubernetes Engine platform and not inside minikube first.
$ kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
appka-1 appka-8b88f7bdd-97zfl 1/1 Running 0 9m10s
appka-1 appka-8b88f7bdd-bhv44 1/1 Running 0 9m10s
appka-1 appka-8b88f7bdd-ff76s 1/1 Running 0 9m10s
$ #### THIS COMMAND IS NOT IN THE README FILE OF THE DEMO ####
$ kubectl port-forward appka-8b88f7bdd-ff76s 8080
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080now I can see the output:
$ http GET http://127.0.0.1:8080/
HTTP/1.1 200 OK
Content-Length: 11
Content-Type: text/plain; charset=UTF-8
Date: Wed, 06 Jan 2021 10:59:40 GMT
Server: akka-http/10.2.1
Hello worldYou are getting your contact point error because you are trying to use sbt run. sbt run will run a single instance outside of minikube, which isn't what you want. And since it's running outside of Minikube it won't pick up the environment variables being set in the container spec. The scripts do the build/deploy and you should not need to run sbt manually.
Also, the main error is not the connection to 8558, I don't believe that the configuration exposes that admin port outside of minikube.
The fact that all three containers report a status Running indicates that you may actually have a running cluster and the test script may just be missing the messages in the logs. As others have said in comments, the logs from one of the containers would be helpful in determining whether you have a working cluster, and diagnosing any problems in cluster formation.