K
Q

Setting up Spring Cloud Data Flow on Kubernetes

11/20/2016

Do I need to install an instance of Spring Cloud Data Flow on the master server myself, or is this getting installed "automatically" as part of the deployment?

This isn't quite clear from the description at http://docs.spring.io/spring-cloud-dataflow-server-kubernetes/docs/current-SNAPSHOT/reference/htmlsingle/#_deploying_streams_on_kubernetes

I've followed the guide, though removed every config for MySQL. Maybe this is required. Though I'm somewhat stuck since it's just not assigning an external IP and I do not see why, how to debug, and whether I missed to install some required component.

Edit: To clarify, I see a scdf service entry when I run

kubectl get svc

But this service never gets an external IP.

-- benjist
apache-kafka
kubernetes
spring-cloud-dataflow

Similar Questions

Managing authentication across micro-services
Which approach is better for discovering container readiness?
Kubernetes - Creating a specific namespace for "services"
Kubernetes pod gets recreated when deleted

3 Answers

12/19/2016

MySql is required, that's why it's in the steps.

Spring Cloud Data Flow uses an RDBMS instead of Redis for stream/task definitions, application registration, and for job repositories.

You can also use any of the other supported RDMBSes.

-- Remon Sinnema
Source: StackOverflow
11/21/2016

Do I need to install an instance of Spring Cloud Data Flow on the master server myself, or is this getting installed "automatically" as part of the deployment?

Spring Cloud Data Flow server needs to be setup either outside (that knows how to connect to the kubernetes environment) or you can use the Spring Cloud Data Flow server docker image to run inside the kubernetes while the latter approach is better.

Step 6 in the link you posted above runs the SCDF docker image inside the kubernetes cluster:

```

Deploy the Spring Cloud Data Flow Server for Kubernetes using the Docker image and the configuration settings you just modified.

$ kubectl create -f src/etc/kubernetes/scdf-config-kafka.yml

$ kubectl create -f src/etc/kubernetes/scdf-secrets.yml

$ kubectl create -f src/etc/kubernetes/scdf-service.yml

$ kubectl create -f src/etc/kubernetes/scdf-controller.yml

```

-- Ilayaperumal Gopinathan
Source: StackOverflow
4/22/2020

You can install it using Helm Charts.

https://dataflow.spring.io/docs/installation/kubernetes/helm/

  • At first install Helm
  • Then install Spring Cloud Data Flow

helm install --name my-release stable/spring-cloud-data-flow

It will install and config relevant pods such as spring-cloud-dataflow-server, mysql, skipper, rabbitmq, etc.

Also you can customize versions and configurations.

-- Mojtaba
Source: StackOverflow