Integrate locally executed service (minikube) into Kubernetes cluster

4/6/2020

I am currently in the process of designing a Kubernetes cluster for an existing setup which includes around ten services. Previously, I've developed and tested these services locally with minikube and later on I published them a self-hosted Kubernetes instance on a different physical machine (let's call this the staging system). Some of these deployed services need to access resources on other machines in the same network. This means that my development environment (minikube) also needs access to those external resources. Here is a short "mockup" of one process in this cluster.

+-----------+
| External  |
| Service X |
+-----------+
    ^
    |
+-----------+       +-----------+
| Service A |  <--  | Service B |
+-----------+       +-----------+

Due to some networking changes it is no longer possible for me to access these remote resources, that means that External Service X is no longer reachable from my local machine. I'd know ask if it would be somehow possible to host the Service B on my local machine, but to "connect" it to my remote cluster. Is it somehow possible to use minikube as a simple node for an existing Kubernetes instance? I rely heavily on service discovery with CoreDNS which means I'd like to find an infrastructure-level solution instead of an application-level approach. It would be ideal if the staging system didn't even really know that my Service B is not locally hosted. I am sorry if this is maybe a stupid question but it seems like I am a bit out of knowledge about Kubernetes at the moment.

-- user13237363
cluster-computing
kubernetes
minikube
workflow

1 Answer

4/9/2020

If you have two separate kubernetes clusters e.g. local instance of Minikube and some other remote kubernetes cluster and you want them to be able to interact with one another, you can use for that purpose so called Services without selectors.

You can treat your remote kubernetes cluster as any other external resource (e.g. database server running on a separate machine/vm) and you can expose it internally to your cluster. Typically we expose something which is running on our k8s cluster externally but in this case we want to expose an external resource internally to our cluster as if it was one more internal resource. From Pods perspective it actually looks like an internal resource, which is available under its local cluster domain, in exactly the same way as internal resources, exposed by simple ClusterIP service.

Let's take a quick look how to achive it. It's very well explained in this section of the official kubernetes docs:

Services most commonly abstract access to Kubernetes Pods, but they can also abstract other kinds of backends. For example:

  • You want to have an external database cluster in production, but in your test environment you use your own databases.
  • You want to point your Service to a Service in a different Namespace or on another cluster.
  • You are migrating a workload to Kubernetes. Whilst evaluating the approach, you run only a proportion of your backends in Kubernetes.

In any of these scenarios you can define a Service without a Pod selector. For example:

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 9376

Because this Service has no selector, the corresponding Endpoint object is not created automatically. You can manually map the Service to the network address and port where it’s running, by adding an Endpoint object manually:

apiVersion: v1
kind: Endpoints
metadata:
  name: my-service
subsets:
  - addresses:
      - ip: 192.0.2.42
    ports:
      - port: 9376

If there is only network connectivity between mentioned dev and staging environments you can make use of Services without selectors. If you need your Minikube cluster to be able to connect to something exposed on 192.0.2.42:9376 by your staging cluster, you just need to use this IP in your Endpoints object definition. Service is defined to listen on por 80 so it will be eventually exposed internally to you Minikube Pods under this standard port.

The biggest advantage of such solution is that such Service makes specific external resource available internally to apps runnig in Pods on your local k8s cluster and they can refer to it using its DNS name i.e. service-name within the same namespace or by FQDN <my-service-name>.<namespace>.svc.cluster.local from any namespace, cluster-wide.

If your external resource is available under a domain name you may also consider using ExternalName, which is a special type of service without selector, and in this case you don't have to configure manually any Endpoints objects. Its definition may look like this:

apiVersion: v1
kind: Service
metadata:
  name: my-service
  namespace: prod
spec:
  type: ExternalName
  externalName: my.database.example.com
-- mario
Source: StackOverflow