Kubernetes: What's the idiomatic way in K8s to setup a custom proxy between ingress and its services?

9/4/2019

At present we have a lot of ASP.net WebAPI service applications hosted on premises. We are planning to move these to Azure AKS. We've identified a lot of common code across these applications which is mostly implemented as ASP.Net reusable middleware components so that the logic is not duplicated in code.

In a K8s environment it makes sense to offload this common functionality to one or more proxy applications which intercepts the requests being forwarded from the ingress to the services (assuming this is the correct approach). Some of the request inspection / manipulation logic is based on the service host and path to be defined in the ingress and even on the headers in the incoming requests.

For e.g. I considered using OAuth2_proxy but found that even though authentication is quite easy to implement, Azure AD group based authorization is impossible to do out of the box with that. So what's the idiomatic way one goes about setting up such a custom proxy application? (I'm familiar with using libraries such as ProxyKit middleware in ASP.Net to develop http proxies.)

One approach that comes to mind is to deploy such proxies as sidecar containers in each service application pod but that would mean there'd be unnecessary resource usage by all such duplicate container instances in each pod. I don't see the benefit over the use of middleware components as mentioned previously. :(

The ideal setup would be ingress --> custom proxy 1 --> custom proxy 2 --> custom proxy n --> service where custom proxies would be separately deployable and scalable.

-- Harindaka
http-proxy
kubernetes
kubernetes-ingress
nginx-ingress
proxy

1 Answer

9/5/2019

So after a lot of reading and googling I found that the solution was to use API Gateways that are available as libraries (preferrably based on .Net):

Ocelot placed behind the nginx ingress fits the bill perfectly

Ocelot is a .NET API Gateway. This project is aimed at people using .NET running a micro services / service oriented architecture that need a unified point of entry into their system. However it will work with anything that speaks HTTP and run on any platform that ASP.NET Core supports.

Ocelot is currently used by Microsoft and Tencent.

The custom middleware and header/query/claims transformation solves my problem. Here are some worthy links

Microsoft Docs: Implement API Gateways with Ocelot

Ocelot on Github

Ocelot Documentation

Features

A quick list of Ocelot's capabilities for more information see the documentation.

  • Routing

  • Request Aggregation

  • Service Discovery with Consul & Eureka

  • Service Fabric

  • Kubernetes

  • WebSockets

  • Authentication

  • Authorisation

  • Rate Limiting

  • Caching

  • Retry policies / QoS

  • Load Balancing

  • Logging / Tracing / Correlation

  • Headers / Query String / Claims Transformation

  • Custom Middleware / Delegating Handlers

  • Configuration / Administration REST API

  • Platform / Cloud Agnostic

-- Harindaka
Source: StackOverflow