I have deployed a simple dotnet core app into Kubernetes. The service which is exposed is as below
apiVersion: v1
kind: Service
metadata:
creationTimestamp: "2020-01-17T18:07:23Z"
labels:
app.kubernetes.io/instance: expo-api
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: expo-api
app.kubernetes.io/version: 0.0.4
helm.sh/chart: expo-api-0.0.4
name: expo-api-service
namespace: default
resourceVersion: "997971"
selfLink: /api/v1/namespaces/default/services/expo-api-service
uid: 144b9d1d-87d2-4096-9851-9563266b2099
spec:
clusterIP: 10.12.0.122
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
selector:
app.kubernetes.io/instance: expo-api
app.kubernetes.io/name: expo-api
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}The ingress controller I am using is nginx ingress controller and the simple ingress rules are set as below -
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: "true"
creationTimestamp: "2020-01-17T18:07:24Z"
generation: 3
labels:
app.kubernetes.io/instance: expo-api
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: expo-api
app.kubernetes.io/version: 0.0.4
helm.sh/chart: expo-api-0.0.4
name: expo-api
namespace: default
resourceVersion: "1004650"
selfLink: /apis/extensions/v1beta1/namespaces/default/ingresses/expo-api
uid: efef4e15-ed0a-417f-8b34-4e0f46cb1e70
spec:
rules:
- http:
paths:
- backend:
serviceName: expo-api-service
servicePort: 80
path: /expense
status:
loadBalancer:
ingress:
- ip: 34.70.45.62The dotnet core app which has a simple start up -
public class Startup
{
public Startup(IConfiguration configuration)
{
Configuration = configuration;
}
public IConfiguration Configuration { get; }
// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
services.AddControllers();
}
// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
app.UseHttpsRedirection();
app.UseRouting();
app.UseAuthorization();
app.UseEndpoints(endpoints =>
{
endpoints.MapControllers();
});
}
}
This is the ingress output -
Name: expo-api
Namespace: default
Address: 34.70.45.62
Default backend: default-http-backend:80 (10.8.0.9:8080)
Rules:
Host Path Backends
---- ---- --------
*
/expense expo-api-service:80 (10.8.0.26:80,10.8.0.27:80,10.8.1.14:80)
Annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: true
Events: <none>Below is the nginx ingress controller setup -
Name: nginx-nginx-ingress-controller
Namespace: default
Labels: app=nginx-ingress
chart=nginx-ingress-1.29.2
component=controller
heritage=Helm
release=nginx
Annotations: <none>
Selector: app=nginx-ingress,component=controller,release=nginx
Type: LoadBalancer
IP: 10.12.0.107
LoadBalancer Ingress: 34.66.164.70
Port: http 80/TCP
TargetPort: http/TCP
NodePort: http 30144/TCP
Endpoints: 10.8.1.6:80
Port: https 443/TCP
TargetPort: https/TCP
NodePort: https 30469/TCP
Endpoints: 10.8.1.6:443
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>The issue is when I am changing the ingress rules path to only / and access using - curl 34.66.164.70/weatherforecast it works perfectly fine .
However when I change the ingress path to /expense and try to access using - curl 34.66.164.70/expense/weatherforecast . The output is an error as -
{
"type": "https://tools.ietf.org/html/rfc7231#section-6.5.1",
"title": "One or more validation errors occurred.",
"status": 400,
"traceId": "|4dec8cf0-4fddb4d168cb9569.",
"errors": {
"id": [
"The value 'weatherforecast' is not valid."
]
}
}
I am unable to understand what is the issue behind this. Whether it is appearing from dotnet core side or Kubernetes? If dotnet what may be the resolution and if on Kubernetes what is the expected resolution.
Updated 1
I cant see nginx.ingress.kubernetes.io/rewrite-target annotation in your Ingress object. Can't say if you skipped it intentionally.
If this annotation is not present, your app receives "GET: /expense/weatherforecast". If this is what you want, everything fine. But if you want your app receive "GET: /weatherforecast", you should add nginx.ingress.kubernetes.io/rewrite-target: / to you Ingress annotation.
Updated 2
Ingress-nginx documentation has article about "rewrite" annotation: https://kubernetes.github.io/ingress-nginx/examples/rewrite/#rewrite-target
There is pretty concise example which helps to understand how to expose /expense/weatherforecast endpoint. But unfortunately, I couldn't achieve /expense exposal as well. I tried more complex regex, and i tried "app-root" annotation from that article, but nothing worked - Ingress always returned 404 for /expense endpoinnt.
I couldn't find any useful information of how to handle both root and relative endpoints, so i began to improvise. I found out that you can use two backed paths to make it work. Don't know, if this is a bug or feature :).
Following ingress spec works great on my test app. It can handle both /expense and /expense/weatherforecast correctly.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: "nginx"
# nginx.ingress.kubernetes.io/use-regex: "true" # can be true or false, no matter
nginx.ingress.kubernetes.io/rewrite-target: "/$1"
name: app
spec:
rules:
- http:
paths:
- backend:
serviceName: app
servicePort: 80
path: "/expense/(.+)" # handle relative path
- backend:
serviceName: app
servicePort: 80
path: "/expense" # handle rootORIGINAL :Thanks to @heyzling insight I found out the solution to it . I changed the app path into the code startup.cs. The issue was the Api originally was not expecting a route prefix for all controllers . Hence it was giving the error . So that I had to do a slight change in the startup.cs to add app.UsePathBase("/expense") . Below is the configuration which I added -
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
app.UsePathBase("/expense"); // this is the added configuration which identifies the ingress path rule individually.
app.UseHttpsRedirection();
app.UseRouting();
app.UseAuthorization();
app.UseEndpoints(endpoints =>
{
endpoints.MapControllers();
});
}
Ideally I feel this is isn't a good design since kubernetes ingress and dotnet core routes should know nothing about each other . They ideally should not be dependent on each other to comply routing rules . If someone has better solution ? Please do post . The above one solves my purpose but I am not happy with it .
----------------------------------------------------------------------------------UPDATE 2: Thanks to @heyzling . I finally found the solution - It looks like it had to rewrite the url and forward the actual API url which dotnet code expecting to the docker image which is running .
Here is the code example -
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/use-regex: "true"
labels:
app.kubernetes.io/name: expo-api
name: expo-api
namespace: default
spec:
rules:
- http:
paths:
- backend:
serviceName: expo-api-service
servicePort: 80
path: /expense(/|$)(.*)So now you can do both -
curl 35.192.198.231/expense/weatherforecast
curl 35.192.198.231/expense/fakeapiit would rewrite and forward the url as -
localhost:80/weatherforecast
localhost:80/fakeapiinside the container. Hence it works as expected . In that way we DO NOT require app.UsePathBase("/expense") anymore and both dotnet core and ingress does not have to know anything about each other.