Can't access chatbot in AKS using ingress

1/8/2020

I’ve been trying to get a MS Chatbot up and running in Azure and can’t seem to access it using the emulator and the ingress point. I can access it directly using it's public IP. (I'll remove the public ip once this works.) I don't seem to see anyting in the controller logs when I try to access it using the emulator.

I used https://docs.microsoft.com/en-us/azure/aks/ingress-tls and https://medium.com/@AliMazaheri/building-a-chat-bot-using-azure-aks-and-bot-framework-bfa1f698cc3c as inspiration.

Here are the versions of the tools I’m using:

azure-cli                          2.0.78
command-modules-nspkg              2.0.3
core                               2.0.78
nspkg                              3.0.4
telemetry                          1.0.4
docker                             2.1.0.5
helm
version.BuildInfo{Version:"v3.0.0", GitCommit:"e29ce2a54e96cd02ccfce88bee4f58bb6e2a28b6", GitTreeState:"clean", GoVersion:"go1.13.4"}

My docker file exposes ports 80, 443 and 3978.

I built the image with docker-compose and in the override file I have “7009:80” and “7010:3978”. Here’s my deployment file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: chat
  labels:
    app: chat
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: chat
      version: v1
  template:
    metadata:
      labels:
        app: chat
        version: v1
    spec:
      containers:
      - name: chat
        image: chatm3up.azurecr.io/talki/chat:v2-linux-latest
        env:
          ...
        ports:
        - containerPort: 3978
          protocol: TCP
        - containerPort: 80
        imagePullPolicy: Always

---

apiVersion: v1
kind: Service
metadata:
  name: chat
spec:
  selector:
    app: chat
    version: v1
  ports:
    - name: chat
      protocol: TCP
      port: 3978
      targetPort: 3978
    - name: http
      protocol: TCP
      port: 80
      targetPort: 80
  type: LoadBalancer

The service has a public IP and when I use the emulator it works fine.

I then installed nginx-ingress using helm:

helm install stable/nginx-ingress \
    --namespace default \
    --set controller.replicaCount=1 \
    --set controller.nodeSelector."beta\.kubernetes\.io/os"=linux \
    --set defaultBackend.nodeSelector."beta\.kubernetes\.io/os"=linux

I now have a controller and backend service. The controller has a public ip.

I’ve given controller a fqdn. Let’s call it chatty-ingress.canadacentral.cloudapp.azure.com. Calling it directly brings up the default bot page.

I created a cert manager using the instructions in the MS document.

# Install the CustomResourceDefinition resources separately
kubectl apply -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.8/deploy/manifests/00-crds.yaml

# Create the namespace for cert-manager
kubectl create namespace cert-manager

# Label the cert-manager namespace to disable resource validation
kubectl label namespace cert-manager certmanager.k8s.io/disable-validation=true

# Add the Jetstack Helm repository
helm repo add jetstack https://charts.jetstack.io

# Update your local Helm chart repository cache
helm repo update

# Install the cert-manager Helm chart
helm install \
  --name cert-manager \
  --namespace cert-manager \
  --version v0.8.0 \
  jetstack/cert-manager

Then I created the CertificateIssuer using the following deployment:

apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
  namespace: default
spec:
  acme:
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    email: me@here.com
    privateKeySecretRef:
      name: letsencrypt-staging
    http01: {}

I then created the ingress file.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: chatty-ingress
  namespace: default
  annotations:
    kubernetes.io/ingress.class: nginx
    certmanager.k8s.io/cluster-issuer: letsencrypt-staging
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  tls:
  - hosts:
    - chatty-ingress.canadacentral.cloudapp.azure.com
    secretName: tls-secret
  rules:
  - host: chatty-ingress.canadacentral.cloudapp.azure.com
    http:
      paths:
      - backend:
          serviceName: chat
          servicePort: 80 #Ive used 3978 as well.  With 80 the web pages comes up.
        path: /.*

Results from kubectl get svc -A

NAMESPACE      NAME                                     TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)                       AGE
cert-manager   cert-manager-webhook                     ClusterIP      y.y.y.y        <none>           443/TCP                       4d22h
default        apigateway-service                       LoadBalancer   y.y.y.y        x.x.x.x          80:31475/TCP                  28d
default        randomclient-service                     LoadBalancer   y.y.y.y        x.x.x.x          80:30397/TCP                  28d
default        identity-service                         LoadBalancer   y.y.y.y        x.x.x.x          80:30446/TCP                  28d
default        kubernetes                               ClusterIP      y.y.y.y        <none>           443/TCP                       29d
default        my-nginx-nginx-ingress-controller        LoadBalancer   y.y.y.y        x.x.x.x          80:31570/TCP,443:32741/TCP    4d18h
default        my-nginx-nginx-ingress-default-backend   ClusterIP      y.y.y.y        <none>           80/TCP                        4d18h
default        chat                                     LoadBalancer   y.y.y.y        x.x.x.x          3978:31738/TCP,80:31591/TCP   40m
default        rabbitmq                                 ClusterIP      y.y.y.y        <none>           5672/TCP                      28d
default        reports-service                          ClusterIP      y.y.y.y        <none>           80/TCP                        28d
kube-system    healthmodel-replicaset-service           ClusterIP      y.y.y.y        <none>           25227/TCP                     29d
kube-system    kube-dns                                 ClusterIP      y.y.y.y        <none>           53/UDP,53/TCP                 29d
kube-system    kubernetes-dashboard                     ClusterIP      y.y.y.y        <none>           80/TCP                        29d
kube-system    metrics-server                           ClusterIP      y.y.y.y        <none>           443/TCP                       29d
-- Maleki
azure-aks
botframework
kubernetes
kubernetes-ingress

0 Answers