K
Q

Telegram bot: Wrong response from the webhook: 426 Upgrade Required

1/16/2021

I'm trying to run Telegram bot using webhook and always have the same error while trying to perform any request e.g. https://api.telegram.org/botTOKEN/getWebhookInfo.

My setup:

Kubernetes/Istio -> Istio Gateway -> nginx -> python-telegram-bot

  1. I issued a certificate with LetsEncrypt and set it for Istio gateway for a sub domain as I always do. If I go here https://www.ssllabs.com/ssltest/analyze.html to check my certificate it returns that everything is set correctly and I can see certificate information.

  2. I setup nginx behind istio gateway (it is already used for other endpoints so I just add the following rules). Gateway forwards 443 port to nginx 80:

server {
        listen 80;
        server_name DOMAIN;

        location /${TG_BOT_TOKEN} {
            proxy_pass http://pp-telegram-bot.default.svc.cluster.local:8000/${TG_BOT_TOKEN}/;
        }

        location /check {
            return 200 'true';
        }
    }

I checked that this setup works correctly with certificate by making a request to /check endpoint with https - all works fine.

  1. Next I setup a bot with python-telegram-bot
bot = telegram.Bot(token=TG_BOT_TOKEN)

def main():
    updater = Updater(bot=bot, use_context=True)

    dispatcher = updater.dispatcher
    # add handlers

    updater.start_webhook(listen='0.0.0.0', port=8000, url_path=TG_BOT_TOKEN)
    updater.bot.set_webhook(f'https://{DOMAIN}/{TG_BOT_TOKEN}')
    updater.idle()

if __name__ == '__main__':
    main()
  1. I run the setup. Everything is working, no crash. Trying to check with: https://api.telegram.org/botTOKEN/getWebhookInfo and got the following response:
{
    "ok": true,
    "result": {
        "url": "https://DOMAIN/TOKEN",
        "has_custom_certificate": false,
        "pending_update_count": 1,
        "last_error_date": 1610810736,
        "last_error_message": "Wrong response from the webhook: 426 Upgrade Required",
        "max_connections": 40,
        "ip_address": IP_ADDRESS
    }
}

No logs for bot app available. Nginx logs display only a general one liner error: "POST /TOKEN HTTP/1.1" 426 0 "-" "-" "10.244.1.13"

What I tried:

  • Export certificate from k8s to pem file and use it in set_webhook. But that should not be the case since it is not self signed certificate and look like SSL part works ok (maybe not).
  • Use 127.0.0.1 instead of 0.0.0.0
  • remove updater.idle()
  • Setting webhook by myself from terminal
  • Reread this manual many times https://core.telegram.org/bots/webhooks
  • I even tried to set webhook url to /check for Telegram and it actually stats return correct response (probably because this endpoint returns just 200) but obviously there is no bot behind that URL.

All the above tells me that probably something is wrong with the bot setup itself, but everything looks correct according to python-telegram-bot manual.

Last but not least, if I use getUpdates instead of webhook bot works perfectly fine.

So I have no idea what this 426 error means in that context and how to make it work.

Headers: enter image description here

-- Max
kubernetes
nginx
python
python-telegram-bot

Similar Questions

Can django project and django app have different docker image?
How do kubernetes services balance load internally?
GKE Kubernetes RBAC bind default role to my limited custom
How to remove (delete) annotation in Kubernetes
Why can't access my gRPC REST service that is running in Minikube?

2 Answers

1/16/2021

The problem is caused by nginx .Every requests that passed through nginx to your python-telegram-bot will return the HTTP Status "426 Upgrade Required." By default, nginx still uses HTTP/1.0 for upstream connections while istio via the envoy proxy does not support HTTP/1.0 So You need to force Nginx to use HTTP/1.1 for upstream connections.

server {
        listen 80;
        server_name DOMAIN;

        location /${TG_BOT_TOKEN} {
            proxy_pass http://pp-telegram-bot:8000/${TG_BOT_TOKEN}/;
         proxy_http_version 1.1; # there is will force 1.1 
        }

        location /check {
            return 200 'true';
        }
    }
-- Shay
Source: StackOverflow
1/16/2021

You must explicitly set proxy_http_version to 1.1 to make it work, otherwise it uses 1.0 by default.

server {
    listen 80;
    server_name DOMAIN;

    location /${TG_BOT_TOKEN} {
        proxy_http_version  1.1;
        proxy_pass http://pp-telegram-bot.default.svc.cluster.local:8000/${TG_BOT_TOKEN}/;
    }
}
-- Max
Source: StackOverflow