K
Q

Connecting to external networks from inside minikube VM behind proxy in docker container

1/7/2021

I have an active kubernetes cluster inside Minikube VM (using VirtualBox as driver), so for deploying new containers I am able to download the images as this connection is already laid out using istio service, now if I ssh into my minikube VM first of all I am not able to wget https content but http contents are connected after setting proxies and no_proxies but if I want to access any link from inside of my containers, say simple pod with python image and urllib library and I want to connect from inside this pod and then print the contents from any link (eg.http://python.org) I am not able to do so, all I am getting is no route to host error in logs which points to some problem with the connection due to proxies.

def basic():
	import urllib.request
	print("inside basic funtion")
	with urllib.request.urlopen('http://python.org/') as response:
   		html = response.read()
   		print(html)

this is the python code I am running from inside my container as a pipeline component.

Most recent error I got-

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/urllib/request.py", line 1317, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/usr/local/lib/python3.7/http/client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.7/http/client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.7/http/client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.7/http/client.py", line 1016, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.7/http/client.py", line 956, in send
    self.connect()
  File "/usr/local/lib/python3.7/http/client.py", line 928, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/usr/local/lib/python3.7/socket.py", line 727, in create_connection
    raise err
  File "/usr/local/lib/python3.7/socket.py", line 716, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 110] Operation timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "<string>", line 11, in <module>
  File "<string>", line 3, in basic
  File "/usr/local/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/local/lib/python3.7/urllib/request.py", line 525, in open
   response = self._open(req, data)
  File "/usr/local/lib/python3.7/urllib/request.py", line 543, in _open
    '_open', req)
  File "/usr/local/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/local/lib/python3.7/urllib/request.py", line 1345, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/local/lib/python3.7/urllib/request.py", line 1319, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 110] Operation timed out>

I have started minikube as-

minikube start --cpus 6 --memory 12288 --disk-size=80g --extra-config=apiserver.service-account-issuer=api --extra-config=apiserver.service-account-signing-key-file=/var/lib/minikube/certs/apiserver.key --extra-config=apiserver.service-account-api-audiences=api --kubernetes-version v1.14.0

after setting the env variables as well.

Update: I created a different container just to check the curl from inside the component as- (I am using kfp libraries for creating containers)

def curl_op(text):
    return dsl.ContainerOp(
        name='curl',
        image='tutum/curl',
        command=['sh', '-c'],
        arguments=['curl -x http://<proxy-server>:<proxy-port> "$0"', text]
    )

so using the above argument I am able to connect to external links, which again makes it certain that i need to create the containers with proxies set.

-- Chinmay Singh
docker
kubeflow
kubernetes
minikube
urllib

Similar Questions

"gcloud container clusters create fails" with "container.clusters.create" permission error
can an instance of pilot configured to query both k8s and consul?
traefik ingress custom error in kubernetes
Check the feature is enabled /disabled in kubernetes
Kubernetes - Passing in environment variable and service name (from DNS)

1 Answer

2/3/2021

So for running the above python code I mentioned as pipeline component. I added the environment variables using the os library and this individual piece was able to connect to external networks. Updated python code-

def basic():
    import urllib.request
    import os
    proxy = 'http://proxy-path:port'

	os.environ['http_proxy'] = proxy 
	os.environ['HTTP_PROXY'] = proxy
	os.environ['https_proxy'] = proxy
	os.environ['HTTPS_PROXY'] = proxy

    print("inside basic funtion")
    with urllib.request.urlopen('http://python.org/') as response:
        html = response.read()
        print(html)

And if the docker image is created from scratch without taking help of pipeline library function then we need to just add the env details into our dockerfile the usual way after the base image call-

ENV HTTP_PROXY http://proxy-path:port
ENV HTTPS_PROXY http://proxy-path:port
-- Chinmay Singh
Source: StackOverflow