I need to deploy Kubeflowfor ML pipeline and TFX on GCP. Unfortunately I cannot use the following UI for the installation, because I need to setup manually, the Zone, Network and Subnet.
There is a nice documentation page: https://www.kubeflow.org/docs/gke/deploy/deploy-cli/
I tried with both OAuth credentials and basic authentication. I also tried to install kfctl_v0.5.1_linux.tar.gz
and kfctl_v0.5.0_linux.tar.gz
. I always got the following:
WARN[0036] could not open /home/user/.kube/config Error stat /home/user/.kube/config: no such file or directory filename=“apps/group.go:188”
WARN[0036] could not load config Error: open /home/user/.kube/config: no such file or directory filename=“apps/group.go:208"
I have kubectl installed:
kubectl version
Client Version: version.Info{Major:"1", Minor:"12+", GitVersion:"v1.12.8-dispatcher", GitCommit:"1215389331387f57594b42c5dd024a2fe27334f8", GitTreeState:"clean", BuildDate:"2019-05-1
3T18:09:56Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server localhost:8080 was refused - did you specify the right host or port?
Clearly ~/.kube/config doesn't exist. Which application is suppose to create it ?
ls ~/.kube/config
ls: cannot access '/home/user/.kube/config': No such file or directory
I did the following:
ls home/user/folder/kubeflow
kfctl kfctl_v0.5.1_linux.tar.gz
export KUBEFLOW_USERNAME=xxx
export KUBEFLOW_PASSWORD=xxx
export PATH=$PATH:/home/user/folder/kubeflow
export ZONE=europe-west1-b
export PROJECT=project
export KFAPP=kubeflow-test
from home/user/folder/kubeflow:
kfctl init ${KFAPP} --platform gcp --project ${PROJECT} --use_basic_auth -V
INFO[0014] Not skipping GCP project init, running gcpInitProject. filename="gcp/gcp.go:1619"
WARN[0017] batch API enabling is running: [deploymentmanager.googleapis.com servicemanagement.googleapis.com container.googleapis.com cloudresourcemanager.googleapis.com endpoints.googleapis.com file.googleapis.com ml.googleapis.com iam.googleapis.com sqladmin.googleapis.com] (op = operations/acf.954cc3b6-f1f4-46a2-832d-596ccb5a3d5a) filename="gcp/gcp.go:1594"
WARN[0018] batch API enabling is running: [deploymentmanager.googleapis.com servicemanagement.googleapis.com container.googleapis.com cloudresourcemanager.googleapis.com endpoints.googleapis.com file.googleapis.com ml.googleapis.com iam.googleapis.com sqladmin.googleapis.com] (op = operations/acf.954cc3b6-f1f4-46a2-832d-596ccb5a3d5a) filename="gcp/gcp.go:1594"
WARN[0019] batch API enabling is running: [deploymentmanager.googleapis.com servicemanagement.googleapis.com container.googleapis.com cloudresourcemanager.googleapis.com endpoints.googleapis.com file.googleapis.com ml.googleapis.com iam.googleapis.com sqladmin.googleapis.com] (op = operations/acf.954cc3b6-f1f4-46a2-832d-596ccb5a3d5a) filename="gcp/gcp.go:1594"
WARN[0021] batch API enabling is running: [deploymentmanager.googleapis.com servicemanagement.googleapis.com container.googleapis.com cloudresourcemanager.googleapis.com endpoints.googleapis.com file.googleapis.com ml.googleapis.com iam.googleapis.com sqladmin.googleapis.com] (op = operations/acf.954cc3b6-f1f4-46a2-832d-596ccb5a3d5a) filename="gcp/gcp.go:1594"
WARN[0024] batch API enabling is running: [deploymentmanager.googleapis.com servicemanagement.googleapis.com container.googleapis.com cloudresourcemanager.googleapis.com endpoints.googleapis.com file.googleapis.com ml.googleapis.com iam.googleapis.com sqladmin.googleapis.com] (op = operations/acf.954cc3b6-f1f4-46a2-832d-596ccb5a3d5a) filename="gcp/gcp.go:1594"
WARN[0027] batch API enabling is running: [deploymentmanager.googleapis.com servicemanagement.googleapis.com container.googleapis.com cloudresourcemanager.googleapis.com endpoints.googleapis.com file.googleapis.com ml.googleapis.com iam.googleapis.com sqladmin.googleapis.com] (op = operations/acf.954cc3b6-f1f4-46a2-832d-596ccb5a3d5a) filename="gcp/gcp.go:1594"
WARN[0030] batch API enabling is running: [deploymentmanager.googleapis.com servicemanagement.googleapis.com container.googleapis.com cloudresourcemanager.googleapis.com endpoints.googleapis.com file.googleapis.com ml.googleapis.com iam.googleapis.com sqladmin.googleapis.com] (op = operations/acf.954cc3b6-f1f4-46a2-832d-596ccb5a3d5a) filename="gcp/gcp.go:1594"
INFO[0037] batch API enabling is completed: [deploymentmanager.googleapis.com servicemanagement.googleapis.com container.googleapis.com cloudresourcemanager.googleapis.com endpoints.googleapis.com file.googleapis.com ml.googleapis.com iam.googleapis.com sqladmin.googleapis.com] filename="gcp/gcp.go:1590"
INFO[0037] reading from /home/user/folder/kubeflow/kubeflow-test/app.yaml filename="coordinator/coordinator.go:341"
WARN[0037] could not open /home/user/.kube/config Error stat /home/user/.kube/config: no such file or directory filename="apps/group.go:188"
WARN[0037] could not load config Error: open /home/user/.kube/config: no such file or directory filename="apps/group.go:208"
Some file were created:
ls kubeflow-test/
app.yaml
Some check:
kubectl config view
apiVersion: v1
clusters: []
contexts: []
current-context: ""
kind: Config
preferences: {}
users: []
I also try to update kubectl
gcloud components update kubectl
You have specified individual components to update. If you are trying
to install new components, use:
$ gcloud components install kubectl
Do you want to run install instead (y/N)? y
All components are up to date.
Any idea why I don't have ~/.kube/config
doesn't exist ? I try to create it manually but then I have other issue later. What should I do to create the mising configuration. Is there another recommend way than using CloudShell to deploy Kubelow with cli on GCP ?
GKE has a great description of how you can configure access to a cluster via kubectl (which uses the ~/.kube/config
file as the default location to store credentials). There are two ways to populate the file:
gcloud container clusters create CLUSTER_NAME
gcloud container clusters get-credentials CLUSTER_NAME
From the Kubeflow docs, it looks like the init step doesn't create a cluster; the apply step should create a cluster. You don't describe what issue you have later if you create an empty file.
Also, you pointed to the deploy cli docs, but the customizing Kubeflow on GKE page sounds more like what you are trying to accomplish so you might see if that page answers any of your questions.