I created 2 single master/single node clusters on bare metal servers (one for testing other for production)

The dev one works fine, however when I try to access the prod from a client machine, I am getting an unauthorized error. More specifically:

error: You must be logged in to the server (Unauthorized)

Needless to say when I SSH into the master of the prod cluster and do kubectl get nodes or any kubectl command, it works

On both servers, after setting up the respective clusters, I copied the .conf file from the respective master nodes to my client machine using scp

Both my clusters have the same authinfo, I am wondering if this is an issue.

The output of kubectl config get-contexts:

CURRENT   NAME                            CLUSTER        AUTHINFO           NAMESPACE
          kubernetes-admin@dev-cluster    dev-cluster    kubernetes-admin
*         kubernetes-admin@prod-cluster   prod-cluster   kubernetes-admin