Can you explain what you mean when you say you can access it by token but not through a kubeconfig? Kubeconfigs simply store authentication information in them, which can include authentication via a token. Assuming the rest of your kubeconfig file is populated, you just need to modify it so that your user information contains the token, like so:

users:
- name: my-user
  user:
    token: <token-here>