The pod in the cluster would only try to access the clusterip defined in the kubernetes.default.svc, the clusterip is nothing different with the other cluster ip except the svc's name.

Absolutely correct

So how can a http request to the specific clusterip be routed to the api server, does it configured by the api server proxy when create the kubernetes.default.svc?

This magic happens via kube-proxy, which usually delegates down to iptables, although I think it more recent kubernetes installs they are using ipvs to give a lot more control over ... well, almost everything. The kube-proxy receives its instructions from the API informing it of any changes, which it applies to the individual Nodes to keep the world in sync.

If you have access to the Nodes, you can run sudo iptables -t nat -L -n and see all the KUBE-SERVICE-* rules that are defined -- usually with helpful comments, even -- and see how they are mapped from the ClusterIP down to the Pod's IP of the Pods which match the selector on the Service