But how can I get the ReplicaSet based on the deployment?

With quite some gymnastics... If you inspect how kubectl does it (by executing kubectl -n my-namespace describe deploy my-deployment --v=9) you will see that it dos the following:

• first gets deployment details with: /apis/extensions/v1beta1/namespaces/my-namespace/deployments/my-deployment. From there it gets labels for replicaset selection.
• then gets replicaset details using labels from deployment in previous step (say labels were my-key1:my-value1 and my-key2:my-value2) like so: /apis/extensions/v1beta1/namespaces/my-namespace/replicasets?labelSelector=my-key1%3Dmy-value1%2Cmy-key2%3Dmy-value2

Funny part here begins with extracting multiple labels from deployment output and formatting them for replicaset call, and that is task for grep, awk, jq or even python, depending on your actual use case (from bash, python, some client or whatever...)