If I have a service that is running on a cluster, can I deploy and add some new pods to the service dynamically?
For instance, if the service has the label "app=MyApp", can the new created pods with the same label "app=MyApp" be managed by the service?
Short answer - yes. And this is exactly what a kubernetes replicaSet enables, and these days you're most often encouraged to create one of those through a Deployment. Once created, you can scale up (or down) and the service will dynamically update it's internal forwarding based on the state of the pod's response to "Ready" (see the docs for a readiness probe).
The label/selector mechanism that you're describing is exactly how that works within Kubernetes in a variety of places, and service <-> Pods that match is one of the most common