All system services in Kubernetes are deployed to a namespace, usually called kube-system. Where does that come from?

As noted in the nice documentation there are three namespaces that Kubernetes initially starts wtih:

• default - The default namespace for objects with no other namespace.
• kube-system - The namespace for objects created by the Kubernetes system.
• kube-public - The namespace is created automatically and readable by all users (including those not authenticated). This namespace is mostly reserved for cluster usage, in case that some resources should be visible and readable publicly throughout the whole cluster. The public aspect of this namespace is only a convention, not a requirement.

You can change default namespace to any namespace of your liking using kubectl config context handling.

What if I would like to change that to another namespace?

That would be a convoluted and rather risky undertaking... For kubeadm created cluster you can find appropriate manifests in /etc/kubernetes/manifests but it is not just sufficient to change namespace there, there is an array of config maps, certificates and things to consider namespace-wise. And even if you manage to do so there is reason behind the deprication of api-server flag master-service-namespace since you can break GKE implicit references and similar issues can arise. It all boils down to that it is not really advisable to change kube-system namespace.

Edit:

Below is excerpt from kuberentes source where you can see how those namespaces are initially defined.

// NamespaceDefault means the object is in the default namespace which is applied when not specified by clients
NamespaceDefault string = "default"
// NamespaceAll is the default argument to specify on a context when you want to list or filter resources across all namespaces
NamespaceAll string = ""
// NamespaceNone is the argument for a context when there is no namespace.
NamespaceNone string = ""
// NamespaceSystem is the system namespace where we place system components.
NamespaceSystem string = "kube-system"
// NamespacePublic is the namespace where we place public info (ConfigMaps)
NamespacePublic string = "kube-public"

You can find more references to kube-system through the codebase, here is another example:

// "kube-system" is the default scheduler lock object namespace
SchedulerDefaultLockObjectNamespace string = "kube-system"

And so on...