Do I then need to setup encryption for these services or is in-cluster-communication considered as secure?

The level of security you want to use is up to you. In regulated industries, e.g. in banks, it is popular to apply a zero trust security architecture, where no network is considered secure - e.g. in this case, it is common to use mutual TLS between applications within the cluster - with both authentication, authorization and encryption. On Kubernetes its common to use a service mesh like e.g. Istio to implement this.

In-cluster networking is typically its own local network, it is up to you to consider that secure enough for your use-case.

If I have two services ServiceA and ServiceB. Both are of ServiceType ClusterIP, so if I understand correctly both services are not accessible from outside of the cluster.

Commonly, yes. But there are now common with load balancers that can route traffic to applications with Service type ClusterIP. This depends on what load balancer / Gateway you use.