Try this:

apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
  name: ingress-external
  namespace: gitlab-managed-apps
spec:
  selector:
    app == 'nginx'
  types:
    - Ingress
  ingress:
    - action: Deny
      source:
       nets:
       - x.x.x.x/32
    - action: Allow

calico docs suggests:

If one or more network policies apply to a pod containing ingress rules, then only the ingress traffic specifically allowed by those policies is allowed.

So this means that any traffic is denied by default and only allowed if you explicitly allow it. This is why adding additional rule action: Allow should allow all other traffic that was not matched by the previous rule.

Also remember what docs mention about rules:

A single rule matches a set of packets and applies some action to them. When multiple rules are specified, they are executed in order.

So default Allow rule has to follow the Deny rule for the specific IP, not the other way around.