Here is my setup.

I have 2 AWS accounts. 1) Applications account 2) Monitoring account

Application account has EKS + Istio + Application related microservices + promtail agents.

Monitoring account has centralized logging system within EKS + Istio + (Grafana & Prometheus & loki pods running)

From Applications account, I want to push logs to Loki on Monitoring a/c. I tried exposing Loki service outside monitoring a/c but I am facing issues to set loki url as https://<DNS_URL>/loki. This change I tried by using suggestions at here and here, but that is not working for me. I have installed the loki-stack from this url

The question is how can I access loki URL from applications account so that it can be configured in promtail in applications a/c? Please note both accounts are using pods within EKS and not standalone loki or promtail.

Thanks and regards.

apiVersion: v1                                
kind: Service                                 
metadata:                                     
  annotations:                                
    meta.helm.sh/release-name: loki           
    meta.helm.sh/release-namespace: monitoring
  creationTimestamp: "2021-10-25T14:59:20Z"   
  labels:                                     
    app: loki                                 
    app.kubernetes.io/managed-by: Helm        
    chart: loki-2.5.0                         
    heritage: Helm                            
    release: loki                             
  name: loki                                  
  namespace: monitoring                       
  resourceVersion: "18279654"                 
  uid: 7eba14cb-41c9-445d-bedb-4b88647f1ebc   
spec:                                         
  clusterIP: 172.20.217.122                   
  clusterIPs:                                 
  - 172.20.217.122                            
  ports:                                      
  - name: metrics                             
    port: 80                                  
    protocol: TCP                             
    targetPort: 3100                          
  selector:                                   
    app: loki                                 
    release: loki                             
  sessionAffinity: None                       
  type: ClusterIP                             
status:                                       
  loadBalancer: {}                            
---

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  generation: 14
  name: grafana-vs
  namespace: monitoring
  resourceVersion: "18256422"
  uid: e8969da7-062c-49d6-9152-af8362c08016
spec:
  gateways:
  - my-gateway
  hosts:
  - '*'
  http:
  - match:
    - uri:
        prefix: /grafana/
    name: grafana-ui
    rewrite:
      uri: /
    route:
    - destination:
        host: prometheus-operator-grafana.monitoring.svc.cluster.local
        port:
          number: 80
  - match:
    - uri:
        prefix: /loki
    name: loki-ui
    rewrite:
      uri: /loki
    route:
    - destination:
        host: loki.monitoring.svc.cluster.local
        port:
          number: 80

---
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"networking.istio.io/v1alpha3","kind":"Gateway","metadata":{"annotations":{},"name":"my-gateway","namespace":"monitoring"},"spec":{"selector":{"istio":"ingressgateway"},"servers":[{"hosts":["*"],"port":{"name":"http","number":80,"protocol":"HTTP"}}]}}
  creationTimestamp: "2021-10-18T12:28:05Z"
  generation: 1
  name: my-gateway
  namespace: monitoring
  resourceVersion: "16618724"
  uid: 9b254a22-958c-4cc4-b426-4e7447c03b87
spec:
  selector:
    istio: ingressgateway
  servers:
  - hosts:
    - '*'
    port:
      name: http
      number: 80
      protocol: HTTP

---
apiVersion: v1
items:
- apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      alb.ingress.kubernetes.io/scheme: internal
      alb.ingress.kubernetes.io/target-type: ip
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"networking.k8s.io/v1beta1","kind":"Ingress","metadata":{"annotations":{"alb.ingress.kubernetes.io/scheme":"internal","alb.ingress.kubernetes.io/target-type":"ip","kubernetes.io/ingress.class":"alb"},"name":"ingress-alb","namespace":"istio-system"},"spec":{"rules":[{"http":{"paths":[{"backend":{"serviceName":"istio-ingressgateway","servicePort":80},"path":"/*"}]}}]}}
      kubernetes.io/ingress.class: alb
    finalizers:
    - ingress.k8s.aws/resources
    generation: 1
    name: ingress-alb
    namespace: istio-system
    resourceVersion: "4447931"
    uid: 74b31fba-0f03-41c6-a63f-6a10dee8780c
  spec:
    rules:
    - http:
        paths:
        - backend:
            service:
              name: istio-ingressgateway
              port:
                number: 80
          path: /*
          pathType: ImplementationSpecific
  status:
    loadBalancer:
      ingress:
      - hostname: internal-k8s-istiosys-ingressa-25a256ef4d-1368971909.us-east-1.elb.amazonaws.com
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

The ingress is associated with AWS ALB.

I want to access Loki from ALB URL like http(s)://my-alb-url/loki I hope I have provided the required details now. Let me know. Thanks.