AZURE: Events are not flowing into Azure Application insight when job is scheduled using kubernetes CronJob

11/17/2020

We have written java application (Job) which reads some files from Azure blob and writes content to Azure Event hub. This is batch job runs at scheduled interval We have deployed and scheduled the application as Kubernetes CronJob. We are recording events with some details when files are moved from blob to Event hub, but those events are not reflecting in application insight. But we can see events when we run locally from IDE(Eclipse or intellij)

Below is deployment yaml file

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: job-name-1.0.0.0
spec:
  schedule: "*/5 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          nodeSelector:
            agentpool: agentpoolname
          containers:
            - name: job-name-0
              image: opsregistry.azurecr.io/job-name:v1.0.0.0
              imagePullPolicy: Always
              command: ["java", "-jar","job-name-1.0.0.0.jar","$(connection_string)"]
              env:
                - name: connection_string
                  valueFrom:
                    configMapKeyRef:
                      name: job-configmap
                      key: connectionString
              resources:
                limits:
                  cpu: "15"
                requests:
                  cpu: "0.5"
          restartPolicy: Never

below is the java code used to write event to azure application insigh

 TelemetryClient telemetry = new TelemetryClient();
 telemetry.getContext().setInstrumentationKey(instrumentationKey);
 telemetry.getContext().getCloud().setRole("CloudRoleName");
 telemetry.trackTrace("SOME INFORMATION ABOUT JOB", SeverityLevel.Information);

Please note we have deployed another Kafka stream job with same code but kind of deployment as kind: Deployment in yaml file and the events are flowing into Application insight without any issue we are facing for the kind: CronJob

Is there any changes we have to do for cron jobs?

Thanks in advance.

-- chandu ram
azure
azure-aks
azure-application-insights
java
kubernetes

1 Answer

11/17/2020

It is quite possible that job is ending before TelemetryClient could flush the pending telemetry from buffer. For continuous running job, it's not a problem (like in this case your Kafka stream job), but for scheduled job, execution ends leaving the pending telemetry. To fix this, add below in the code at the end of execution to ensure pending telemetries are written in the channel before execution ends.

// here 'telemetry' is the instance of TelemetryClient as per your shared code 
telemetry.flush(); 
-- krishg
Source: StackOverflow