K
Q

What does "Expected one of #, =>" in Logstash mean?

4/29/2020

we wrote a config file for a Logstash 7.6.2 pipeline we called kubernetes-internal.

We checked the syntax by bin/logstash --config.test_and_exit -f ../kubernetes-internal/02-filter.conf and it appears we are missing something in the syntax:

[FATAL] 2020-04-29 10:41:42.006 [LogStash::Runner] runner - The given configuration is invalid. Reason: Expected one of #, => at line 152, column 8 (byte 5565) after filter {

Looking at that part of code, we have:

...
  else {
    if [kubernetes][labels][version] != "v2" {
      date {
        match => [ "syslog_timestamp", "ISO8601" ]
        remove_field => [ "syslog_timestamp" ]
           }
                                             }
    prune {
      whitelist_names => ["^message_csv
quot;,"^host
quot;,"^beat","^source
quot;,"^type
quot;,"^offset
quot;,"@timestamp","kubernetes"]
          }
    mutate {
      add_tag => [ "haproxy-logs" ]
      remove_tag => [ "_csvparsefailure" ]
           }
       }
  else {
    if "ambassador" in [kubernetes][labels][service] { #Line152
      grok {
        match => { "message" => '^%{TIMESTAMP_ISO8601:time} %{IP:clientip}:%{NUMBER:port} %{WORD:verb} %{NOTSPACE:request} HTTP/%{NUMBER:httpversion} %{NUMBER:http_status} %{NOTSPACE:response_flag} %{NUMBER:bytes_received} %{NUMBER:bytes_sent} %{NUMBER:request_time} \'%{NOTSPACE:X-Forwarded-for}\' \'%{NOTSPACE:X-OURDOMAIN-Api}\' \'%{DATA:agent}\' \'%{NOTSPACE:UUID}\' \'%{NOTSPACE:X-Forwarded-Client-Cert}\' \'%{NOTSPACE:authority}\' \'%{NOTSPACE:upstream_host}\'
#x27; }
           }
      mutate {
      convert => {
        "http_status"    => "integer"
        "bytes_sent"     => "integer"
        "bytes_received" => "integer"
        "request_time"   => "integer"
                 }
...

What are we missing so far ?

-- Luigi Sambolino
kubernetes
logstash
logstash-grok
yaml

Similar Questions

coredns is running but not ready after conjure-up k8s cdk
Need help running two OS containers in a single pod on kubernetes
kubernetes -I changed the deployment name ,then redeploying to the environment,how to cleanup the old deployment and pods with old name?
Kubernetes authorization via local groups

1 Answer

4/29/2020

Difficult to tell, because the brackets are that messy and it's truncated.

I mean, it clearly reads after filter { ...but there is no filter {.

Ever tried to flip the condition? Or tried to add brackets, as the in operator expects?

if [kubernetes][labels][service] == "ambassador" {

if [kubernetes][labels][service] in ("ambassador") {

That condition might even be wrong altogether, as this might be the component label; eg. app.kubernetes.io/component: ambassador-service. But without knowing how the labels were defined, one can only assume.

https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/

Just try to query them directly, in order to find the condition:

kubectl get pods -l 'service in (ambassador)'
-- Martin Zeitler
Source: StackOverflow