Spring boot stackdriver logging is textPayload and not jsonPayload

4/26/2019

I have a log filter that logs out essential request information for debugging and log analytics. But as you can see, the text payload is really hard to read.

I don't want to have to copy + paste this text payload into a text editor every single time. Is there a way to make stack driver print this in a collapsable json instead?

image of text payload in stackdriver

More info: - GKE pod

@Component
class LogFilter : WebFilter {

    private val logger = LoggerFactory.getLogger(LogFilter::class.java)

    override fun filter(exchange: ServerWebExchange, chain: WebFilterChain): Mono<Void> {
        return chain
                .filter(exchange)
                .doAfterTerminate {
                    val request = exchange.request
                    val path = request.uri.path
                    val routesToExclude = listOf("actuator")
                    var isExcludedRoute = false

                    for (r in routesToExclude) { if (path.contains(r)) { isExcludedRoute = true; break; } }

                    if (!isExcludedRoute) {
                        val startTime = System.currentTimeMillis()
                        val statusCode = exchange.response.statusCode?.value()
                        val requestTime = System.currentTimeMillis() - startTime
                        val msg = "Served $path as $statusCode in $requestTime msec"
                        val requestPrintMap = mutableMapOf<Any, Any>()
                        requestPrintMap["method"] = if (request.method != null) {
                            request.method.toString()
                        } else "UNKNOWN"
                        requestPrintMap["path"] = path.toString()
                        requestPrintMap["query_params"] = request.queryParams
                        requestPrintMap["headers"] = request.headers

                        requestPrintMap["status_code"] = statusCode.toString()
                        requestPrintMap["request_time"] = requestTime
                        requestPrintMap["msg"] = msg

                        logger.info(JSONObject(requestPrintMap).toString())
                    }
                }
    }

}
-- Clement
google-kubernetes-engine
logging
spring-boot
stackdriver

2 Answers

4/29/2019

What you will need to do is customize Fluentd in GKE. Pretty much it's creating a Fluend daemonset for logging instead of the default logging method.

Once that is done, you can setup structured logging to send jsonPayload logs to Stackdriver Logging.

-- vaqars
Source: StackOverflow

4/29/2019

The default Stackdriver logging agent configuration for Kubernetes will detect single-line JSON and convert it to jsonPayload. You can configure Spring to log as single-line JSON (e.g., via JsonLayout1) and let the logging agent pick up the JSON object (see https://cloud.google.com/logging/docs/agent/configuration#process-payload).

1Some of the JSON field names are different (e.g., JsonLayout uses "level" for the log level, while the Stackdriver logging agent recognizes "severity"), so you may have to override addCustomDataToJsonMap to fully control the resulting log entries.

See also GKE & Stackdriver: Java logback logging format?

-- Igor Peshansky
Source: StackOverflow