http4 not sending exchange data further for kubernetes API

9/14/2019

I am using kubernetes rest api API to watch configmap and calling it with http4 component, but http4 is not forwarding exchange further in route, from logs i observed that http4 keep getting updates from openshift on first open connection as its not closing connection so dont forward exchange further, how i can get this data down in route

from("timer://foo?fixedRate=true&period=60000")
.setHeader("Authorization", simple("Bearer xxx"))
.setHeader("Accept", simple("application/json"))
.setHeader("CamelHttpMethod", constant("GET"))
.to("https4://localhost:8443/api/v1/watch/namespaces/my-qa/configmaps/my-config-map?httpClientConfigurer=#trustCert")
.to("log:my?showAll=true&multiline=true")
.to("mock:result");

Trace:

 DEBUG http-outgoing-0 >> GET /api/v1/watch/namespaces/my-qa/configmaps/my-config-map HTTP/1.1
 DEBUG http-outgoing-0 >> Accept: application/json
 DEBUG http-outgoing-0 >> Authorization: Bearer xxx
 DEBUG http-outgoing-0 >> breadcrumbId: ID-NB-118-1568476509695-0-1
 DEBUG http-outgoing-0 >> firedTime: Sat Sep 14 17:55:14 CEST 2019
 DEBUG http-outgoing-0 >> Host: localhost:8443
 DEBUG http-outgoing-0 >> Connection: Keep-Alive
 DEBUG http-outgoing-0 >> User-Agent: Apache-HttpClient/4.5.6 (Java/1.8.0_162)
 DEBUG http-outgoing-0 >> Accept-Encoding: gzip,deflate
 DEBUG http-outgoing-0 >> "GET /api/v1/watch/namespaces/my-qa/configmaps/my-config-map HTTP/1.1[\r][\n]"
 DEBUG http-outgoing-0 >> "Accept: application/json[\r][\n]"
 DEBUG http-outgoing-0 >> "Authorization: Bearer xxx[\r][\n]"
 DEBUG http-outgoing-0 >> "breadcrumbId: ID-NB-118-1568476509695-0-1[\r][\n]"
 DEBUG http-outgoing-0 >> "firedTime: Sat Sep 14 17:55:14 CEST 2019[\r][\n]"
 DEBUG http-outgoing-0 >> "Host: m.devokd.younicos.local:8443[\r][\n]"
 DEBUG http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
 DEBUG http-outgoing-0 >> "User-Agent: Apache-HttpClient/4.5.6 (Java/1.8.0_162)[\r][\n]"
 DEBUG http-outgoing-0 >> "Accept-Encoding: gzip,deflate[\r][\n]"
 DEBUG http-outgoing-0 >> "[\r][\n]"
 DEBUG http-outgoing-0 << "HTTP/1.1 200 OK[\r][\n]"
 DEBUG http-outgoing-0 << "Cache-Control: no-store[\r][\n]"
 DEBUG http-outgoing-0 << "Content-Type: application/json[\r][\n]"
 DEBUG http-outgoing-0 << "Date: Sat, 14 Sep 2019 15:55:14 GMT[\r][\n]"
 DEBUG http-outgoing-0 << "Transfer-Encoding: chunked[\r][\n]"
 DEBUG http-outgoing-0 << "[\r][\n]"
 DEBUG http-outgoing-0 << HTTP/1.1 200 OK
 DEBUG http-outgoing-0 << Cache-Control: no-store
 DEBUG http-outgoing-0 << Content-Type: application/json
 DEBUG http-outgoing-0 << Date: Sat, 14 Sep 2019 15:55:14 GMT
 DEBUG http-outgoing-0 << Transfer-Encoding: chunked
 DEBUG Connection can be kept alive indefinitely
 DEBUG Http responseCode: 200
 TRACE Copying InputStream: org.apache.http.conn.EofSensorInputStream@3ab22477 -> OutputStream: CachedOutputStream[size: 0] with buffer: 4096 and flush on each write false
 DEBUG http-outgoing-0 << "158[\r][\n]"
 DEBUG http-outgoing-0 << "{"type":"ADDED","object":{"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"my-config-map","namespace":"my-qa","selfLink":"/api/v1/namespaces/my-qa/configmaps/my-config-map","uid":"a8130369-d703-11e9-b37c-d2c8065c1507","resourceVersion":"33040706","creationTimestamp":"2019-09-14T15:23:51Z"},"data":{"test":"added first modification"}}}[\n]"
 DEBUG http-outgoing-0 << "[\r][\n]"

Below data received after 2 minutes on same open connection

DEBUG http-outgoing-0 << "156[\r][\n]"
DEBUG http-outgoing-0 << "{"type":"MODIFIED","object":{"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"my-config-map","namespace":"yq-qa","selfLink":"/api/v1/namespaces/my-qa/configmaps/my-config-map","uid":"a8130369-d703-11e9-b37c-d2c8065c1507","resourceVersion":"33044977","creationTimestamp":"2019-09-14T15:23:51Z"},"data":{"test":"second modification"}}}[\n]"
DEBUG http-outgoing-0 << "[\r][\n]"

Update1: it look like http4 dont support reading of chunked data until it receive all data, but in my case i want to receive/process data chunk by chunk. similar behaviour working with curl, Below Added and Modified are two chunks second got print after two minutes, in similar way i want camel route should process chunk rather than whole message

# curl -v -k -H "Authorization: Bearer xxx" -H 'Accept: application/json' -H "Accept-Encoding: gzip" https://localhost:8443/api/v1/watch/namespaces/my-qa/configmaps/my-config-map
* About to connect() to localhost port 8443 (#0)
*   Trying x.x.x.x...
* Connected to localhost (x.x.x.x) port 8443 (#0)

> GET /api/v1/watch/namespaces/my-qa/configmaps/my-config-map HTTP/1.1
> User-Agent: curl/7.29.0
> Host: localhost:8443
> Authorization: Bearer xxx
> Accept: application/json
> Accept-Encoding: gzip
>
< HTTP/1.1 200 OK
< Cache-Control: no-store
< Content-Type: application/json
< Date: Sun, 15 Sep 2019 11:48:14 GMT
< Transfer-Encoding: chunked
<
{"type":"ADDED","object":{"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"my-config-map","namespace":"my-qa","selfLink":"/api/v1/namespaces/yq-qa/configmaps/my-config-map","uid":"a8130369-d703-11e9-b37c-d2c8065c1507","resourceVersion":"33217315","creationTimestamp":"2019-09-14T15:23:51Z"},"data":{"test":"second modification 20tefdsfsfsc"}}}

{"type":"MODIFIED","object":{"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"my-config-map","namespace":"my-qa","selfLink":"/api/v1/namespaces/my-qa/configmaps/my-config-map","uid":"a8130369-d703-11e9-b37c-d2c8065c1507","resourceVersion":"33217608","creationTimestamp":"2019-09-14T15:23:51Z"},"data":{"test":"second modification 20tefdsfsfsc gzip"}}}
-- ImranRazaKhan
apache-camel
chunked-encoding
http
kubernetes
openshift

0 Answers