Proper configuration for Fabric Peers in Kubernetes

10/10/2017

I am attempting to configure Fabric to work inside of a Kubernetes cluster, and while I have everything standing up, I am having difficulting deploying chaincode (using composer-cli) to the network. It appears that the chaincode containers are not able to see the peer that created them.

2017-10-10 20:51:12.590 UTC [ccprovider] NewCCContext -> DEBU 437 NewCCCC (chain=lynnhurst,chaincode=lynnhurst-composer,version=0.13.2,txid=14cc34b63f20838b904116a03d39fba2a0eabf8ab7076f51982c746a17c667f1,syscc=false,proposal=0xc42195c0f0,canname=lynnhurst-composer:0.13.2
2017-10-10 20:51:12.605 UTC [chaincode] Launch -> DEBU 438 launchAndWaitForRegister fetched 2902002 bytes from file system
2017-10-10 20:51:12.605 UTC [chaincode] launchAndWaitForRegister -> DEBU 439 chaincode lynnhurst-composer:0.13.2 is being launched
2017-10-10 20:51:12.605 UTC [chaincode] getArgsAndEnv -> DEBU 43a Executable is chaincode
2017-10-10 20:51:12.605 UTC [chaincode] getArgsAndEnv -> DEBU 43b Args [chaincode -peer.address=peer-0.peer:7052]
2017-10-10 20:51:12.605 UTC [chaincode] launchAndWaitForRegister -> DEBU 43c start container: lynnhurst-composer:0.13.2(networkid:dev,peerid:peer-0.peer)
2017-10-10 20:51:12.605 UTC [chaincode] launchAndWaitForRegister -> DEBU 43d start container with args: chaincode -peer.address=peer-0.peer:7052
2017-10-10 20:51:12.605 UTC [chaincode] launchAndWaitForRegister -> DEBU 43e start container with env:
    CORE_CHAINCODE_ID_NAME=lynnhurst-composer:0.13.2
    CORE_PEER_TLS_ENABLED=true
    CORE_CHAINCODE_LOGGING_LEVEL=info
    CORE_CHAINCODE_LOGGING_SHIM=warning
    CORE_CHAINCODE_LOGGING_FORMAT=%{color}%{time:2006-01-02 15:04:05.000 MST} [%{module}] %{shortfunc} -> %{level:.4s} %{id:03x}%{color:reset} %{message}
2017-10-10 20:51:12.605 UTC [container] lockContainer -> DEBU 43f waiting for container(dev-peer-0.peer-lynnhurst-composer-0.13.2) lock
2017-10-10 20:51:12.605 UTC [container] lockContainer -> DEBU 440 got container (dev-peer-0.peer-lynnhurst-composer-0.13.2) lock
2017-10-10 20:51:12.606 UTC [dockercontroller] Start -> DEBU 441 Cleanup container dev-peer-0.peer-lynnhurst-composer-0.13.2
2017-10-10 20:51:12.607 UTC [dockercontroller] stopInternal -> DEBU 442 Stop container dev-peer-0.peer-lynnhurst-composer-0.13.2(No such container: dev-peer-0.peer-lynnhurst-composer-0.13.2)
2017-10-10 20:51:12.608 UTC [dockercontroller] stopInternal -> DEBU 443 Kill container dev-peer-0.peer-lynnhurst-composer-0.13.2 (No such container: dev-peer-0.peer-lynnhurst-composer-0.13.2)
2017-10-10 20:51:12.609 UTC [dockercontroller] stopInternal -> DEBU 444 Remove container dev-peer-0.peer-lynnhurst-composer-0.13.2 (No such container: dev-peer-0.peer-lynnhurst-composer-0.13.2)
2017-10-10 20:51:12.609 UTC [dockercontroller] Start -> DEBU 445 Start container dev-peer-0.peer-lynnhurst-composer-0.13.2
2017-10-10 20:51:12.609 UTC [dockercontroller] getDockerHostConfig -> DEBU 446 docker container hostconfig NetworkMode: bridge
2017-10-10 20:51:12.610 UTC [dockercontroller] createContainer -> DEBU 447 Create container: dev-peer-0.peer-lynnhurst-composer-0.13.2
2017-10-10 20:51:12.648 UTC [dockercontroller] createContainer -> DEBU 448 Created container: dev-peer-0.peer-lynnhurst-composer-0.13.2-49b014bf4f406b2b248c840a95c16be5a45845e67e4f7c1af9a5e7b7e69037bf
2017-10-10 20:51:12.836 UTC [dockercontroller] Start -> DEBU 449 Started container dev-peer-0.peer-lynnhurst-composer-0.13.2
2017-10-10 20:51:12.836 UTC [container] unlockContainer -> DEBU 44a container lock deleted(dev-peer-0.peer-lynnhurst-composer-0.13.2)
2017-10-10 20:51:12.955 UTC [dev-peer-0.peer-lynnhurst-composer-0.13.2] func2 -> INFO 44b 2017-10-10 20:51:12.955 UTC [Composer] Info -> INFO 001 Setting the Composer pool size to 8
2017-10-10 20:51:15.957 UTC [dev-peer-0.peer-lynnhurst-composer-0.13.2] func2 -> INFO 44c 2017-10-10 20:51:15.956 UTC [shim] userChaincodeStreamGetter -> ERRO 002 Error trying to connect to local peer: context deadline exceeded
2017-10-10 20:51:16.001 UTC [dockercontroller] func2 -> INFO 44d Container dev-peer-0.peer-lynnhurst-composer-0.13.2 has closed its IO channel
 2017-10-10 20:56:12.567 UTC [eventhub_producer] validateEventMessage -> DEBU 44e ValidateEventMessage starts for signed event 0xc421507680
2017-10-10 20:56:12.569 UTC [eventhub_producer] deRegisterHandler -> DEBU 44f deregistering event type: BLOCK
2017-10-10 20:56:12.578 UTC [eventhub_producer] Chat -> ERRO 450 error during Chat, stopping handler: rpc error: code = Canceled desc = context canceled
2017-10-10 20:56:12.836 UTC [chaincode] launchAndWaitForRegister -> DEBU 451 stopping due to error while launching Timeout expired while starting chaincode lynnhurst-composer:0.13.2(networkid:dev,peerid:peer-0.peer,tx:14cc34b63f20838b904116a03d39fba2a0eabf8ab7076f51982c746a17c667f1)
2017-10-10 20:56:12.836 UTC [container] lockContainer -> DEBU 452 waiting for container(dev-peer-0.peer-lynnhurst-composer-0.13.2) lock
2017-10-10 20:56:12.836 UTC [container] lockContainer -> DEBU 453 got container (dev-peer-0.peer-lynnhurst-composer-0.13.2) lock
2017-10-10 20:56:12.837 UTC [dockercontroller] stopInternal -> DEBU 454 Stop container dev-peer-0.peer-lynnhurst-composer-0.13.2(Container not running: dev-peer-0.peer-lynnhurst-composer-0.13.2)
2017-10-10 20:56:12.838 UTC [dockercontroller] stopInternal -> DEBU 455 Kill container dev-peer-0.peer-lynnhurst-composer-0.13.2 (API error (500): {"message":"Cannot kill container dev-peer-0.peer-lynnhurst-composer-0.13.2: Container db3e259d3c98fbb97a10a100724b71f861a7ee6b60317686329e1e6439be1ebd is not running"}
)
2017-10-10 20:56:12.843 UTC [dockercontroller] stopInternal -> DEBU 456 Removed container dev-peer-0.peer-lynnhurst-composer-0.13.2
2017-10-10 20:56:12.843 UTC [container] unlockContainer -> DEBU 457 container lock deleted(dev-peer-0.peer-lynnhurst-composer-0.13.2)
2017-10-10 20:56:12.844 UTC [chaincode] launchAndWaitForRegister -> DEBU 458 error on stop Error stopping container: context canceled(Timeout expired while starting chaincode lynnhurst-composer:0.13.2(networkid:dev,peerid:peer-0.peer,tx:14cc34b63f20838b904116a03d39fba2a0eabf8ab7076f51982c746a17c667f1))
2017-10-10 20:56:12.844 UTC [chaincode] func1 -> DEBU 459 chaincode lynnhurst-composer:0.13.2 launch seq completed
2017-10-10 20:56:12.844 UTC [chaincode] Launch -> ERRO 45a launchAndWaitForRegister failed Timeout expired while starting chaincode lynnhurst-composer:0.13.2(networkid:dev,peerid:peer-0.peer,tx:14cc34b63f20838b904116a03d39fba2a0eabf8ab7076f51982c746a17c667f1)
2017-10-10 20:56:12.844 UTC [endorser] callChaincode -> DEBU 45b Exit
2017-10-10 20:56:12.844 UTC [endorser] simulateProposal -> ERRO 45c failed to invoke chaincode name:"lscc"  on transaction 14cc34b63f20838b904116a03d39fba2a0eabf8ab7076f51982c746a17c667f1, error: Timeout expired while starting chaincode lynnhurst-composer:0.13.2(networkid:dev,peerid:peer-0.peer,tx:14cc34b63f20838b904116a03d39fba2a0eabf8ab7076f51982c746a17c667f1)
2017-10-10 20:56:12.844 UTC [endorser] simulateProposal -> DEBU 45d Exit
2017-10-10 20:56:12.844 UTC [lockbasedtxmgr] Done -> DEBU 45e Done with transaction simulation / query execution [f693a082-3404-4f01-98a1-bed5c3f6b24e]
2017-10-10 20:56:12.844 UTC [endorser] ProcessProposal -> DEBU 45f Exit

My configuration currently mounts the host's /var/run folder as /host/var/run and sets CORE_VM_DOCKER_ENDPOINT=unix:///host/var/run/docker.sock. I currently have CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=bridge, but I'm not sure if that's the correct setting (I've tried host) to allow these chaincode containers to connect to the peer. Any tips would be appreciated.

Fabric version 1.0.3, Composer version 0.13.2

-- jmcnevin
hyperledger-composer
hyperledger-fabric
kubernetes

2 Answers

10/11/2017

I was able to get this working, through a combination of configuration parameters:

CORE_PEER_ADDRESSAUTODETECT=true
CORE_PEER_TLS_SERVERHOSTOVERRIDE=%(hostname).peer

In addition, I needed to make sure NOT to set CORE_PEER_CHAINCODELISTENADDRESS, so that the peer's actual IP is passed to the chaincode containers.

-- jmcnevin
Source: StackOverflow

10/14/2017

As @christo4ferris indicated, Kubenetes is the basis for the IBM Blockchain developer experience.

All the scripts are available here:

https://github.com/IBM-Blockchain/ibm-container-service/tree/master/cs-offerings/kube-configs

-- Dan Selman
Source: StackOverflow