I'm trying to run Cassandra on kubernetes.
One thing I understood is Cassandra is trying to access kubernetes api server on port 443 (Secure Connection) but I'm running api server on non secure connection port 8080.
Also there is Has no permission to create /cassandra_data/data directory error
Error (Cassandra pod log):
INFO 13:04:02 Getting endpoints from https://10.100.0.1:443/api/v1/namespaces/default/endpoints/cassandra
WARN 13:04:11 Request to kubernetes apiserver failed
java.io.IOException: Server returned HTTP response code: 401 for URL: https://10.100.0.1:443/api/v1/namespaces/default/endpoints/cassandra
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1628) ~[na:1.7.0_95]
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) ~[na:1.7.0_95]
at io.k8s.cassandra.KubernetesSeedProvider.getSeeds(KubernetesSeedProvider.java:143) ~[kubernetes-cassandra.jar:na]
at org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:659) [apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:136) [apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:168) [apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:564) [apache-cassandra-2.1.13.jar:2.1.13]
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:653) [apache-cassandra-2.1.13.jar:2.1.13]
INFO 13:04:11 JVM vendor/version: OpenJDK 64-Bit Server VM/1.7.0_95
WARN 13:04:11 OpenJDK is not recommended. Please upgrade to the newest Oracle Java release
INFO 13:04:11 Heap size: 526385152/526385152
INFO 13:04:11 Code Cache Non-heap memory: init = 2555904(2496K) used = 795136(776K) committed = 2555904(2496K) max = 50331648(49152K)
INFO 13:04:11 Eden Space Heap memory: init = 83886080(81920K) used = 79390336(77529K) committed = 83886080(81920K) max = 83886080(81920K)
INFO 13:04:11 Survivor Space Heap memory: init = 10485760(10240K) used = 0(0K) committed = 10485760(10240K) max = 10485760(10240K)
INFO 13:04:11 CMS Old Gen Heap memory: init = 432013312(421888K) used = 0(0K) committed = 432013312(421888K) max = 432013312(421888K)
INFO 13:04:11 CMS Perm Gen Non-heap memory: init = 21757952(21248K) used = 19488984(19032K) committed = 21757952(21248K) max = 174063616(169984K)
INFO 13:04:11 Classpath: /kubernetes-cassandra.jar:/etc/cassandra:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/guava-16.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jline-1.0.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/logback-classic-1.1.2.jar:/usr/share/cassandra/lib/logback-core-1.1.2.jar:/usr/share/cassandra/lib/lz4-1.2.0.jar:/usr/share/cassandra/lib/metrics-core-2.2.0.jar:/usr/share/cassandra/lib/netty-all-4.0.23.Final.jar:/usr/share/cassandra/lib/reporter-config-2.1.0.jar:/usr/share/cassandra/lib/slf4j-api-1.7.2.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.0.5.2.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/super-csv-2.1.0.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-2.1.13.jar:/usr/share/cassandra/apache-cassandra-thrift-2.1.13.jar:/usr/share/cassandra/apache-cassandra.jar:/usr/share/cassandra/cassandra-driver-core-2.0.9.2.jar:/usr/share/cassandra/netty-3.9.0.Final.jar:/usr/share/cassandra/stress.jar::/usr/share/cassandra/lib/jamm-0.3.0.jar
INFO 13:04:11 JVM Arguments: [-ea, -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar, -XX:+CMSClassUnloadingEnabled, -XX:+UseThreadPriorities, -XX:ThreadPriorityPolicy=42, -Xms512M, -Xmx512M, -Xmn100M, -XX:+HeapDumpOnOutOfMemoryError, -Xss256k, -XX:StringTableSize=1000003, -XX:+UseParNewGC, -XX:+UseConcMarkSweepGC, -XX:+CMSParallelRemarkEnabled, -XX:SurvivorRatio=8, -XX:MaxTenuringThreshold=1, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+UseTLAB, -XX:CompileCommandFile=/etc/cassandra/hotspot_compiler, -XX:CMSWaitDuration=10000, -XX:+CMSParallelInitialMarkEnabled, -XX:+CMSEdenChunksRecordAlways, -XX:CMSWaitDuration=10000, -XX:+UseCondCardMark, -Djava.net.preferIPv4Stack=true, -Dcassandra.jmx.local.port=7199, -XX:+DisableExplicitGC, -Dlogback.configurationFile=logback.xml, -Dcassandra.logdir=/var/log/cassandra, -Dcassandra.storagedir=, -Dcassandra-foreground=yes]
WARN 13:04:13 Unable to lock JVM memory (ENOMEM). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
WARN 13:04:13 JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
ERROR 13:04:20 Directory /cassandra_data/data doesn't exist
ERROR 13:04:20 Has no permission to create /cassandra_data/data directory
ReplicationController:
apiVersion: v1
kind: ReplicationController
metadata:
labels:
app: cassandra
name: cassandra
spec:
replicas: 2
selector:
app: cassandra
template:
metadata:
labels:
app: cassandra
spec:
containers:
- command:
- /run.sh
resources:
limits:
cpu: 0.1
env:
- name: MAX_HEAP_SIZE
value: 512M
- name: HEAP_NEWSIZE
value: 100M
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: gcr.io/google-samples/cassandra:v8
name: cassandra
ports:
- containerPort: 9042
name: cql
- containerPort: 9160
name: thrift
volumeMounts:
- mountPath: /cassandra_data
name: data
volumes:
- name: data
emptyDir: {}You changed cassandra default data directory so first create data directory . Give permission to write and read to created new data directory or run cassandra as root . if you set credential for cassandra then provide user name and password also .