I am running a set of dockerized applications on spark using YARN and I am wondering if it is feasible to deploy the cluster within Kubernetes or if Mesos is the better choice? I'd be very grateful for any architectural gotchas before deciding on one manager over the other.
You can use the Apache Livy build for Kubernetes: https://github.com/jahstreet/spark-on-kubernetes-helm . This setup fully replaces Yarn supported features for Spark and adds some additional capabilities atop.