Is it possible to create a docker container in a Google Container Engine cluster, using a dockerfile, which builds an image on-the-fly and deploy in cluster, rather than creating an image first and uploading it to a Google Container Registry, and then using it from there?
I feel like that is cumbersome, and there should be a way to create containers in cluster directly using a dockerfile.
It is not possible to do this in Google Container Engine. Google Container Engine is designed to help orchestrate container deployment and does not itself provide a source -> deployment workflow.
You may want to look at Google App Engine or Openshift 3 (which is built on Kubernetes) as a more fully featured platform-as-a-service offering.
You can also build this type of tooling on top of a Google Container Engine cluster yourself as all of the building blocks are available.
One service to take a look at when constructing a workflow is Google Container Builder, which can simplify the process of building a container from source and pushing it to Google Container Registry.
It is currently a fairly low level service, but offers some advantages for environments where it may be impractical to run docker build locally.