I have a database that's running using local storage in Kubernetes. Whenever I start up the Pod with the database I would like to run a Job that can look at our backups and backfill any data that we have that isn't on the local disk.
I was looking at the PostStart lifecycle hook, but that just lets me run a command from the main container, which would be a very hacky way to submit this Job. Are there any better approaches to this other than writing my own controller?
You can use Helm for deploy your database and manage it's lifecycle.
Helm has many type of hooks and you can use any container for a hook.
I think in your case you can use post-upgrade + post-install for do everything you want. Just check a documentation.
You can use an init container that mounts in the same volumes as the main container, populates it with any missing data and then exits so the main container can start.
If you need the job container to be running at the same time as your main container, you can instead just put the container described above as a second container in the Pod.
If you need an actual kubernetes Job to be created then, as you say, I think the only options would be to create a custom controller or to run an apiserver client such as kubectl as a sidecar container, but you could use the ServiceAccount token that's automatically mounted into your Pod to authenticate with the apiserver and then just apply the necessary RBAC rules to the ServiceAccount to create a Job. You'd have to use some sort of shared data volume to mount the same data into both the Pods spawned by the Job and the main Pod in order to share the data (there are a few other options that are possible also).