I am using spark with Kubernetes and facing an issue while building a fat jar with "sbt assembly" command with libraryDependencies += "org.apache.spark" %% "spark-kubernetes" % "2.4.5" in my build.sbt. Everything is working fine if i downgrade the spark kubernetes version to "2.4.2" but not with "2.4.5".` And in another case when i rebuild the same code again using sbt assembly after deleted the target folder generated by previous sbt assembly command then it starts working fine with the same 2.4.5 spark Kubernetes jar. I just want to avoid this multiple times assembly jar building activity and i have also tried by adding spark tags library in my build.sbt file but still it is not working. So Please help me with this issue. I am using spark 2.4.4 and scala 2.11.12 and sbt 1.2.8 Kindly find the attached logs .

[error] import org.apache.spark.sql.{SQLContext, SparkSession}
[error]        ^
[error] /d02/scala/sparktest/app/controllers/MainController.scala:92:15: Symbol 'term  org.apache.spark.annotation' is missing from the classpath.
[error] This symbol is required by ' <none>'.
[error] Make sure that term annotation is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
[error] A full rebuild may help if 'SparkSession.class' was compiled against an incompatible version of org.apache.spark.
[error]   var spark = SparkSession.builder().appName("sparktest").config(conf).getOrCreate()
[error]               ^
[error] /d02/scala/sparktest/app/controllers/MainController.scala:94:24: Symbol 'term org.apache.spark.annotation' is missing from the classpath.
[error] This symbol is required by ' <none>'.
[error] Make sure that term annotation is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
[error] A full rebuild may help if 'SQLContext.class' was compiled against an incompatible version of org.apache.spark.
[error]   var sqlContext = new SQLContext(sc)
[error]                        ^
[error] three errors found
[error] (Compile / compileIncremental) Compilation failed.