Is using function in transformation causing Not Serializable exceptions?
Is using function in transformation causing Not Serializable exceptions? I have a Breeze DenseMatrix , i find mean per row and mean of squares per row and put them in another DenseMatrix , one per column. But i get Task Not Serializable exception. I know that sc is not Serializable but i think that the exception is because i call functions in a transformation in Safe Zones. Breeze DenseMatrix mean mean DenseMatrix Task Not Serializable sc Serializable Am i right? And how could be a possible way to be done without any functions? Any help would be great! Code: object MotitorDetection { case class MonDetect() extends Serializable { var sc: SparkContext = _ var machines: Int=0 var counters: Int=0 var GlobalVec= BDM.zeros[Double](counters, 2) def findMean(a: BDM[Double]): BDV[Double] = { var c = mean(a(*, ::)) c} def toMatrix(x: BDV[Double], y: BDV[Double], C: Int): BDM[Double]={ val m = BDM.zeros[Double](C,2) m(::, 0) := x m(::, 1) := y m} def SafeZones(stream: DStream[(In...
