Mgiyuk

Is using function in transformation causing Not Serializable exceptions? I have a Breeze DenseMatrix , i find mean per row and mean of squares per row and put them in another DenseMatrix , one per column. But i get Task Not Serializable exception. I know that sc is not Serializable but i think that the exception is because i call functions in a transformation in Safe Zones. Breeze DenseMatrix mean mean DenseMatrix Task Not Serializable sc Serializable Am i right? And how could be a possible way to be done without any functions? Any help would be great! Code: object MotitorDetection { case class MonDetect() extends Serializable { var sc: SparkContext = _ var machines: Int=0 var counters: Int=0 var GlobalVec= BDM.zeros[Double](counters, 2) def findMean(a: BDM[Double]): BDV[Double] = { var c = mean(a(*, ::)) c} def toMatrix(x: BDV[Double], y: BDV[Double], C: Int): BDM[Double]={ val m = BDM.zeros[Double](C,2) m(::, 0) := x m(::, 1) := y m} def SafeZones(stream: DStream[(In...

Conjugate transpose of self using numpy syntax I am trying to translate this MATLAB code into Python. The following is the code: Y=C*Up(:,1:p-1)'*Y; And this is my translation thus far: Y = C * Up[:, 1:p-1] * Y I am having trouble with the syntax for the conjugate transpose of self that is used in the MATLAb code. I am not certain that my first idea: Y = C * Up[:, 1:p-1].getH() * Y would be correct. Does anyone have any ideas? Can you provide sample inputs for the matrix (matrices)? – rahlf23 Jun 29 at 17:47 1:p-1 - remember, python indexing is 0 based, MATLAB 1 based. – hpaulj Jun 29 at 18:32 1:p-1 Is Up (possibly) complex? If not then plain transpose is enou...

Intelligent method for particular traversing of a matrix I have to traverse an n x n matrix in java (so indices are 0,...,n-1), to assign values to the single elements. I must start from the bottom right and arrive to the top left. The particularity is that I do not have to consider the matrix[n-1][n-1] element, that has been initialised before. The adjacent values depend on each other for initialazing and it must be initialized first. n x n matrix[n-1][n-1] One way could be inserting an if in the for cycle if for for (i = n-1; i >= 0; i--) for (j = n-1; j >= 0; j--) if (i == n - 1 && j == n - 1) //initialize particular value else //initialize others but it seems to me a bit inefficient. Another way could be to initialize the value matrix[n-1][n-1] outside the cycle, then doing 3 for cycles (one for the bottom line, one for the rightest column, one for the other elements). But it seems a bit inelegant. matrix[n-1][n-1] for So I'm searching,...

Efficient way to work with 2D Arrays (multiple formats) I have an efficiency/performance question in Python: I am planning to store data in some kind of matrix format in a "storage" For example: elementID - nodeID - elementType - Value My current process is to loop through the raw data information and check if there is already an entry with "elementID, nodeID, elementType" in the storage. Now, when considering a second case, also a second value is occuring. What I want to do is now to add the second value to the associated case in the storage. So: elementID - nodeID - elementType - Value1 - Value2 Currently I am working with 2D arrays, so [[a,b,c],[b,c,d],....]. Since this is implying "for loops", the time required to check if the entry is already in the storage is increasing tremendously. Also to add the additional column to the storage is requiring the search for the corresponding entry. In some cases, it is also possible that there are many value column...

Sparse.model.matrix error message I'm trying to create a sparse-matrix and get this error message: Error: fnames == names(mf) are not all TRUE I think it has something to do with the column names of my data, maybe you can help. Here are the column names: Error: fnames == names(mf) are not all TRUE colnames(trainDataShrinkage) <-"Bildungsgrad2_Lower_secondary_education" ,"Bildungsgrad3_Upper_secondary_education" ,"Bildungsgrad4_Post-secondary_non-tertiary_education" ,"Bildungsgrad5_Short-cycle_tertiary_education" ,"Bildungsgrad6_Bachelors_or_equivalent_level" ,"Bildungsgrad7_Masters_or_equivalent_level" ,"Bildungsgrad8_Doctoral_or_equivalent_level" ,"Familienstand2_Verheiratet,_getrenntlebend" ,"Familienstand3_Ledig" ,"F...