What is rank in ALS machine Learning Algorithm in Apache Spark Mllib

Question

I wanted to try an example of ALS machine learning algorithm. And my code works fine, However I do not understand parameter rank used in algorithm.

I have following code in java

    // Build the recommendation model using ALS

    int rank = 10;

    int numIterations = 10;

    MatrixFactorizationModel model = ALS.train(JavaRDD.toRDD(ratings),

            rank, numIterations, 0.01);

I have read some where that it is the number of latent factors in the model.

Suppose I have a dataset of (user,product,rating) that has 100 rows. What value should be of rank (latent factors).

Answer 1

Here the rank refers to the presumed latent or hidden factors. It can help to measure, how much different people liked movies and tried to cross-predict them. You might have three fields: person, movie, number of stars. All the movie ratings could be perfectly predicted by just 3 hidden factors, sex, age, and income. Here, the "rank" of your run should be 3.

We are mostly unaware of the underlying factors, that drive your data. The more you use, the better the results up to a point, but the more memory and computation time you will need.

Hope this answer helps you!

If you want to learn Spark then go through this Apache Spark tutorial:

Join Intellipaat's Machine learning course, to learn more about ML.