Back

Explore Courses Blog Tutorials Interview Questions
0 votes
2 views
in Big Data Hadoop & Spark by (6.5k points)
What is shuffle used for?

1 Answer

0 votes
by (11.3k points)

The process due to which a mapper is able to transfer its current output to the reducer is called shuffling. A reducer then gets 1 or multiple keys sent to it and the entirety of the values associated with that key after the mappers have finished executing. The values that the mappers generate are called the intermediate key and these are sorted automatically by the key name. Basically, the merging and sorting of the mapper output are conducted.

If you want to learn Hadoop, understanding how the MapReduce operation works are very critical to moving ahead with practice or projects.  

If you want to grasp the concepts like shuffle in Hadoop MapReduce, you can join Hadoop Online Training.

Browse Categories

...