How to specify the number of reducers to be used by a hadoop operation?

Question

The default seems to be having only 1 reducer when it executes.

Answer 1

You can add this:

Dmapred.reduce.tasks=x  (where x is the number of reducers you'd like to use)

as an option to your MapReduce execution code to set the number of reducers explicitly. For example, this would how to execution command would look like:

hadoop jar wordcount.jar WordCount -Dmapred.reduce.tasks=2 wordcountiput wordcountoput

Here, the number of reducers is 2.

Keep in mind that the significance of using a specified number of reducers is only practical in a multiple node hadoop cluster. To understand why using a specified number of reducers can be advantageous in a real-world scenario, you should take up a big data course.