Explore Courses Blog Tutorials Interview Questions
0 votes
in Big Data Hadoop & Spark by (6.5k points)
Don't they both allocate jobs for Hadoop operations?

1 Answer

0 votes
by (11.3k points)

It isn't as simple as just 'allocating' jobs. Both of these applications are vastly different from each other down to the basic purpose. To put it simply, 

YARN allocates resources to any hadoop job that has been submitted and accepted. These resources are present in the cluster itself, YARN just manages where to allocate these resources and when to do it so there is no clash or slowdown of execution due to two jobs infinitely waiting for resources to be freed up (Deadlock) or slowing down of the entire execution architecture because of inefficient resource management. YARN is simply a resource management and resource scheduling tool.

Now, the definition of Zookeeper will seem somewhat similar but understand the difference. Zookeeper acts as a job scheduling agent on cluster level basis, it is used to achieve synchronicity in a multi-node hadoop distributed architecture. It is used by YARN as well to manage its resource allocation properties. 

You should take a big data course to understand both Zookeeper and YARN and to understand the differences between a training environment and a real-world environment of big data analytics and how crucial resource management and job scheduling is in such large systems and clusters that operate on enormous amounts of data. 

Browse Categories