Explore Courses Blog Tutorials Interview Questions
0 votes
in Big Data Hadoop & Spark by (6.5k points)
I'm confused if a pig query is only used to operate upon Hadoop clusters or can we use it for local files as well?

1 Answer

0 votes
by (11.3k points)

Yes, Apache Pig has two modes:  Local and Mapreduce 

If you'd like to run Pig Queries and operations on data present on your local filesystem, you can go ahead and use the 'pig -x local' command to achieve the same. Here, instead of providing HDFS directories for file locations, you can use the directories present on your local filesystem. 

And similarly, if you'd like to operate on a Hadoop Cluster, you can use 'pig -x mapreduce'. But keep in my mind to use Hadoop Filesystem directories for operation execution.

Apache Pig is widely used in Big Data Analytics in Hadoop, you should definitely get some more hands-on in this subject and complete a hadoop certification to show something for it. 

Browse Categories