Intellipaat Back

Explore Courses Blog Tutorials Interview Questions
0 votes
2 views
in Big Data Hadoop & Spark by (6.5k points)
Can anyone tell me why do we need Spark?

1 Answer

0 votes
by (11.3k points)

We use Apache Spark when we have to analyze large amounts of data in real-time, extremely fast. Spark provides a processing engine that is about 100 times faster than Hadoop's MapReduce. This is achieved due to Spark's in-memory processing methodology where the data in question is loaded into the physical memory for instant access.

Using traditional Data Analytics tools such as Python and R, without the Spark API, to analyze Big Data would result in inefficient, slow, and error-prone results.

If you are looking for an online course to learn Spark, I recommend this Apache Spark Training program by Intellipaat.

31k questions

32.8k answers

501 comments

693 users

Browse Categories

...