Explore Courses Blog Tutorials Interview Questions
0 votes
1 view
in Big Data Hadoop & Spark by (44.2k points)

Could someone tell me what is ‘In-Memory Computation’ in Spark?

1 Answer

0 votes
by (98.6k points)

Spark is considered to be 10x times quicker than traditional HDFS or MapReduce techniques, to process Big Data. How can Spark do it? It’s because of ‘In-Memory Computaion’ that Spark uses to achieve these performances. In Spark, the Big Data is not stored in the data drive which makes the system slow, for pulling and processing the data, rather Spark takes another route by keeping data in Random Access Memory (RAM), and that helps Spark to detect a pattern, analyze large data quickly, and also reduce the dependency on memory, eventually cutting down costs for memory requirement. 

Wish to learn Spark and get certified? Enroll today in an industry-grade Spark course from Intellipaat, and also check out the following YouTube video on Spark Scala Full Course to get a better understanding. 

Browse Categories