What is 'In-Memory Computation' in Spark?

Question

Could someone tell me what is ‘In-Memory Computation’ in Spark?

Answer 1

Spark is considered to be 10x times quicker than traditional HDFS or MapReduce techniques, to process Big Data. How can Spark do it? It’s because of ‘In-Memory Computaion’ that Spark uses to achieve these performances. In Spark, the Big Data is not stored in the data drive which makes the system slow, for pulling and processing the data, rather Spark takes another route by keeping data in Random Access Memory (RAM), and that helps Spark to detect a pattern, analyze large data quickly, and also reduce the dependency on memory, eventually cutting down costs for memory requirement. 

Kickstart your career by enrolling in the Apache Spark course in Singapore.

Wish to learn Spark and get certified? Enroll today in an industry-grade Spark course from Intellipaat, and also check out the following YouTube video on Spark Scala Full Course to get a better understanding.