Related to my other question, but distinct:
If I save an RDD to HDFS, how can I tell spark to compress the output with gzip? In Hadoop, it is possible to set
mapred.output.compress = true
and choose the compression algorithm with
mapred.output.compression.codec = <<classname of compression codec>>
How would I do this in spark? Will this work as well?
edit: using spark-0.7.2