0 votes
1 view
in Big Data Hadoop & Spark by (11.5k points)

I tried df.orderBy("col1").show(10) but it sorted in ascending order. df.sort("col1").show(10) also sorts in descending order. I looked on other communities and the answers I found were all outdated or referred to RDDs. I'd like to use the native dataframe in spark.

1 Answer

0 votes
by (32.5k points)
edited by

You can sort the column according to your need by: 

import org.apache.spark.sql.functions._

df.orderBy(asc("col1"))

Or

import org.apache.spark.sql.functions._

df.sort(desc("col1"))

You can also sort colums by importing sqlContext.implicits._

import sqlContext.implicits._

df.orderBy($"col1".desc)

Or

import sqlContext.implicits._

df.sort($"col1".desc)

If you want to know more about Spark, then do check out this awesome video tutorial:

Welcome to Intellipaat Community. Get your technical queries answered by top developers !


Categories

...