Back

Explore Courses Blog Tutorials Interview Questions
0 votes
2 views
in Big Data Hadoop & Spark by (11.4k points)

I tried df.orderBy("col1").show(10) but it sorted in ascending order. df.sort("col1").show(10) also sorts in descending order. I looked on other communities and the answers I found were all outdated or referred to RDDs. I'd like to use the native dataframe in spark.

1 Answer

0 votes
by (32.3k points)
edited by

You can sort the column according to your need by: 

import org.apache.spark.sql.functions._

df.orderBy(asc("col1"))

Or

import org.apache.spark.sql.functions._

df.sort(desc("col1"))

You can also sort colums by importing sqlContext.implicits._

import sqlContext.implicits._

df.orderBy($"col1".desc)

Or

import sqlContext.implicits._

df.sort($"col1".desc)

If you want to know more about Spark, then do check out this awesome video tutorial:

You can learn in-depth about SQL statements, queries and become proficient in SQL queries by enrolling in an industry-recognized SQL online course.

Browse Categories

...