Intellipaat Back

Explore Courses Blog Tutorials Interview Questions
0 votes
in Big Data Hadoop & Spark by (11.4k points)

I would like to access to the min and max of a specific column from my dataframe but I don't have the header of the column, just its number, so I should I do using scala?

maybe something like this :

val q = nextInt(ncol) //we pick a random value for a column number
col = df(q)

val minimum = col.min()

Sorry if this sounds like a silly question but I couldn't find any info on SO about this question

1 Answer

0 votes
by (32.3k points)

I would suggest you to get the column name from the metadata. Do something like this:

val selectedColumnName = df.columns(q) //pull the (q + 1)th column from the columns array

df.agg(min(selectedColumnName), max(selectedColumnName))

Browse Categories