fetch more than 20 rows and display full value of column in spark-shell

Question

I am using CassandraSQLContext from spark-shell to query data from Cassandra. So, I want to know two things one how to fetch more than 20 rows using CassandraSQLContext and second how do Id display the full value of column. As you can see below by default it append dots in the string values.

Code :

val csc = new CassandraSQLContext(sc)
csc.setKeyspace("KeySpace")
val maxDF = csc.sql("SQL_QUERY" )
maxDF.show

Output:

+--------------------+--------------------+-----------------+--------------------+
|                  id|               Col2|              Col3|                Col4|
+--------------------+--------------------+-----------------+--------------------+
|8wzloRMrGpf8Q3bbk...|             Value1|                 X|                  K1|
|AxRfoHDjV1Fk18OqS...|             Value2|                 Y|                  K2|
|FpMVRlaHsEOcHyDgy...|             Value3|                 Z|                  K3|
|HERt8eFLRtKkiZndy...|             Value4|                 U|                  K4|
|nWOcbbbm8ZOjUSNfY...|             Value5|                 V|                  K5|

Answer 1

In order to print the whole value of a column, in scala, you have to set the argument truncate from the show method to false :

maxDf.show(false)

and if you wish to show more than 20 rows, do something like this:

// example showing 30 columns of 

// maxDf untruncated

maxDf.show(30, false) 

For pyspark, you'll need to specify the argument name :

maxDF.show(truncate = False)