Back
I am using spark-csv to load data into a DataFrame. I want to do a simple query and display the content:
val df = sqlContext.read.format("com.databricks.spark.csv").option("header", "true").load("my.csv")df.registerTempTable("tasks")results = sqlContext.sql("select col from tasks");results.show()
The col seems truncated:
scala> results.show();+--------------------+| col|+--------------------+|2015-11-16 07:15:...||2015-11-16 07:15:...||2015-11-16 07:15:...||2015-11-16 07:15:...||2015-11-16 07:15:...||2015-11-16 07:15:...||2015-11-16 07:15:...||2015-11-16 07:15:...||2015-11-16 07:15:...||2015-11-16 07:15:...||2015-11-16 07:15:...||2015-11-16 07:15:...||2015-11-16 07:15:...||2015-11-16 07:15:...||2015-11-16 07:15:...||2015-11-06 07:15:...||2015-11-16 07:15:...||2015-11-16 07:21:...||2015-11-16 07:21:...||2015-11-16 07:21:...|+--------------------+How do I show the full content of the column?
To prevent your result from truncating, add this to your code:
results.show(False)
or
results.show(false)
If you want to know more about Spark, then do check out this awesome video tutorial:
31k questions
32.4k answers
500 comments
693 users