I am trying to convert a column which is in String format to Date format using the to_date function but its returning Null values.
df.createOrReplaceTempView("incidents")
spark.sql("select Date from incidents").show()
+----------+
| Date|
+----------+
|08/26/2019|
|08/26/2019|
|08/26/2019|
|06/14/2019|
spark.sql("select to_date(Date) from incidents").show()
+---------------------------+
|to_date(CAST(Date AS DATE))|
+---------------------------+
| null|
| null|
| null|
| null|
The Date column is in String format:
|-- Date: string (nullable = true)