Intellipaat Back

Explore Courses Blog Tutorials Interview Questions
0 votes
2 views
in BI by (17.6k points)

I have set up my spark cluster and I am successful in connecting Tableau through Spark SQL connector.

I created my tables from spark-shell and saved dataframes from MySQL using (saveAsTable).

How can I access the tables that I have saved from Tableau? Do I need to give a path of the warehouse directory when starting the spark thrift server? If yes, how it can be done and if no, how can this be done?

1 Answer

0 votes
by (47.2k points)
  • You have to make sure you are pointing to the same metastore for spark-shell and thriftserver

  • Metastore sharing can be 2 ways, in simple

  1. Starting both shell and thrift from the same location

  2. Setting up a remote database for metastore

  • You can pass hive confs to Spark thrift server with --hiveconf and Spark confs with --conf

./sbin/start-thriftserver.sh \ --conf spark.sql.warehouse.dir=path/to/warehouse/dir \ --hiveconf hive.server2.thrift.port=<listening-port> \ --hiveconf hive.server2.thrift.bind.host=<listening-host> \ --master <master-uri> ...

31k questions

32.8k answers

501 comments

693 users

Browse Categories

...