Spark SQL 配置

配置好hive後，spark sql 配置就很簡單了，實際上只要配置一個hive-site.xml文件，拷貝到spark/conf目錄下。這個文件是給spark使用的，hive的配置文件用自己的配置文件。

1. 配置hive-site.xml

<configuration>
  <property>
    <name>hive.metastore.uris</name>
    <value>thrift://localhost:9083</value>
    <description>Thrift URI for the remote metastore. Used by metastore client to
        connect to remote metastore.</description>
  </property>
  <property>
    <name>hive.server2.thrift.min.worker.threads</name>
    <value>5</value>
    <description>Minimum number of Thrift worker threads</description>
  </property>
  <property>
    <name>hive.server2.thrift.max.worker.threads</name>
    <value>500</value>
    <description>Maximum number of Thrift worker threads</description>
  </property>
  <property>
    <name>hive.server2.thrift.port</name>
    <value>10000</value>
    <description>Port number of HiveServer2 Thrift interface. Can be overridden by
     setting $HIVE_SERVER2_THRIFT_PORT</description>
  </property>
  <property>
    <name>hive.server2.thrift.bind.host</name>
    <value>d-hdp-05</value>
    <description>Bind host on which to run the HiveServer2 Thrift interface.Can be
      overridden by setting$HIVE_SERVER2_THRIFT_BIND_HOST</description>
  </property>
</configuration>

3. 運行命令 啓動spark-sql

spark-sql --master spark://d-hdp-01:7077 --executor-memory 1g

3. 啓動thriftserver

start-thriftserver.sh --master spark://d-hdp-01:7077 --executor-memory 1g

使用beeline連接thriftserver

beeline

!connect jdbc:hive2://d-hdp-05:10000

運行效果