Hive on Spark 注意事項

1、hive-site.xml

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
 <configuration>
	<property>
	<name>javax.jdo.option.ConnectionURL</name>
	<value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value>
	<description></description>
	</property>

	<property>
	    <name>javax.jdo.option.ConnectionDriverName</name>
	    <value>com.mysql.jdbc.Driver</value>
	    <description></description>
	</property>

	<property>
	    <name>javax.jdo.option.ConnectionUserName</name>
	    <value>root</value>
	    <description></description>
	</property>

	<property>
	    <name>javax.jdo.option.ConnectionPassword</name>
	    <value>root</value>
	    <description></description>
	</property>

	<property>  
                   <name>spark.master</name>  
                   <value>spark://pc2:7077</value>  
	</property> 
</configuration>

2、將此配置文件分發到集羣中【每一個結點】的spark的conf目錄下

3、將【每一個】spark結點下的jars目錄中的hive開頭的jar包刪除

rm hive*

4、將hive的lib目錄下的hive開頭的jar包複製到【每一個】spark結點的

scp hive* chenjie@pc3:/home/chenjie/spark-2.0.2-hadoop2.7

5、啓動hive

6、hive命令 set hive.execution.engine=spark;

7、hive命令 set hive.execution.engine=spark;

如果要切換回mapreduce方式

set hive.execution.engine=mr;