HiveServer2高可用配置

hive—high Avaliable

​ hive的搭建方式有三種，分別是

​ 1、Local/Embedded Metastore Database (Derby)

​ 2、Remote Metastore Database

​ 3、Remote Metastore Server

​ 一般情況下，我們在學習的時候直接使用hive –service metastore的方式啓動服務端，使用hive的方式直接訪問登錄客戶端，除了這種方式之外，hive提供了hiveserver2的服務端啓動方式，提供了beeline和jdbc的支持，並且官網也提出，一般在生產環境中，使用hiveserver2的方式比較多，如圖：

使用hiveserver2的優點如下：

​ 1、在應用端不需要部署hadoop和hive的客戶端

​ 2、hiveserver2不用直接將hdfs和metastore暴露給用戶

​ 3、有HA機制，解決應用端的併發和負載問題

​ 4、jdbc的連接方式，可以使用任何語言，方便與應用進行數據交互

本文檔主要介紹如何進行hive的HA的搭建：

如何進行搭建，參照之前hadoop的HA，使用zookeeper完成HA

1、環境如下:

	Node01	Node02	Node03	Node04
Namenode	1	1
Journalnode	1	1	1
Datanode		1	1	1
Zkfc	1	1
zookeeper	1	1	1
resourcemanager		1	1	1
nodemanager		1	1	1
Hiveserver2			1
beeline				1

2、node02—hive-site.xml

<property>  
  <name>hive.metastore.warehouse.dir</name>  
  <value>/user/hive/warehouse</value>  
</property>  
<property>  
  <name>javax.jdo.option.ConnectionURL</name>  
  <value>jdbc:mysql://node01:3306/hive?createDatabaseIfNotExist=true</value>  
</property>  
<property>  
  <name>javax.jdo.option.ConnectionDriverName</name>  
  <value>com.mysql.jdbc.Driver</value>  
</property>     
<property>  
  <name>javax.jdo.option.ConnectionUserName</name>  
  <value>root</value>  
</property>  
<property>  
  <name>javax.jdo.option.ConnectionPassword</name>  
  <value>123</value>  
</property>
<property>
  <name>hive.server2.support.dynamic.service.discovery</name>
  <value>true</value>
</property>
<property>
  <name>hive.server2.zookeeper.namespace</name>
  <value>hiveserver2_zk</value>
</property>
<property>
  <name>hive.zookeeper.quorum</name>
  <value>node01:2181,node02:2181,node03:2181</value>
</property>
<property>
  <name>hive.zookeeper.client.port</name>
  <value>2181</value>
</property>
<property>
  <name>hive.server2.thrift.bind.host</name>
  <value>node02</value>
</property>
<property>
  <name>hive.server2.thrift.port</name>
  <value>10001</value> 
</property>

3、node4—hive-site.xml

<property>  
  <name>hive.metastore.warehouse.dir</name>  
  <value>/user/hive/warehouse</value>  
</property>  
<property>  
  <name>javax.jdo.option.ConnectionURL</name>  
  <value>jdbc:mysql://node01:3306/hive?createDatabaseIfNotExist=true</value>  
</property>  
<property>  
  <name>javax.jdo.option.ConnectionDriverName</name>  
  <value>com.mysql.jdbc.Driver</value>  
</property>     
<property>  
  <name>javax.jdo.option.ConnectionUserName</name>  
  <value>root</value>  
</property>  
<property>  
  <name>javax.jdo.option.ConnectionPassword</name>  
  <value>123</value>  
</property>
<property>
  <name>hive.server2.support.dynamic.service.discovery</name>
  <value>true</value>
</property>
<property>
  <name>hive.server2.zookeeper.namespace</name>
  <value>hiveserver2_zk</value>
</property>
<property>
  <name>hive.zookeeper.quorum</name>
  <value>node01:2181,node02:2181,node03:2181</value>
</property>
<property>
  <name>hive.zookeeper.client.port</name>
  <value>2181</value>
</property>
<property>
  <name>hive.server2.thrift.bind.host</name>
  <value>node04</value>
</property>
<property>
  <name>hive.server2.thrift.port</name>
  <value>10001</value> 
</property>

啓動zk,再啓動hiveserver2

4、使用jdbc或者beeline兩種方式進行訪問

1） beeline

!connect jdbc:hive2://node01,node02,node03/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2_zk root 123

2）jdbc

public class HiveJdbcClient2 {

	private static String driverName = "org.apache.hive.jdbc.HiveDriver";

	public static void main(String[] args) throws SQLException {
		try {
			Class.forName(driverName);
		} catch (ClassNotFoundException e) {
			e.printStackTrace();
		}

		Connection conn = DriverManager.getConnection("jdbc:hive2://node01,node02,node03/default;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2_zk", "root", "");
		Statement stmt = conn.createStatement();
		String sql = "select * from tbl";
		ResultSet res = stmt.executeQuery(sql);
		while (res.next()) {
			System.out.println(res.getString(1));
		}
	}
}

​