R中HIVE實戰

R連接hive報錯：

R中報錯java.lang.NoClassDefFoundError: com/google/protobuf/ProtocolMessageEnum

hiveserver2端報錯java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: Administrator is not allowed to impersonate hive

解決辦法：hadoop中配置文件core-site.xml加以下內容

<property> 
<name>hadoop.proxyuser.root.hosts</name>
<value>*</value> 
</property> 
<property> 
<name>hadoop.proxyuser.root.groups</name> 
<value>*</value> 
</property>

hdfs-site.xml中加：

<property>   
<name>dfs.webhdfs.enabled</name>   
<value>true</value>   
</property>

將root更改爲被禁止的用戶，這裏是administrator

install.package("rJava")
install.package("DBI")
install.packages("RJDBC")
library('rJava')
library('DBI')
library('RJDBC')
drv<-JDBC("org.apache.hive.jdbc.HiveDriver","hive-jdbc-2.3.6-standalone.jar;hadoop-common-2.9.2.jar")
conn <- dbConnect(drv,"jdbc:hive2://ip:10000/default","hive","hive")
test_data<-dbGetQuery(conn,"select * from test ") 

向HIVE寫入數據報錯

org.apache.hadoop.security.AccessControlException Permission denied: user=hive, access=WRITE, inode="/hive/warehouse":Administrator:supergroup:drwxr-xr-x

更改權限：hdfs-site.xml文件中加

<property>  
<name>dfs.permissions.enabled</name>  
<value>false</value>  
</property>

R連接HIVE並提取數據，提取數據的進程可以在hive的hiveserver2命令窗口看到 

drv<-JDBC("org.apache.hive.jdbc.HiveDriver","hive-jdbc-2.3.6-standalone.jar;hadoop-common-2.9.2.jar")
conn <- dbConnect(drv,"jdbc:hive2://IP:10000/default","hive","hive")
test_data<-dbGetQuery(conn,"select * from test")

對於向HIVE中插入/創建/更新數據遇到了個bug，雖然數據似乎可以進行操作，hive命令窗口顯示成功操作，但R端口返回報錯，暫未解決