1.文檔編寫目的
在前面的文章中,Fayson介紹了《如何在Redhat7.4安裝CDH6.0》,這裏我們基於這個環境開始安裝Kerberos。關於CDH啓用Kerberos的文章,前面Fayson也介紹過《如何在CDH集羣啓用Kerberos》、《如何在Redhat7.3的CDH5.14中啓用Kerberos》、《如何在Redhat7.4的CDH5.15中啓用Kerberos》和《如何在CDH6.0.0-beta1中啓用Kerberos》,通過本文,我們也可以來看看CDH6啓用Kerberos有哪些不一樣的地方。
- 內容概述:
1.如何安裝及配置KDC服務
2.如何通過CDH啓用Kerberos
3.如何登錄Kerberos並訪問Hadoop相關服務
4.總結
- 測試環境:
1.操作系統:Redhat7.4
2.CDH6.0
3.採用root用戶進行操作
2.KDC服務安裝及配置
本文檔中將KDC服務安裝在Cloudera Manager Server所在服務器上(KDC服務可根據自己需要安裝在其他服務器)
1.在Cloudera Manager服務器上安裝KDC服務
[root@ip-172-31-6-83 ~]# yum -y install krb5-server krb5-libs krb5-auth-dialog krb5-workstation
(可左右滑動)
2.修改/etc/krb5.conf配置
[root@ip-172-31-6-83 ~]# vim /etc/krb5.conf # Configuration snippets may be placed in this directory as well includedir /etc/krb5.conf.d/ [logging] default = FILE:/var/log/krb5libs.log kdc = FILE:/var/log/krb5kdc.log admin_server = FILE:/var/log/kadmind.log [libdefaults] dns_lookup_realm = false ticket_lifetime = 24h renew_lifetime = 7d forwardable = true rdns = false default_realm = FAYSON.COM #default_ccache_name = KEYRING:persistent:%{uid} [realms] FAYSON.COM = { kdc = ip-172-31-6-83.ap-southeast-1.compute.internal admin_server = ip-172-31-6-83.ap-southeast-1.compute.internal } [domain_realm] .ap-southeast-1.compute.internal = FAYSON.COM ap-southeast-1.compute.internal = FAYSON.COM
(可左右滑動)
標紅部分爲需要修改的信息。
3.修改/var/kerberos/krb5kdc/kadm5.acl配置
[root@ip-172-31-6-83 ~]# vim /var/kerberos/krb5kdc/kadm5.acl */[email protected] *
(可左右滑動)
4.修改/var/kerberos/krb5kdc/kdc.conf配置
[root@ip-172-31-6-83 ~]# vim /var/kerberos/krb5kdc/kdc.conf [root@ip-172-31-6-83 ~]# cat /var/kerberos/krb5kdc/kdc.conf [kdcdefaults] kdc_ports = 88 kdc_tcp_ports = 88 [realms] FAYSON.COM = { #master_key_type = aes256-cts max_renewable_life= 7d 0h 0m 0s acl_file = /var/kerberos/krb5kdc/kadm5.acl dict_file = /usr/share/dict/words admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal }
(可左右滑動)
標紅部分爲需要修改的配置。
5.創建Kerberos數據庫
[root@ip-172-31-6-83 ~]# kdb5_util create –r FAYSON.COM -s Loading random data Initializing database '/var/kerberos/krb5kdc/principal' for realm 'FAYSON.COM', master key name 'K/[email protected]' You will be prompted for the database Master Password. It is important that you NOT FORGET this password. Enter KDC database master key: Re-enter KDC database master key to verify:
(可左右滑動)
此處需要輸入Kerberos數據庫的密碼。
6.創建Kerberos的管理賬號
[root@ip-172-31-6-83 ~]# kadmin.local Authenticating as principal root/[email protected] with password. kadmin.local: addprinc admin/[email protected] WARNING: no policy specified for admin/[email protected]; defaulting to no policy Enter password for principal "admin/[email protected]": Re-enter password for principal "admin/[email protected]": Principal "admin/[email protected]" created. kadmin.local: exit
(可左右滑動)
標紅部分爲Kerberos管理員賬號,需要輸入管理員密碼。
7.將Kerberos服務添加到自啓動服務,並啓動krb5kdc和kadmin服務
[root@ip-172-31-6-83 ~]# systemctl enable krb5kdc Created symlink from /etc/systemd/system/multi-user.target.wants/krb5kdc.service to /usr/lib/systemd/system/krb5kdc.service. [root@ip-172-31-6-83 ~]# systemctl enable kadmin Created symlink from /etc/systemd/system/multi-user.target.wants/kadmin.service to /usr/lib/systemd/system/kadmin.service. [root@ip-172-31-6-83 ~]# systemctl start krb5kdc [root@ip-172-31-6-83 ~]# systemctl start kadmin
(可左右滑動)
8.測試Kerberos的管理員賬號
[root@ip-172-31-6-83 ~]# kinit admin/[email protected] Password for admin/[email protected]: [root@ip-172-31-6-83 ~]# klist Ticket cache: FILE:/tmp/krb5cc_0 Default principal: admin/[email protected] Valid starting Expires Service principal 09/12/2018 12:54:17 09/13/2018 12:54:17 krbtgt/[email protected] renew until 09/19/2018 12:54:17
(可左右滑動)
9.爲集羣安裝所有Kerberos客戶端,包括Cloudera Manager
使用批處理腳本爲集羣所有節點安裝Kerberos客戶端
[root@ip-172-31-6-83 shell]# sh ssh_do_all.sh node.list 'yum -y install krb5-libs krb5-workstation'
(可左右滑動)
10.在Cloudera Manager Server服務器上安裝額外的包
[root@ip-172-31-6-83 shell]# yum -y install openldap-clients
(可左右滑動)
11.將KDC Server上的krb5.conf文件拷貝到所有Kerberos客戶端
使用批處理腳本將Kerberos服務端的krb5.conf配置文件拷貝至集羣所有節點的/etc目錄下:
[root@ip-172-31-6-83 shell]# sh bk_cp.sh node.list /etc/krb5.conf /etc/
(可左右滑動)
3.CDH集羣啓用Kerberos
1.在KDC中給Cloudera Manager添加管理員賬號
[root@ip-172-31-6-83 shell]# kadmin.local Authenticating as principal admin/[email protected] with password. kadmin.local: addprinc cloudera-scm/[email protected] WARNING: no policy specified for cloudera-scm/[email protected]; defaulting to no policy Enter password for principal "cloudera-scm/[email protected]": Re-enter password for principal "cloudera-scm/[email protected]": Principal "cloudera-scm/[email protected]" created. kadmin.local: exit
(可左右滑動)
2.進入Cloudera Manager的“管理” --> “安全”界面
3.選擇“啓用Kerberos”,進入如下界面
4.確保如下列出的所有檢查項都已完成
5.點擊“繼續”,配置相關的KDC信息,包括類型、KDC服務器、KDC Realm、加密類型以及待創建的Service Principal(hdfs,yarn,,hbase,hive等)的更新生命期等
6.不建議讓Cloudera Manager來管理krb5.conf, 點擊“繼續”
7.輸入Cloudera Manager的Kerbers管理員賬號,一定得和之前創建的賬號一致,點擊“繼續”
8.點擊“繼續”啓用Kerberos
9.Kerberos啓用完成,點擊“繼續”
10.勾選重啓集羣,點擊“繼續”
11.集羣重啓完成,點擊“繼續”
12.點擊“繼續”
點擊“完成”,至此已成功啓用Kerberos。
4.Kerberos使用
使用fayson用戶運行MapReduce任務及操作Hive,需要在集羣所有節點創建fayson用戶。
1.使用kadmin創建一個fayson的principal
[root@ip-172-31-6-83 shell]# kadmin.local Authenticating as principal admin/[email protected] with password. kadmin.local: addprinc [email protected] WARNING: no policy specified for [email protected]; defaulting to no policy Enter password for principal "[email protected]": Re-enter password for principal "[email protected]": Principal "[email protected]" created. kadmin.local: exit You have new mail in /var/spool/mail/root
(可左右滑動)
2.使用fayson用戶登錄Kerberos
[root@ip-172-31-6-83 shell]# kdestroy You have new mail in /var/spool/mail/root [root@ip-172-31-6-83 shell]# kinit fayson Password for [email protected]: [root@ip-172-31-6-83 shell]# klist Ticket cache: FILE:/tmp/krb5cc_0 Default principal: [email protected] Valid starting Expires Service principal 09/12/2018 13:10:19 09/13/2018 13:10:19 krbtgt/[email protected] renew until 09/19/2018 13:10:19
(可左右滑動)
3.在集羣所有節點添加fayson用戶
使用批量腳本在所有節點添加fayson用戶
[root@ip-172-31-6-83 shell]# sh ssh_do_all.sh node.list "useradd fayson"
(可左右滑動)
4.運行MapReduce作業
[root@ip-172-31-6-83 hadoop-mapreduce]# hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 10 1
(可左右滑動)
5.使用beeline連接hive進行測試
[root@ip-172-31-6-83 114-hdfs-NAMENODE]# beeline WARNING: Use "yarn jar" to launch YARN applications. SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Beeline version 2.1.1-cdh6.0.0 by Apache Hive beeline> !connect jdbc:hive2://localhost:10000/;principal=hive/[email protected] Connecting to jdbc:hive2://localhost:10000/;principal=hive/[email protected] Connected to: Apache Hive (version 2.1.1-cdh6.0.0) Driver: Hive JDBC (version 2.1.1-cdh6.0.0) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://localhost:10000/> show tables; INFO : Compiling command(queryId=hive_20180912131436_b299404d-8a2a-4d79-8ff5-490959188ab7): show tables INFO : Semantic Analysis Completed INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from deserializer)], properties:null) INFO : Completed compiling command(queryId=hive_20180912131436_b299404d-8a2a-4d79-8ff5-490959188ab7); Time taken: 1.407 seconds INFO : Executing command(queryId=hive_20180912131436_b299404d-8a2a-4d79-8ff5-490959188ab7): show tables INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20180912131436_b299404d-8a2a-4d79-8ff5-490959188ab7); Time taken: 0.067 seconds INFO : OK +-----------+ | tab_name | +-----------+ +-----------+ No rows selected (2.076 seconds) 0: jdbc:hive2://localhost:10000/> create table t1 (s1 string,s2 string); INFO : Compiling command(queryId=hive_20180912131453_e0afea04-95ce-49b3-9c93-4ae99e2d477f): create table t1 (s1 string,s2 string) INFO : Semantic Analysis Completed INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20180912131453_e0afea04-95ce-49b3-9c93-4ae99e2d477f); Time taken: 0.07 seconds INFO : Executing command(queryId=hive_20180912131453_e0afea04-95ce-49b3-9c93-4ae99e2d477f): create table t1 (s1 string,s2 string) INFO : Starting task [Stage-0:DDL] in serial mode INFO : Completed executing command(queryId=hive_20180912131453_e0afea04-95ce-49b3-9c93-4ae99e2d477f); Time taken: 0.107 seconds INFO : OK No rows affected (0.219 seconds)
(可左右滑動)
向test表中插入數據
0: jdbc:hive2://localhost:10000/> insert into t1 values('1','2'); 0: jdbc:hive2://localhost:10000/> select * from t1;
(可左右滑動)
執行一個Count語句
0: jdbc:hive2://localhost:10000/> select count(*) from t1;
(可左右滑動)
5.常見問題
1.使用Kerberos用戶身份運行MapReduce作業報錯
main : run as user is fayson main : requested yarn user is fayson Requested user fayson is not whitelisted and has id 501,which is below the minimum allowed 1000 Failing this attempt. Failing the application. 17/09/02 20:05:04 INFO mapreduce.Job: Counters: 0 Job Finished in 6.184 seconds java.io.FileNotFoundException: File does not exist: hdfs://ip-172-31-6-148:8020/user/fayson/QuasiMonteCarlo_1504382696029_1308422444/out/reduce-out at org.apache.hadoop.hdfs.DistributedFileSystem$20.doCall(DistributedFileSystem.java:1266) at org.apache.hadoop.hdfs.DistributedFileSystem$20.doCall(DistributedFileSystem.java:1258) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1258) at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1820) at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1844) at org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:314) at org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:354) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:363) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71) at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144) at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
(可左右滑動)
問題原因:是由於Yarn限制了用戶id小於1000的用戶提交作業;
解決方法:修改Yarn的min.user.id來解決
2.進行kinit操作後,執行MR作業報“User fayson not found”
問題原因:在集羣的節點上沒有fayson這個用戶
解決方法:需要在集羣所有節點添加fayson用戶
6.總結
- CDH6與CDH5啓用Kerberos的過程基本沒差別,除了CDH6的界面有些許變化外。
- 在CDH集羣中啓用Kerberos需要先安裝Kerberos服務(krb5kdc和kadmin服務)
- 在集羣所有節點需要安裝Kerberos客戶端,用於和kdc服務通信
- 在Cloudera Manager Server節點需要額外安裝openldap-clients包
- CDH集羣啓用Kerberos後,使用自己定義的fayson用戶向集羣提交作業需確保集羣所有節點的操作系統中存在fayson用戶,否則作業會執行失敗