原创 How-to: deploy hadoop client with some special user based on acl enbaled cluster
This is to deploy a hadoop client with some user besides hadoop admin user to connect an acl(configure user access perm
2020-07-03 14:12:35
原创 How-to: make spark streaming collect data from Kafka topics and store data into hdfs
Develop steps: Develop class which is used for connect kafka topics and store data into hdfs. In spark project: ./exam
2020-07-03 14:12:35
原创 How-to: resolve regionserver died with "No lease on /hbase/oldWALs/..."
Error Log from died regionserver: 2015-06-11 16:23:03,072 ERROR [regionserver/slave04/172.31.34.64:60020] regionserver
2020-07-03 14:12:35
原创 How-to resolve hbase shutdown caused by "KeeperErrorCode = ConnectionLoss for /hbase"
Error log: 2015-06-23 17:35:05,995 WARN [main] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=s
2020-07-03 14:12:35
原创 How-to: effective store kafka data into hdfs via spark streaming
This is an improvement to How-to: make spark streaming collect data from Kafka topics and store data into hdfs In H
2020-07-03 14:12:35
原创 How-to: install puppet via yum
Master: sudo rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-5.noarch.rpmsudo yum install puppet-serverAdd fol
2020-02-21 01:21:16
原创 How-to: resolve hbase "org.apache.hadoop.hbase.TableExistsException: hbase:namespace"
Error: 2015-06-24 13:34:05,251 FATAL [master:60000.activeMasterManager] master.HMaster: Failed to become active master
2020-02-21 01:21:15
原创 How-to: enable HDFS HA at a new cluster
deploy hadoop cluster with non-HA: make sure hadoop could work normallyConfigure passwd-less from new namenode to othe
2020-02-21 01:21:15
原创 how-to: resolve "Connection refused" during connext hiveserver2 via beeline
Issue: At hiveserver2 node, could connect hiveserver2 via localhost/127.0.0.1, but connection refused when using ip. At
2020-02-21 01:21:15
原创 how-to: resolve "java.lang.OutOfMemoryError: Java heap space" during using beeline && hiveserver2
Error log in hive.log file: 2015-06-10 00:33:18,207 ERROR [HiveServer2-Handler-Pool: Thread-47]: thrift.ProcessFuncti
2020-02-21 01:21:04
原创 How-to: resolve "Unapproved licenses:" issue during building with mvn+rat
This issue happened during building flume, but this has nothing to do with flume. mvn verbose output will mention abou
2020-02-21 01:21:04
原创 How-to: Enable yarn ResourceManager HA
Update yarn-site.xml. Add following in yarn-site.xml(master.chff.dc and slave01.chff.dc are resourcemanager nodes):
2020-02-21 01:21:04
原创 How-to: resolve " java.lang.OutOfMemoryError: unable to create new native thread" for hbase thrift
Error: Caused by: java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Nat
2020-02-21 01:21:04
原创 How-to: Enbale HMaster HA(high availability) and High Available Reads
At first, please make sure the backup HMaster node's hostname is configured at all hbase nodes /etc/hosts file. Then ad
2020-02-21 01:21:04
原创 Hadoop cluster security2: How to enable hadoop Service Level Authorization
Reference:http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ServiceLevelAuth.html Steps: Add fo
2020-02-21 01:21:04