Sqoop1.4.7安裝

一.簡介

Sqoop 是一款進行數據傳輸的工具, 可在hadoop 的 hdfs 和關係型數據庫之間傳輸數據
可以使用Sqoop把數據從MySQL 或 Oracle導入到hdfs中, 也可以把數據從hdfs導入到MySQL或Oracle中

二.安裝與配置

基於<<centos7 Hive2.3.6安裝與配置>>

1. 下載與解壓

cd /usr/local

wget http://www.apache.org/dist/sqoop/1.4.7/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz

tar -zxvf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz

2. 配置環境變量

# 配置環境變量
vi /etc/profile
export SQOOP_HOME=/usr/local/sqoop-1.4.7.bin__hadoop-2.6.0
export PATH=$PATH:$SQOOP_HOME/bin

# 使環境變量生效
source /etc/profile

3. $SQOOP_HOME/conf 修改sqoop_env.sh配置

cd /usr/local/sqoop-1.4.7.bin__hadoop-2.6.0/conf
cp sqoop-env-template.sh sqoop-env.sh

vim sqoop-env.sh
#在sqoop_env.sh中
export HADOOP_COMMON_HOME=/usr/local/hadoop-2.7.7
export HADOOP_MAPRED_HOME=/usr/local/hadoop-2.7.7
export HIVE_HOME=/usr/local/apache-hive-2.3.6-bin
export HBASE_HOME=/usr/local/hbase-2.1.1
# export ZOOCFGDIR=

4. 拷貝 mysql驅動到$SQOOP_HOME/lib目錄下

cp $HIVE_HOME/lib/mysql-connector-java-5.1.48.jar /usr/local/sqoop-1.4.7.bin__hadoop-2.6.0/lib

5. 測試sqoop環境

sqoop version

出現以下內容表示安裝成功

Warning: /usr/local/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /usr/local/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /usr/local/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
Error: Could not find or load main class org.apache.hadoop.hbase.util.GetJavaProperty
19/12/06 20:05:29 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
Sqoop 1.4.7
git commit id 2328971411f57f0cb683dfb79d19d4d19d185dd8
Compiled by maugli on Thu Dec 21 15:59:58 STD 2017

列舉所有的數據庫

sqoop list-databases --connect jdbc:mysql://localhost:3306/ --username root --password Cocktail_py