python批量從sqlserver導表到hive

最近團隊要從公司某個部門拉取業務數據,由於某部門的數據庫是sqlserver,沒有生成hive表的腳本,於是自己寫了個粗糙版,裏邊有自動生成跑任務腳本的部分,就懶得貼了,自己定義個啓動腳本模板定義就去行了

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import pymssql
import sys
import os
import shutil
reload(sys)
sys.setdefaultencoding( "utf-8" )
centerConn=pymssql.connect(host='',
                           user=',
                           password='',
                           database='')
dwCursor = centerConn.cursor()

typeConvertDict={"bigint":"bigint", "int":"int", "varchar":"string", "decimal":"Double", "tinyint" : "int",
                 "datetime":"string", "date":"string", "time":"string", "year":"string",'timestamp':"bigint","char":"string","samllint":"int",
                 "double":"Double","text":"string","mediumtext":"string","varbinary":"binary","nvarchar":"string"
                 }

typeParquetMap= {
    "varchar":"String", "decimal":"Double", "datetime":"String",
    "date":"String", "time":"String", "year":"String",'timestamp':"bigint","varbinar
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章