最近在做sqoop導出數據的時候,mysql中有的字段到hive中自動轉換成boolean類型了.這個需要解決,不然數據對不上.
查了查官方文檔,官方文檔裏有說明,這裏貼一下文檔.
27.2.5. MySQL: Import of TINYINT(1) from MySQL behaves strangely
Problem: Sqoop is treating TINYINT(1) columns as booleans, which is for example causing issues with HIVE import. This is because by default the MySQL JDBC connector maps the TINYINT(1) to java.sql.Types.BIT, which Sqoop by default maps to Boolean.
Solution: A more clean solution is to force MySQL JDBC Connector to stop converting TINYINT(1) to java.sql.Types.BIT by adding tinyInt1isBit=false into your JDBC path (to create something like jdbc:mysql://localhost/test?tinyInt1isBit=false). Another solution would be to explicitly override the column mapping for the datatype TINYINT(1) column. For example, if the column name is foo, then pass the following option to Sqoop during import: --map-column-hive foo=tinyint. In the case of non-Hive imports to HDFS, use --map-column-java foo=integer.
官方文檔裏也給出瞭解決方法,有兩個解決方法
- 在jdbc鏈接後面加上：tinyInt1isBit=false. 例:
--connecttion jdbc:mysql://localhost/test?tinyInt1isBit=false - 顯式覆蓋數據類型TINYINT（1）列的列映射. 例:
如果列名爲foo，則在導入期間將以下選項傳遞給Sqoop： – map-column-hive foo = tinyint。在非Hive導入HDFS的情況下，使用–map-column-java foo = integer
參考: 官方文檔