最近在做sqoop导出数据的时候,mysql中有的字段到hive中自动转换成boolean类型了.这个需要解决,不然数据对不上.
查了查官方文档,官方文档里有说明,这里贴一下文档.
27.2.5. MySQL: Import of TINYINT(1) from MySQL behaves strangely
Problem: Sqoop is treating TINYINT(1) columns as booleans, which is for example causing issues with HIVE import. This is because by default the MySQL JDBC connector maps the TINYINT(1) to java.sql.Types.BIT, which Sqoop by default maps to Boolean.
Solution: A more clean solution is to force MySQL JDBC Connector to stop converting TINYINT(1) to java.sql.Types.BIT by adding tinyInt1isBit=false into your JDBC path (to create something like jdbc:mysql://localhost/test?tinyInt1isBit=false). Another solution would be to explicitly override the column mapping for the datatype TINYINT(1) column. For example, if the column name is foo, then pass the following option to Sqoop during import: --map-column-hive foo=tinyint. In the case of non-Hive imports to HDFS, use --map-column-java foo=integer.
官方文档里也给出了解决方法,有两个解决方法
- 在jdbc链接后面加上：tinyInt1isBit=false. 例:
--connecttion jdbc:mysql://localhost/test?tinyInt1isBit=false - 显式覆盖数据类型TINYINT（1）列的列映射. 例:
如果列名为foo，则在导入期间将以下选项传递给Sqoop： – map-column-hive foo = tinyint。在非Hive导入HDFS的情况下，使用–map-column-java foo = integer
参考: 官方文档