tHiveLoad set partitions, make the partition be system date.
Hi everyone:
I am using tHiveLoad to load data into hive table, my hive table is as follow:
hive> show create table jigou;
OK
CREATE TABLE `jigou`(
`id` string,
`column1` string,
`column2` string,
`column3` string,
`column4` string,
`column5` string,
`column6` string,
`column7` string,
`column8` string,
`column9` string,
`column10` string,
`column11` string,
`column12` string)
PARTITIONED BY ( `dt` string) ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\u0001'
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
'hdfs://h1:8020/apps/hive/warehouse/zhangchao.db/jigou'
TBLPROPERTIES (
'transient_lastDdlTime'='1435650326')
Time taken: 0.474 seconds, Fetched: 26 row(s)
My talend work flow is as follow, I want to make the dt partition dynamic, each time the job is scheduled, it read the system date automaticly.
change it to "dt='2015-06-30'" above should work for you, I tested this sqoopLoad but you can test with Hive Load, as per my knowledge it requires string values enclosed with single quote. and for dynamic date you can use below function.