topic Hbase ETL to Teradata issues in Talend Studio

Hbase ETL to Teradata issues

Anonymous — Sat, 16 Nov 2024 03:01:19 GMT

Hi all,

I've got a job i'm working on which ETL's data from HBASE --> TERADATA table and the job will load 9/12 columns but for whatever reason when going into terdata will drop off some of the integer columns and i'm confused as to why. So far i've started by removing the mapping and making it so that the job writes to a text file, i can see that the values for these columns which do not appear in teradata appearing in the text file as expected. Although for whatever reason they're not making it past the mapping into the teradata table. Oddly enough the exact same job works in a lower environment exactly as is with no changes, also to note i've had a play with some of the data types on the receiving table with no luck.

Any comments/advice greatly appreciated

Thanks John

Re: Hbase ETL to Teradata issues

manodwhb — Wed, 11 Mar 2020 03:30:50 GMT

@Baller4sho , when you say it is working in lower environments means in the lower environments teradata tables is it loading correctly? then it may be configuration of teradata issue?

Re: Hbase ETL to Teradata issues

Anonymous — Wed, 11 Mar 2020 04:35:57 GMT

Hi,

Thanks for the response, I actually managed to resolve this one. Despite those columns being integer in nature, I read them in as strings and updated the destination table to accept varchars.
I know the philosophy around schema's with hbase but if someone understands why it would be nice to get an explanation