topic Re: tHiveInput - Extracting Empty Strings as NULL in Talend Studio

tHiveInput - Extracting Empty Strings as NULL

rbk_tal — Sat, 16 Nov 2024 02:58:48 GMT

Hi all,

I currently have a simple job extracting data from Hive and load into target database.

I am currently using a tHiveInput component to extract from Hive and load using a tDbOutput component.

We have several fields having empty strings which I expect to get loaded into the target as NULLs.

Currently I am using a tmap to check if they are empty and then load them as NULL. But considering the number of columns involved the development process is becoming difficult. Is there any settings we can set to get the expected result ? Maybe something in the Hadoop Settings or Hive Settings in the Advanced Tab of the tHiveInput component ?

Thanks in advance for all the help.

Cheers,

RBK

Re: tHiveInput - Extracting Empty Strings as NULL

manodwhb — Mon, 16 Mar 2020 06:30:47 GMT

@rbk_tal ,Write some routine code and call the same for all the columns to empty to null.

Re: tHiveInput - Extracting Empty Strings as NULL

rbk_tal — Thu, 19 Mar 2020 06:15:16 GMT

Thank you so much for your response....We will give that a try....But that would still mean I need to create separate jobs for each tables right since the structure is different for each table...Any setting that we can provide on the Hive or Hadoop properties that would be applicable to all such columns ?

Goal is to have a single job - extracting from source and loading into target using Dynamic schema for all the tables we have in Hive.

Re: tHiveInput - Extracting Empty Strings as NULL

rbk_tal — Thu, 02 Apr 2020 12:07:27 GMT

The values are loaded in Hive as NULL, meaning if I query the hive table checking for the condition where a specific column is NULL, I am able to get results. I merely want the output as NULL from the tHiveInput component. Would be great if there is any setting I can set so that I get this value as NULL instead of empty strings.