topic Re: Joining tables of differing schema into one in Talend Studio

Joining tables of differing schema into one

Anonymous — Fri, 21 Sep 2018 10:40:51 GMT

I've been running a Talend job which picks up information from multiple databases / tables, iterates over these, and then eventually inserts this information together in a final table.

Where I'm having an issue is:

Historically the schema on these input tables have been the same, however more recently, one of the tables has new variables input into it, variables that are not in the other tables. I'm trying to bring these new variables into the rest of the process as it is useful information. The problem is, when the flow goes through the iterations and it hits an iteration where the table does not have these new variables in, the job falls over. Stating an error like 'variable_a does not exist'.

Is there is any work-around for this? I've got around it at for now by only bringing through the table which has this extra information, but in the future I am going to need the other tables. Please see attached a screenshot of my flow for some background.

Thanks,

Pranay

Re: Joining tables of differing schema into one

TRF — Fri, 21 Sep 2018 10:51:19 GMT

So, you hve to harmonize the schemas by addind the desired fields.

This can be done by a tMap.

You can also add pseudo-columns in your select statements to unify the schemas resulting of all your queries.

Re: Joining tables of differing schema into one

fdenis — Fri, 21 Sep 2018 10:52:28 GMT

create 2 version of your job.
then run job 2. on error on job2, run job1
or
add a check of this column (using system table or simple query)
and then choose the right version.