topic How to process multiple input files with different columns in talend in Talend Studio

How to process multiple input files with different columns in talend

Anonymous — Mon, 08 Oct 2018 07:51:32 GMT

Hi Experts,

I am new to talend and want to understand how to process multiple files in same job with different column names.

For ex:

text1:

account

identity

customer

text2:

acctn

cus

I have thousands file like given above and I have to implement same business logic for every file only challenge is column names are different in every file. So please let me know how dynamically we can process these files in talend.

thanks in advance

Re: How to process multiple input files with different columns in talend

TRF — Mon, 08 Oct 2018 08:30:55 GMT

Column names don't matter as soon as the number of column and their signification are the same for all the files.

Re: How to process multiple input files with different columns in talend

sbxr — Tue, 09 Oct 2018 18:39:05 GMT

Hi @abhityagi,

It will be better if you can tell , what exactly you want. Expalin better so that we can help you better.

Regards

Re: How to process multiple input files with different columns in talend

Anonymous — Wed, 10 Oct 2018 05:41:08 GMT

ok thanks for your reply. Let me give you one example.

I have thousands of file all the files have different names and different column names. I want to apply business logic such as if any column is null then assigned 'Y' value to new variable or assigned 'N'. I don 't want to create again and again schema for any new file or for column names. I want to create a job which can read any file with any column name convention and also create new variable as per their assigned value.

file1:

col:

account

name

gender

file2:

col:

acctn

cust

sex

Thanks,

Abhishek

Re: How to process multiple input files with different columns in talend

Anonymous — Thu, 11 Oct 2018 04:41:16 GMT

Hello,

The Dynamic schema feature in talend allows you to design schema with an unknown column structure (unknown name and number of columns).

Please refer to this KB article about:https://community.talend.com/t5/Design-and-Development/How-to-process-changing-data-structure/ta-p/21702

Let us know if it is OK with you.

Best regards

Sabrina