How to convert dynamic column into multiple standard columns ?

[laughsmile]

Hello everyone.

I am using Talend Data Integration, and want to read csv files which have different format(header) , So I use tFileInputDelimited and dynamic column to read them.

Is it possible to convert dynamic column into multiple standard columns? I am trying tJavaRow's something like  "output_row.addColumn" , but it does not work.

Thanks in advance.

[Shicong_Hong]

Hello

I'm afraid you can't add standard columns to schema at runtime. Talend always requires us to configure columns on the schema at design time.  tExtractDynamicFields can extract fields in order from a Dynamic schema, but it requires you to fill in the column names. What is your target app? Can you please provide more information about your use case?

Regards

Shicong

[jlolling]

The normal schema is simply a java class with member fields for the schema columns. This class cannot be dynamically extended with columns.

It would be helpful to know what is your use case or goal.

[laughsmile]

Hello @Shicong_Hong @jlolling 

Thanks for your answers.

We use job to read input csv files and do lots of transformation work and join operations to the specific columns of csv , at last save data into database.

The columns of database table is something like "key,Col2,Col3,Col4,Col5,Col6,Col7..... ",

But the input csv header is something like "key,Col2,Col6..... ", or "key,Col2,Col5,Col6..... ", or "key,Col5,Col7..... " , which change dynamically.

That is to say , I can not configure columns on the schema at design time and can not use tExtractDynamicFields to extract fields from schema.

For executing all kinds of transformation work and join operations to the specific columns , I want to change dynamic column into standard columns.

Is it possible to achieve this goal?

Thanks.

[Shicong_Hong · Accepted solution]

if the csv files always contain part of the full columns of database, extract all columns using tExtractDynamicFields, the corresponding columns will be empty if the current csv file don't contain some of columns. My point is that you should extract all columns so that you can configure transformation and join operations on next components.