Reading delimited file with dynamic column

[AManohar]

Hi,

I have a requirement to read a file with dynamic columns and load the mandatory fields into DB.

The dynamic columns are in the middle of the file.

But, based on the config we can figure out how many additional fields are added in this file.

sample:

col1|col2|col3|col4.1|col4.2|col4.3|col5|col6

I want to read this and store it as

col1|col2|col3|"

col4.1;col4.2;col4.3

"|col5|col6

Based on a config I would know there are 3 fields for col4, but some files might have 6 or even 1. That is the variable column.

How to read and process such files in talend.

Thank you.

Regards,

Anitha

[Anonymous]

I think you need dynamic schema functionality which is only on subscription products. For more details, please refer to this documentation page.

https://help.talend.com/r/en-US/8.0/dynamic-schema

Access the schema properties and filter the columns based on the column name at runtime.

Regards

Shong

[AManohar]

Hi Shong,

But in our case we dont have a column name.

Its just a pipe separated file.

eg : Name|language1|language2|language3|age

in this case, some file will have just one language, but some might have many. so we still need to store those additional languages in separate column.

But, name,language1 and age should be stored in a main table.

[Anonymous]

Name|language1|language2|language3|age

this line is not the column name? There must have a header line when using a dynamic schema feature.