topic File processing using Dynamic Schema in Talend Studio

File processing using Dynamic Schema

Anonymous — Tue, 05 Dec 2017 20:06:12 GMT

Currently using Talend Data Integration Version: 6.4.1 Build id: 20170623_1246 (aka Enterprise)

Problem:

We are receiving clinical text csv files with differing numbers of columns in the files.

There is a core set of columns we want to retrieve from all of the files. There is also

a set of columns that will be listed multiple times which we want to concatenate

together and continue to pass along in the flow.

So how to I retrieve the "dynamic schema" column information and the value of the column coming from a component?

Thank you

Peter

Re: File processing using Dynamic Schema

Anonymous — Wed, 06 Dec 2017 06:41:36 GMT

Hello,

@pd_ wrote:

There is a core set of columns we want to retrieve from all of the files. There is also

a set of columns that will be listed multiple times which we want to concatenate

together and continue to pass along in the flow.

With your job requirement, could you please elaborate your case with an example with input and expected output values?

Here is an article that introduces the dynamic schema feature:https://community.talend.com/t5/Design-and-Development/How-to-process-changing-data-structure/ta-p/21702

Hope it helps.

Best regards

Sabrina