Load schema dynamically from a file/table

[Anonymous]

Hi,

I was reading this blog

https://www.talend.com/blog/2016/03/11/how-to-operationalize-meta-data-in-talend-with-dynamic-schema... that talks about loading a schema at runtime. This blog looks incomplete and I want to know how to use the dynamically read schema 

Does anybody know how to do it or have any other example of loading schema dynamically?

My requirement is of reading an input delimited file (CSV) and load its data into a database table.

We want to make it generic so that any file can be read and loaded into a table.

The schema of the file will be defined in another CSV file or database table.

I am using the Enterprise Edition, so I know I can use the Dynamic datatype but I have heard that with dynamic datatype the column's datatype are not correctly identified. So would like to know if i can load a schema programatically into a job.

[Anonymous]

Since you have the enterprise version, here is a great example of how to load in a dynamic schema.

[Anonymous]

Thanks Eric for sharing your blog and the video.

But I am not sure if the solution will work for my requirement.

My requirement is as follows:

The input file will be a CSV file with 'n' columns.

The schema to be read will be defined in a DB table.

The job needs to read the schema metadata from the table.

Then read the input file based on the schema read.

And then output the data read into a DB table.

[Anonymous]

you could also do something similar to this where you can use the ttablelist to specify your table and then input it and output it to your target. This will be dynamic due to the ttablelist component 

[YPMAL]

Please elaborate... Componentwise

[Anonymous]

The incoming data that the job has to read is from a CSV file and not a table. The data has to be written (output) to a database table. So using tTableList component will not work.

[Anonymous]

Hi @ehigaki,

Can you please elaborate on how I can set the dynamic schema set in a tSetDynamicSchema to a tMSSqlInput component?

Thanks,

Ketan

[Anonymous · Accepted solution]

So you would connect the tSetDynamicSchema to the tMSSqlInput component using an OnComponentOk. Once connected change the schema of the tMSSql Component to take dynamic type and tables name to use the global variable from tMSSqltableList.

[Anonymous]

Any solution for openstudio?