topic Re: Load schema dynamically from a file/table in Talend Studio

Load schema dynamically from a file/table

Anonymous — Sat, 16 Nov 2024 08:21:21 GMT

Hi,

I was reading this blog

https://www.talend.com/blog/2016/03/11/how-to-operationalize-meta-data-in-talend-with-dynamic-schemas/ that talks about loading a schema at runtime. This blog looks incomplete and I want to know how to use the dynamically read schema

Does anybody know how to do it or have any other example of loading schema dynamically?

My requirement is of reading an input delimited file (CSV) and load its data into a database table.

We want to make it generic so that any file can be read and loaded into a table.

The schema of the file will be defined in another CSV file or database table.

I am using the Enterprise Edition, so I know I can use the Dynamic datatype but I have heard that with dynamic datatype the column's datatype are not correctly identified. So would like to know if i can load a schema programatically into a job.

Re: Load schema dynamically from a file/table

Anonymous — Tue, 24 Apr 2018 17:38:32 GMT

Since you have the enterprise version, here is a great example of how to load in a dynamic schema.

Re: Load schema dynamically from a file/table

Anonymous — Thu, 26 Apr 2018 07:43:01 GMT

Thanks Eric for sharing your blog and the video.

But I am not sure if the solution will work for my requirement.

My requirement is as follows:

The input file will be a CSV file with 'n' columns.

The schema to be read will be defined in a DB table.

The job needs to read the schema metadata from the table.

Then read the input file based on the schema read.

And then output the data read into a DB table.

Re: Load schema dynamically from a file/table

Anonymous — Thu, 26 Apr 2018 20:06:41 GMT

you could also do something similar to this where you can use the ttablelist to specify your table and then input it and output it to your target. This will be dynamic due to the ttablelist component

Re: Load schema dynamically from a file/table

YPMAL — Thu, 26 Apr 2018 21:02:16 GMT

Please elaborate... Componentwise

Re: Load schema dynamically from a file/table

Anonymous — Fri, 27 Apr 2018 07:50:01 GMT

The incoming data that the job has to read is from a CSV file and not a table. The data has to be written (output) to a database table. So using tTableList component will not work.

Re: Load schema dynamically from a file/table

Anonymous — Mon, 07 May 2018 12:03:12 GMT

Hi @ehigaki,

Can you please elaborate on how I can set the dynamic schema set in a tSetDynamicSchema to a tMSSqlInput component?

Thanks,

Ketan

Re: Load schema dynamically from a file/table

Anonymous — Mon, 14 May 2018 18:10:17 GMT

So you would connect the tSetDynamicSchema to the tMSSqlInput component using an OnComponentOk. Once connected change the schema of the tMSSql Component to take dynamic type and tables name to use the global variable from tMSSqltableList.

Re: Load schema dynamically from a file/table

Anonymous — Sat, 23 Mar 2019 14:23:07 GMT

Any solution for openstudio?