topic Files load with Generic Schema in Talend Studio

Files load with Generic Schema

Anonymous — Sat, 16 Nov 2024 05:17:10 GMT

Hi All,

I've multiple csv files with different schema being sent to shared folder and i need to load the same to target i.e. HDFS. Also, note once loaded files have to be moved and new files will be sent to the folder over time.

Is there a way where i can define a generic schema and load all files at once.

Thanks!

Re: Files load with Generic Schema

Anonymous — Wed, 17 Jul 2019 08:02:27 GMT

Hello,

Here is a dynamic schema feature in talend which allows you to design schema with an unknown column structure (unknown name and number of columns).

Let us know if this article helps.

https://community.talend.com/t5/Design-and-Development/How-to-process-changing-data-structure/ta-p/21702?collapse_discussion=true&q=changing&search_type=thread

Best regards

Sabrina

Re: Files load with Generic Schema

Anonymous — Sat, 20 Jul 2019 15:52:14 GMT

Hi, Thanks for the reply.

However the example shows loading one kinda of schema to a table. I was looking for dynamic option such as loading multiple files to multiple different tables.

Cheers!