Retrive Schema of a file Dynamically

[DSuryadevara]

I have a requirement where we get 40 *.txt files with different schemas and we need to load it to respective tables by reading file after file.

is there a way i can pull schema of the particular file based on filename and run dynamically for each file in the same manner.

Can anyone suggest how i should proceed with this senario

[Anonymous]

Hello,

Here is a feature "Dynamic Schema" in Talend Subscription solution which allows you to design schema with an unknown column structure (unknown name and number of columns).

For your use case, you can use tfilelist to fetch your multiple files with mask *.txt and try to load them into respective tables by using dynamic schema.

The work flow should be: tfilelist-->tfileinputdelimited(dynamicSchema)-->tDBOutput(dynamicSchema).

Please take a look at KB article about:

https://community.talend.com/s/article/How-to-process-changing-data-structure-ExK7j

Feel free to let us know if it is OK with you.

Best regards

Sabrina

[DSuryadevara]

Hello Sabrina,

thanks for the response, this works but few columns coming from file are string or long timestamps like 20220909334455 but i need to load to tables in 09/09/2022 33:44:55 format. how will i be able to do it when running dynamically. will there be any way to convert long data type columns to Date data type for this particular job.

[Anonymous]

Hello,

You are able to extract and map dynamic fields in tmap or tjavarow component.

Hope these online documentations help.

https://help.talend.com/r/en-US/8.0/processing/twritedynamicfields-tfixedflowinput-tjavarow-tlogrow-...

Best regards

Sabrina