Dynamic Schema

[Anonymous]

I am evaluating Talnd Big Data edition for data consumption job.
I have a set of 5 tables , my requirements is join these five tables and generate subset of columns as o/p in a hive table.
Number of Output columns are not fixed , that is why I need to generate schema dynamically.
I am using tHiveInput to read the data and tsqrow to join these five tables.
Questions -
1. Can tSqlRow generate schema dynamically ?
2. I can use tjava but the problem with t Java is it can't take more than one input.
3. I can't find tSetDynamic component in talend big data studio.
Thanks

[Anonymous]

Hi,

    I would recommend to go through the below blogs.

https://www.talend.com/blog/2019/10/31/magically-shuffle-columns-talend-dynamic-schema/

https://www.talend.com/blog/2019/11/11/migrate-data-between-databases-one-job-using-dynamic-schema/

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

[Anonymous]

Thanks for the response. 

However the links are not applicable to Talend Big Data Jobs.

Talend Big Data Job does not support dynamic schema which is a major limitation. Is there a workaround of this limitation ?

if we cant overcome this limitatatin we cant really use talend , in that case we need to write spark code in pure java 

Thanks