topic Re: Dynamic Target Structure based on Data availability in source. in Talend Studio

Dynamic Target Structure based on Data availability in source.

NarenS701 — Sat, 16 Nov 2024 00:05:46 GMT

Hi Team,

I have a scenario,

Please refer to attachment for sample data and expected output.

i need to create target structure dynamically based on data in source file. Need to check the Product and remove corresponding column set. For example: if product_5 is null then remove the corresponding column set(Product_5, Price_5, Quantity_5). Similarly i have to check from 5 to backwards, If product column have data then stop checking and define the structure and write all rows to output file. Can someone suggest how to do this?

Thanks.

Re: Dynamic Target Structure based on Data availability in source.

gjeremy1617088143 — Tue, 01 Jun 2021 06:51:19 GMT

Hi, if you have a licensed version you can use dynamics type.

in a tjava you can create a new dynamics object an generate the column you want with the conditions you want

or get the input as a dynamic object and manipulate the column in a tjava.

Send me Love and Kudos

Re: Dynamic Target Structure based on Data availability in source.

NarenS701 — Thu, 03 Jun 2021 05:41:22 GMT

Thanks for your reply @guenneguez jeremy . This is not working, I have tried with dynamic object data type. My requirement is little different, I want to have dynamic target schema.

Re: Dynamic Target Structure based on Data availability in source.

gjeremy1617088143 — Thu, 03 Jun 2021 06:41:14 GMT

You set your input as dynamic :

in a tjava row :

you create a new dynamic object

in a loop you get all the metadata of the dynamic input, you filter on your conditions, you add only the columns with metadata you want in the new dynamic and you set the output with the new dynamic.

Re: Dynamic Target Structure based on Data availability in source.

gjeremy1617088143 — Thu, 03 Jun 2021 06:48:30 GMT

here an example to get all the values in a loop :

for (int i = 0; i < input_row.getColumnCount(); i++)

{

DynamicMetadata columnMetadata = input_row.getColumnMetadata(i);

String inp_col=columnMetadata.getName().toString();

Object col_val=(Object)input_row.getColumnValue(inp_col);

}

Re: Dynamic Target Structure based on Data availability in source.

gjeremy1617088143 — Thu, 03 Jun 2021 07:06:31 GMT

here an example with filter on column name and dynamic output :

Dynamic out = new Dynamic();

for (int i = 0; i < input_row.newColumn.getColumnCount(); i++)

{

if((input_row.newColumn.getColumnMetadata(i).getName().toString()).equals("the column name you want"))

{

out.addColumn(input_row.newColumn.getColumnMetadata(i));

}

output_row.newColumn= out;

Re: Dynamic Target Structure based on Data availability in source.

User5923427525177642 — Thu, 11 May 2023 15:21:29 GMT

Hi gjeremy1617088143,

Thank you very much, it helped me a lot.

But good to know for others, i'm on Talend 7.3 and for me, this work only if you do like this :

out = out.addColumn(input_row.newColumn.getColumnMetadata(i));

And you have also to add values of added columns because this method add only metadata of columns, not values with it.

So I did this :

Dynamic out = new Dynamic();

for (int i = 0; i < input_row.newColumn.getColumnCount(); i++)

{

if((input_row.newColumn.getColumnMetadata(i).getName().toString()).equals("the column name you want"))

{

out = out.addColumn(input_row.newColumn.getColumnMetadata(i));

out.setColumnValue(i, input_row.newColumn.getColumnValue(i));

}

output_row.newColumn= out;