topic Re: Duplicate records based on the input number in Talend Studio

Duplicate records based on the input number

lmit — Tue, 15 Nov 2022 08:47:16 GMT

Hi All,

I have a requirement like below

col1;col2;col3

1;a;2

2;b;3

3;c;1

the records should be generated based on the col3 , if like col3 have 2 then the records should be 2 my output should like below

col1;col2;col3

1;a;1

1;a;2

2;b;1

2;b;2

2;b;3

3;c;1

any suggestions would be really appreciated

thanks,

lmit

Re: Duplicate records based on the input number

Anonymous — Tue, 15 Nov 2022 10:33:50 GMT

Hello,

You could use tLoop to output the same line several times base on the 'col3' field. The job looks like:

Input--tFlowToIterate--iterate-->tLoop--iterate-->tFixedFlowInput--main--tLogRow

on tLoop, set the To parameter as: (Integer)globalMap.get("row1.col3")

Please have a look at my screenshots for details.

Best regards

Sabrina

Re: Duplicate records based on the input number

lmit — Tue, 15 Nov 2022 18:55:12 GMT

Hi @Xiaodi Shi ,

thanks for the suggestion and it worked but when am storing the data in file only the last record is getting saved

can you suggest what I can do ? Should I use the append option in the excel output

thanks,

lmit

Re: Duplicate records based on the input number

lmit — Wed, 16 Nov 2022 02:59:42 GMT

Hi All,

any suggestions would really appreciate

thanks in advance,

lmit

Re: Duplicate records based on the input number

Anonymous — Wed, 16 Nov 2022 03:40:02 GMT

Hello,

Please check out "Append" option in the excel output component.

Note: Due to "Append" is written to cache, so if your inserted data is very huge, the job could get out Of memory exception.

Best regards

Sabrina

Re: Duplicate records based on the input number

lmit — Wed, 16 Nov 2022 04:49:31 GMT

Thanks @Xiaodi Shi

yes append is giving out of memory error as I have huge data

any other alternative??

thanks,

lmit

Re: Duplicate records based on the input number

Anonymous — Wed, 16 Nov 2022 09:44:42 GMT

Hello,

Is it getting loaded successfully when you try to append with less data using tfileoutputexcel component?

What the total number of records are you trying to load?

Best regards

Sabrina

Re: Duplicate records based on the input number

lmit — Wed, 16 Nov 2022 11:12:29 GMT

I have more than 1 lakh of records

no for less data (10 records)also am not getting @Xiaodi Shi

thanks,

lmit

Re: Duplicate records based on the input number

Anonymous — Thu, 17 Nov 2022 02:47:19 GMT

Hello

Could you please try to allocate more memory to the job execution to see if it helps?

Here is a KB article about: OutOfMemory Exception

Best regards

Sabrina