topic Re: Collect a fixed amount of rows in Talend Studio

Collect a fixed amount of rows

BooWseR — Tue, 03 Sep 2019 20:01:32 GMT

hi guys, i have a question again:

I have a certain number of rows and would like to put them into packages of the same size. Let's say I have 500 rows and always want to collect and summarize 20.

For example:

Row1

Row2

Row3

...

Row21

Row22

...

Should be:

Row1|Row2|Row3|...

Row21|Row22|... and so on

I've been looking for a solution for several hours now, but I haven't found any. I hope you can help me.

Best regards,

BooWseR

Re: Collect a fixed amount of rows

fdenis — Wed, 04 Sep 2019 08:45:23 GMT

hi, you have 2 questions.
first is to group by n rows.
you can use tmap creating seqence and using modulo to get a unique index by n lines Numeric.sequence("seq1",0,1)%n+1 -- id
then use taggregate to group lines in list.

an other solution is to manualy concaténate fields into tmap variable
var.var1 = var.var1 + row1.value
then use taggregate to keep only the last line.

good luck

Re: Collect a fixed amount of rows

BooWseR — Wed, 04 Sep 2019 17:19:57 GMT

Hi Francois, thank you for your answer.

In the meantime, I have been able to find my own solution, which also works quite well. Let me show you:

First I get the data from a REST interface. Then I extract the data I need from the JSON, split it with "tNormalize" and write it into an ArryList with tJavaFlex.

Then I iterate with tLoop in predefined steps (10 each) up to the maximum of the ArrayList.

I'm pretty sure I could have used the output from the JSON as a list somehow, but I couldn't get it right. I'm sure that your solution will certainly perform better, but currently mine works. When I find the time, I'll try your solution. Anyway, thanks for your help.

Best regards,

BooWseR

Re: Collect a fixed amount of rows

fdenis — Mon, 09 Sep 2019 07:33:21 GMT

hi mark your topic as solved,
just think about maintenance, less java code easier is the understanding of a talend job.

Re: Collect a fixed amount of rows

rmartin2 — Mon, 09 Sep 2019 08:27:10 GMT

Hi !

Wow, so much brain functioning for a simple thing ^^

Use tSampleRow and you should be ok