topic Re: Only capture latest record in tmap in Talend Studio

Only capture latest record in tmap

Anonymous — Fri, 15 Mar 2019 21:06:49 GMT

Hi,

I have a question please advise -

I am trying to output only latest record in tMap - for below example I only need 3/12/2019 record show in my output report. for now I can only catch all record or the first one. those input data are showing no sequence.

Thank you!

Trans DateTime	record
3/11/2019 6:51	A
3/12/2019 10:43	A
3/11/2019 8:08	A

Re: Only capture latest record in tmap

Anonymous — Fri, 15 Mar 2019 21:17:54 GMT

Are those three records already loaded or this particular record is getting loaded when you run your job, so you wanted to take only with the latest time stamp, can you give bit more details about it

3/12/2019 10:43

Re: Only capture latest record in tmap

DataTeam1 — Fri, 15 Mar 2019 21:51:00 GMT

@szhou1 first you need to sort records in descending order

and then filter in tMap using sequence

result:

Re: Only capture latest record in tmap

DataTeam1 — Fri, 15 Mar 2019 21:58:56 GMT

But if you want to have natural order you have to use 2x tMap

In first you have to add new column with sequence

Then you have tu use tSortRow and sort your rows in descending order of sequence column

Then you can use the second tMap in which you have add filter the same way as I show in my previous post.

Re: Only capture latest record in tmap

vapukov — Sun, 17 Mar 2019 01:25:17 GMT

Hi,

why just not use tAggregateRow

group by - record
and take MAX for DateTime

cheers

Re: Only capture latest record in tmap

DataTeam1 — Sun, 17 Mar 2019 20:48:35 GMT

@vapukov , I think @szhou1 needs last record (last row of dataset) not max date

Re: Only capture latest record in tmap

vapukov — Sun, 17 Mar 2019 22:00:35 GMT

it still will work, even without group by, just select last

Function: Select the operator among: count, min, max, avg, sum, first, last, list, list(objects), count(distinct), standard deviation.

Re: Only capture latest record in tmap

Anonymous — Mon, 18 Mar 2019 14:25:50 GMT

Thank you.
Can I know What I should fill in Numeric.sequence function? What do "S2" and "1" mean? I checked function description but I still do not know how to apply it. the one I showed was just an example. the trans date can be different in the actual source. for example it also can have 3/6 and 3/9 for the same record. and the record name is random not in sequence

Re: Only capture latest record in tmap

Anonymous — Mon, 18 Mar 2019 15:09:01 GMT

Hi,

I tried to use taggregaterow first and then link with tmap but it did not work well it showed "null point" error. I have other columns to output. I put "record" in group by both input and output. and I put "trans_time" max for the operation. In my tmap I have other columns that I need.

Re: Only capture latest record in tmap

DataTeam1 — Mon, 18 Mar 2019 15:49:48 GMT

"s2", "s1" etc is an unique name of the sequence. It can be also "blablabla";

Re: Only capture latest record in tmap

DataTeam1 — Mon, 18 Mar 2019 15:56:52 GMT

@vapukov you're right. I didn't notice that there are first and last functions. Thx

Re: Only capture latest record in tmap

DataTeam1 — Mon, 18 Mar 2019 16:00:31 GMT

@szhou1 you can do this using tAggregateRow as @vapukov mentioned.

You have to put all your columns to operation panel (not group by)