extract the latest row from a file

jackd9999 — Sat, 16 Nov 2024 01:14:04 GMT

I have a file with an email address, a timestamp and a series of values on each line

I only want the line with the latest timestamp for the email address

I have set up a tFileInputDelimited component for the file bit I've hit a snag

i) i tried using tSortRow which sorts by email address and timestamp. This works but I don't really know what component goes next in the pipeline. tUniqRow doesn't seem to allow me to say I only want the latest timestamp

ii) i tried using tAggregateRow but I'm not sure how this relates to aggregation windows : I only want the latest row (by timestamp) for the email address, not the file. This may be "first" for the timestamp itself but I don;t see how I extract the other values - these I suppose should be "first" but only in relation to timestamp.

i guess i need the equivalent of sql windows partition. Any ideas ?

jack

Re: extract the latest row from a file

Anonymous — Mon, 16 Nov 2020 03:02:02 GMT

The 1st way is right, use tSortRow to sort the rows by timestamp (desc model), and then link tSortRow to a tMap where add a sequence ID to each row, filter the row which has sequence id==1, this is the latest row.

Please try and let me know if you have any issues/questions.

Regards

Shong

topic Re: extract the latest row from a file in Talend Studio

extract the latest row from a file

Re: extract the latest row from a file