topic Re: [resolved] How to get the last 10 lines from file using tsamplerow? in Talend Studio

[resolved] How to get the last 10 lines from file using tsamplerow?

orawajar — Fri, 06 Feb 2015 03:36:25 GMT

Hi Master,
I use tfileinputDelimited to read a file and then need to get the last 10 lines from this file.
i try to use tSamplerow but it can not read NB_LINE from tFileInputDelimited when using Main Connection.
i try to save it first to context but still return null when using Main connection.
Do you have any idea?
Thank you and best regards.

Re: [resolved] How to get the last 10 lines from file using tsamplerow?

Anonymous — Fri, 06 Feb 2015 03:58:34 GMT

Add a sequence id to each row on tMap, and then sort the rows with desc model on tSortRow, define a var to generate the sequence id on another tMap, filter the first 10 lines using the sequence id.

Re: [resolved] How to get the last 10 lines from file using tsamplerow?

orawajar — Fri, 06 Feb 2015 04:18:51 GMT

Hi shong thank you for your fast response,
Nice idea but a little bit inefficient i think because sorting large data will take much time.
It will be great if i can use NB_LINE that already provided by tFileInputDelimited. Do you have any idea that i can do, maybe doing something with tSampleRow, so it can read the NB_LINE?
Thank you and best regards,

Re: [resolved] How to get the last 10 lines from file using tsamplerow?

Anonymous — Fri, 06 Feb 2015 04:42:47 GMT

Hi
The global var NB_LINE can be only used on another subjob, you need to read the source file two times (or one time if you want to store the data into memory). For example:
tFileInputDelimited_1--main--tJava_1
|
onsubjobok
|
tJava_2
|
onsubjobok
|
tFileInputDelimited_2--main--tSampleRow--main--tLogRow
or
tFileInputDelimite--main--tHashOutput
|
onsubjobok
|
tJava_2
|
onsubjobok
|
tHashInput--main--tSampleRow--main--tLogRow
on tJava_2: get the NB_line, define the range expression for used later on tSampleRow

int nb_line=((Integer)globalMap.get("tFileInputDelimited_1_NB_LINE"));
int begin=nb_line-9;
context.range=begin+".."+nb_line;

//Define a context var called range in the job.
on tSampleRow, set the range expression with context var.

context.range

Re: [resolved] How to get the last 10 lines from file using tsamplerow?

orawajar — Fri, 06 Feb 2015 06:51:22 GMT

Okay shong. i prefer your second solution..
it work for me.
Thank you shong.

Re: [resolved] How to get the last 10 lines from file using tsamplerow?

Anonymous — Thu, 13 Feb 2020 08:04:33 GMT

Hi,

You can try this one.

Job SnapshottJavatFileInputDelimied