[resolved] How to get the last 10 lines from file using tsamplerow?

[orawajar]

Hi Master,
I use tfileinputDelimited to read a file and then need to get the last 10 lines from this file.
i try to use tSamplerow but it can not read NB_LINE from tFileInputDelimited when using Main Connection.
i try to save it first to context but still return null when using Main connection.
Do you have any idea?
Thank you and best regards.

[Anonymous]

Add a sequence id to each row on tMap, and then sort the rows with desc model on tSortRow, define a var to generate the sequence id on another tMap, filter the first 10 lines using the sequence id.

[orawajar]

Hi shong thank you for your fast response,
Nice idea but a little bit inefficient i think because sorting large data will take much time.
It will be great if i can use NB_LINE that already provided by tFileInputDelimited.  Do you have any idea that i can do, maybe doing something with tSampleRow, so it can read the NB_LINE?
Thank you and best regards,

[Anonymous · Accepted solution]

Hi 
The global var NB_LINE can be only used on another subjob, you need to read the source file two times (or one time if you want to store the data into memory). For example:
tFileInputDelimited_1--main--tJava_1
   |
onsubjobok
   |
tJava_2
   |
onsubjobok
   |
tFileInputDelimited_2--main--tSampleRow--main--tLogRow
or 
tFileInputDelimite--main--tHashOutput
   |
onsubjobok
   |
tJava_2
   |
onsubjobok
   |
tHashInput--main--tSampleRow--main--tLogRow
on tJava_2: get the NB_line, define the range expression for used later on tSampleRow

int nb_line=((Integer)globalMap.get("tFileInputDelimited_1_NB_LINE"));
int begin=nb_line-9;
context.range=begin+".."+nb_line;

//Define a context var called range in the job.
on tSampleRow, set the range expression with context var.

context.range

[orawajar]

Okay shong. i prefer your second solution..
it work for me.
Thank you shong.

[Anonymous]

Hi,

You can try this one.

Job SnapshottJavatFileInputDelimied