topic Re: how to validate the input file in Talend Studio

how to validate the input file

_AnonymousUser — Sat, 03 Aug 2013 19:21:20 GMT

Hi,
Is there a way to validate the input file according to some conditions before the process ?
For example, if the input file's record number in Footer (one column names records_total) is 100, but the actual input rows are 95. Then I will reject the file and log into the rejected table. Otherwise, I will go to normal process.
My first though was tJava, but it is not that dorable.
Thanks !
James

Re: how to validate the input file

alevy — Sun, 04 Aug 2013 23:57:12 GMT

Not automatically. You'd have to read the last row and count the number of lines and compare them yourself.

Re: how to validate the input file

Anonymous — Mon, 05 Aug 2013 02:44:20 GMT

Thanks for your reply.
Then, is there a way to read the last line (or particular line) ?
Thanks !
James

Re: how to validate the input file

alevy — Mon, 05 Aug 2013 13:28:18 GMT

To read a particular line, you would set the header to be the number of rows before and the limit to 1.
Reading the last line is not so easy. I think you would have to count the rows in the file and use that (minus 1) as a variable for the header of your input component.

Re: how to validate the input file

_AnonymousUser — Mon, 05 Aug 2013 18:49:40 GMT

Thank you for the reply.
I am think below process,
tFileList
--> tFileInputPositional { if I set the header to limit = 1, I only get one record, but if file is valid, I will process all records} --> tJava{1. how to get the previous line as method input parameter ?, 2. how to output the method result of "if... else if ...."} --> tmap {1. how to get the tJava "if ...else..." result here and map the previous input file } ---> table1, ---> reject records, ----> reject files
Is it doable ? If yes, could you help to answer questions above?

Thanks !
James

Re: how to validate the input file

Anonymous — Mon, 05 Aug 2013 19:33:38 GMT

Here is the picture for previous process.

Re: how to validate the input file

alevy — Tue, 06 Aug 2013 00:32:36 GMT

Header and Limit are two different parameters.
Use tFileRowCount to count the number of rows in the file, then use the globalMap variable it creates in tFileInputPositional to read just the last line and pass it to tJavaRow. You can then have RunIf triggers from tJavaRow to your processing if the file passes validation or if the file fails validation.
tFileList -iterate-> tFileRowCount -iterate-> tFileInput -row-> tJavaRow -if-> tFileInput -row-> tMap -row-> ... (valid file)
(1 row) (full file)
-if-> tSendMail (or whatever for invalid file)

Re: how to validate the input file

Anonymous — Tue, 06 Aug 2013 15:12:25 GMT

Thanks for your message.
But I have a detail question about "-if" for tJavaRow -if-> tFileInput -row->
In tJavaRow, I set a Boolean variable "valid", how can I access this varialbe's value in "if" condition ?
In old posts, you once mentioned the RUN IF can only access the global variable. Do I need a tSetGlobalVar between tJavaRow -if-> tFileInput -row-> ?
James

Re: how to validate the input file

Anonymous — Tue, 06 Aug 2013 15:27:25 GMT

OK, I got it.
I can set the global variable in jJavaRow.
thanks !
James

Re: how to validate the input file

alevy — Tue, 06 Aug 2013 23:51:49 GMT

You don't actually need to set a global variable in tJavaRow and reference that in the RunIf; you can just put the same boolean expression you're using in tJavaRow directly in the RunIf (although you have to use the real row names instead of input_row).

Re: how to validate the input file

Anonymous — Wed, 07 Aug 2013 13:26:15 GMT

Yes, it works better. Thanks a lot !
James