topic Re: Data validation - Empty file Handling in Data Quality

Data validation - Empty file Handling

RA6 — Wed, 27 Sep 2017 14:13:54 GMT

Hello,

I want to know how to implement some validation before processing a file.

In fact, i have a file (csv) have contain header and i want to check if the file is empty(excluding the header) then process to tmap else die.

Can you advice please?

Thank you.

Best regards,

asadasing

Re: Data validation - Empty file Handling

Anonymous — Thu, 28 Sep 2017 06:52:39 GMT

Hello,

In Talend, you can use tFileProperties to get the file properties, there is a field called size that populate the file size, if it is 0, means the file is empty. According to your description, a job design for example looks like:
tFileFlist--iterate--tFileProperties--tJavaRow--runIf_1-->normal processing flow
--runIf_2--->another processing if the file is empty
on tJavaRow, check the file if it is empty and put the result to a global variable, for example:

if(input_row.size==0){ globalMap.put("isEmpty", true); }else{ globalMap.put("isEmpty", false); }

set the condition of runIf_1 as:
!(Boolean)globalMap.get("isEmpty")

set the condition of runIf_2 as:
(Boolean)globalMap.get("isEmpty")

Let us know if it is OK with you.

Best regards

Sabrina

Re: Data validation - Empty file Handling

RA6 — Fri, 29 Sep 2017 13:15:47 GMT

Hello,

Thank you for your response, in tfileproperties, it does the size calculation in bytes.
Hence, i cannot check if file = 0 ; I have tried to check the flow with an empty file and it returns 2 and now my empty contain header, i guess the value will change.

Can you advice please on how i can proceed?

Re: Data validation - Empty file Handling

Anonymous — Thu, 05 Oct 2017 18:24:40 GMT

@asadasing i think you can use tFileRowCount component.