topic Re: tWaitForFile - Trigger email when no-file or wrong-file in Talend Studio

tWaitForFile - Trigger email when no-file or wrong-file

Anonymous — Sat, 16 Nov 2024 04:57:44 GMT

Hello Champs,

Need your help for my issue.

My requirement:

I expect “*.csv” as source file. Once the file is in folder the job should transform into required format and drop it to output folder.

My job…

tFileList --> (iterate) --> tFileInputDelim --> (main) --> tMap --> (out) --> tFileOutputDelim

it works great.

My issues:

Currently the job runs every 1 hour. Instead, client asked me to make it to run automatically and process it whenever the source file exist in directory.

I am used “tWaitForFile” to achieve above. I made the job to run first time, and then it picks whenever there is a file.

tWaitForFile --> (iterate) --> tFileList --> (iterate) --> tFileInputDelim --> (main) --> tMap --> (out) --> tFileOutputDelim

Quest:

Is it mandatory to use tSleep?
Will it impact system (cloud) performance if we use ‘tWaitForFile’? As the component always active.

Client asked me trigger an email when “the file format is wrong” (ie., .xlsx or .txt) or “when there is no file”.

Quest:

How do I achieve? “tSendEmail” works fine if I joined to “tFileList” (without using ‘tWaitForFile’ in my flow) , but doesn’t work from “tWaitForFile”.
If I use “tWaitForFile”, if the source folder is empty, it doesn’t trigger any email as the component still waiting for the file.

Thanks heaps for your help. Happy to answer all your queries.

Regards,

Re: tWaitForFile - Trigger email when no-file or wrong-file

Anonymous — Wed, 21 Aug 2019 08:47:09 GMT

Hello,

When there is no file, you could use tSetGlobalVar to set a variable (e.g. "FileProcessed") to false. Then OnSubjobOK to tWaitForFile with the appropriate time between iterations and max number of iterations. Connect tWaitForFile with iterate to another tSetGlobalVar to set the "FileProcessed" variable to true. Then have RunIf from tWaitForFile to tSendMail with the condition (Boolean)globalMap.get("FileProcessed")==false.

Let us know if it helps.

Best regards

Sabrina

Re: tWaitForFile - Trigger email when no-file or wrong-file

Anonymous — Wed, 02 Oct 2019 21:59:32 GMT

Hi xdshi,

Sorry for the delay. Your solution works fine, but it didn't fulfill my requirement (screenshot attached fyr)

1) My job should process only CSV file. If any other formats in the folder, then it should trigger an alert email.

2) (or) if "no file" between 8am-10am then it should trigger an email, saying that "no file found".

I tried point 1 in your solution. Dropped a non-csv file to the folder, but nothing happened as 'twaitforfile' is waiting for CSV file.

Does it make sense?

regards,

Kiruba

twaitforfile_example.JPG

Re: tWaitForFile - Trigger email when no-file or wrong-file

Anonymous — Fri, 04 Oct 2019 16:52:58 GMT

@rvkiruba

Could you please change the file mask to accept all different types of files? Then once the file is identified, you can check the extension of file by storing the file name to a context variable and parse the extension of the file.

Please use a Run if condition to send the data flow to different paths. If the file is csv, you can follow existing path. If the file is not csv, follow different path where you can send the error email.

Coming to second query, you can do the whole process in a tLoop based on a while condition where the time is between start and end time. You can check the file and go to sleep for predefined interval (say 10 minutes). If you receive a file, set a context flag variable as Y. Even after 10 am, if the file has not reached, the flow will come out of loop and you can check whether flag is Y or N. If the flag is still N, you can send a error email through another Run if flow.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

Re: tWaitForFile - Trigger email when no-file or wrong-file

Anonymous — Fri, 04 Oct 2019 19:25:09 GMT

Thanks Nikhil. I will give it a try and get back to you asap.

Regards,
Kiruba