topic Re: Check if another file exists - File name substring and append in Talend Studio

Check if another file exists - File name substring and append

Anonymous — Sat, 16 Nov 2024 12:51:51 GMT

Hi,
I have a set of files shown like below.
DAILY-USER-20110531-USERLOGON.xml
DAILY-USER-20110531-USERLOGOFF.xml
DAILY-USER-20110531-CONTROL.xml
DAILY-USER-20110530-USERLOGON.xml
DAILY-USER-20110530-CONTROL.xml
I have a tFileList component with the source folder which is holding above files. Now for files ending with USERLOGON.xml, USERLOGOFF.xml, I need to check if its corresponding CONTROL.xml file exists with the initial part of the file name as same. As you can see there is a date 20110530, 20110531 which keeps changing. Now I wanted to try out the following ways..
1. Get the file name string in a loop and substring it till the date (constant number of characters in this case 20) and append 'CONTROL.xml' to the string and pass it on to a tFileExist component.
2. If the CONTROL.xml file exists then proceed with next operation on the original file processing.
But I am not able to get the context variables or some other way to get this done.. I am new to Talend... can any body please help with possible easier ways. My Talend code generated is Java. Thanks in advance.

Re: Check if another file exists - File name substring and append

Anonymous — Wed, 15 Jun 2011 15:34:17 GMT

you have the right idea.
use a tFileList to retrieve the list of files. in a tJava, retrieve the file names from the global map and prepare your context variable-- then use it to check if your control file exists.
so
tFileList--iterate-->tJava--onComponentOK-->tFileExist

Re: Check if another file exists - File name substring and append

Anonymous — Thu, 16 Jun 2011 07:03:21 GMT

with tfilelist, the tfileexists doesnt seem to work when there is no file john.

Re: Check if another file exists - File name substring and append

Anonymous — Thu, 16 Jun 2011 10:34:45 GMT

Is there a sample Java code, which can be used for the above purpose. My doubt is how can I say componentOK and pass the actual file name to next module after checking if corresponding CONTROL.xml exists in tJava?

Re: Check if another file exists - File name substring and append

Anonymous — Thu, 16 Jun 2011 14:53:03 GMT

If I understand correctly there are two cases here:
1) some files exist, but no control file
2) all files exist.
so, using this job template:
tFileList--iterate-->tJava--onComponentOK-->tFileExist----OnSubjobOK--->(rest of job)
in the tJava:

String fileName = globalMap.get("tFileList_1_CURRENT_FILEPATH");
<do something with fileName>
context.control_file = fileName;

in the tFileExist, check for the existence of:

context.control_file

in the rest of your job, you can refer to the current file of the tFileList with:

globalMap.get("tFileList_1_CURRENT_FILEPATH")

Re: Check if another file exists - File name substring and append

Anonymous — Fri, 17 Jun 2011 08:02:11 GMT

Thank you very much. But I am now facing error with context.control_file.
My tJava code is:
context.control_file = substring((String)globalMap.get("tFileList_1_CURRENT_FILEPATH"),26);
This is giving error as 'The method substring(string, int) is undefined for the type <jobname>.
In tFileExist_1 -> I gave the file path as context.control_file in file name/stream.
This is giving error as ' control_file cannot be resolved or is not a field'.
What am I missing?

Re: Check if another file exists - File name substring and append

Anonymous — Fri, 17 Jun 2011 13:31:27 GMT

1) context.control_file = ((String)globalMap.get("tFileList_1_CURRENT_FILEPATH")).substring(26);
2) you need to add control_file as a context variable in the context tab.

Re: Check if another file exists - File name substring and append

Anonymous — Fri, 17 Jun 2011 15:57:19 GMT

Thank you very much John for patiently replying to all my questions. It seems to be working, but still there is one problem. Even if there is no corresponding CONTROL.xml file, it is copying all the files. I think it should be the problem with iteration.
My tJava code:
context.control_file = ((String)globalMap.get("tFileList_1_CURRENT_FILEPATH")).substring(0,69)+"CONTROL.xml";
System.out.println( ((String)globalMap.get("tFileList_1_CURRENT_FILE")));
System.out.println( context.control_file);
My tFileExist file path:
context.control_file
I set a context variable. Please find the execution screenshot below.
For me it should not copy files without corresponding CONTROL.xml, but irrespective of result of before steps, it is copying all files.

Re: Check if another file exists - File name substring and append

Anonymous — Sat, 18 Jun 2011 15:59:27 GMT

instead of 'onComponentOK' you need to use an 'if' connector. click on the connector and in the 'if' component tab, you put the condition you want to check:
(Boolean)globalMap.get("tFileExists_EXISTS")

check the generated code for the tFileExists component to make sure i'm giving you the correct key to the globalMap.

Re: Check if another file exists - File name substring and append

Anonymous — Sun, 19 Jun 2011 08:25:30 GMT

Thank you very much John!!!
That worked with ((Boolean)globalMap.get("tFileExist_1_EXISTS")) in If condition. This really helped me understand the flow. Thank you very much for your answers and patience. Where can I get these detailed set of components and their checks or example projects. I have gone through the tutorials provided with talend, but this in depth practical details are not available i guess. Thanks again.