topic Re: Lock file or check if file open in Talend Studio

Lock file or check if file open

_AnonymousUser — Mon, 15 Apr 2013 05:11:38 GMT

Hi,
Are there any component that can support us lock a file or check if that file is open?
Thanks for help!

Re: Lock file or check if file open

Anonymous — Mon, 15 Apr 2013 06:37:51 GMT

Hi,

Are there any component that can support us lock a file or check if that file is open?

We don't have such component as you mentioned. As we known that Talend is a code generator ETL which use JAVA as the underline technology generated to perform the Data Extraction, Transformation and Loading.

Best regards
Sabrina

Re: Lock file or check if file open

_AnonymousUser — Mon, 15 Apr 2013 09:14:01 GMT

Hi
Thanks for reply,
I'm having problem with file locking. I must run 2 parallel subjob which will access the same file. If subjob 1 run first then subjob 2 will be pause; and if subjob 2 run first, subjob 1 must be pause.
Thus, at each subjob, I must check if the file is opened by another or not.
And because there aren't any supported component, I use tJava with the following code, but it's seem not right. Can u help me solve this problem? Thanks
Here is the code:
java.io.File file = new java.io.File(fileName);
java.nio.channels.FileChannel channel = new java.io.RandomAccessFile(file, "rw").getChannel();
// Get an exclusive lock on the whole file
java.nio.channels.FileLock lock = channel.lock();
try {
lock = channel.tryLock();
System.out.println("lock success");
} catch (java.nio.channels.OverlappingFileLockException e) {
System.out.println("pause");
} finally {
lock.release();
}

Re: Lock file or check if file open

Anonymous — Mon, 15 Apr 2013 11:09:55 GMT

Hi
You can try this job design without adding any Java code to check the file status.
Parent job:
tRunJob_1
tRunJob_2
tRunJob_1: call a child job that run the processing1, uncheck the 'die on error' option on tRunJob.
tRunJob_1: call a child job that run the processing2, uncheck the 'die on error' option on tRunJob.
Open the Job view of parent job, click Extra tab, and check the 'Mutiple thread execution' option.

Shong

Re: Lock file or check if file open

alevy — Tue, 16 Apr 2013 00:10:44 GMT

Use tWaitForFile with the options "Include present file" and "Wait for file to be released" at the start of each subjob.

Re: Lock file or check if file open

_AnonymousUser — Wed, 23 Oct 2013 12:07:14 GMT

Hi,
The tWaitForFile is not working for ftp transfer. I have two job. One is getting a file from remote server using tFTPGet.
Other reading the file which is read from remote server. Both job running parallely.

Job 1 : tFTPConnection---->tFTPFileList---->tFTPGet

Job 2: tWaitForFile----on subjob ok----> tFileList--- >tFileInputDelimited--->tMap--->tLogRow.
Problem here, second job is not waiting ftp to complete. I have used "Include present file" and "Wait for file to be released" option in tWaitForFile and checked but still its not working.
As i know there is no file lock component in Talend, how to resolve this problem. Anybody know how to solve it. Please let me know the solution.
Regards,
Akki

Re: Lock file or check if file open

_AnonymousUser — Thu, 24 Oct 2013 07:31:40 GMT

Hi,
I forgot to mention. I'm using talend TOS 5.2.2 in linux machine. Whether tWaitForFile support only in windows??
Regards,
Akshath Hegde

Re: Lock file or check if file open

_AnonymousUser — Thu, 24 Oct 2013 12:35:19 GMT

Hi,
I want to know whther there is any component for this scenario in TIS?? if yes please suggest me the component and its use. And please reply for above questions. i hope Most of people are expecting the answer for these questions.
Regards,
Akki

Re: Lock file or check if file open

Anonymous — Fri, 25 Oct 2013 03:44:34 GMT

Hi
There is a job design problem in your job, you should use iterate instead of onSubjobOk link from tWaitForFile, eg:
tWaitForFile----iterate---> tFileList--- >tFileInputDelimited--->tMap--->tLogRow.
If you select 'continue loop' otion in the Then list, tWaitForFile keeps working forever without end.
Shong

Re: Lock file or check if file open

_AnonymousUser — Fri, 25 Oct 2013 06:02:35 GMT

Hi shong,
Thank you for reply shong. i have checked with iterate in tWaitForFile. But it will not wait file transfer to complete.
"tWaitForFile should wait file transfer to complete, after complete transfer only it should take transferred file".
Please help to resolve this problem. If you reply quick it will be very helpful for us.

Regards,
Akki

Re: Lock file or check if file open

_AnonymousUser — Fri, 25 Oct 2013 06:13:14 GMT

Hi Shong,
Even i have tried with iterate in tWaitForFile, But tWaitForFile will not wait file transfer to complete. Please suggest a appropriate solution for this.
Thanks and Regards,
Mithun

Re: Lock file or check if file open

Anonymous — Fri, 25 Oct 2013 07:37:06 GMT

Both job running parallely.

Why do you run the two jobs parallely? Run the job 2 first, tWaitForFile will monitor the specified directory and trigger the next action once a new file is created, and then run any other jobs.
Shong

Re: Lock file or check if file open

_AnonymousUser — Fri, 25 Oct 2013 07:47:34 GMT

Hi shong,
Thanks for reply shong. That's our requirement. In our project, file will be transferring from remote server. So other job should wait completion of file transfer.
"Is this possible?? there is any way to achieve this". Please suggest any possibilities for this scenario.
Regards,
Akki

Re: Lock file or check if file open

_AnonymousUser — Fri, 25 Oct 2013 07:57:32 GMT

Hi,
As u have mentioned tWaitForFile will trigger on next action, But i want other job to wait until FTP completes its transmission.
"For ex: When one process is writing to file other process should not be able to read that file. "
Regards,
Mithun

Re: Lock file or check if file open

_AnonymousUser — Fri, 25 Oct 2013 10:02:22 GMT

Hi,
Please advice me because i am running out of time.
Regards,
Akki

Re: Lock file or check if file open

_AnonymousUser — Mon, 28 Oct 2013 05:32:24 GMT

Hi,
Can Anybody suggest answer please...
Thanks and regards,
Mithun

Re: Lock file or check if file open

Anonymous — Mon, 28 Oct 2013 05:55:48 GMT

Hi Mithun
Job1 and Job2 are two separated jobs in your case, right? As I suggested, run Job2 (contains tWaitForFile component) first to make tWaitForFile keep monitor the specified folder and trigger next action once a new file arrives.
If I don't understand your request well, can you please export the jobs and email them to me?
Shong

Re: Lock file or check if file open

_AnonymousUser — Mon, 28 Oct 2013 14:10:18 GMT

Hi shong,
I have sent the mail to you. Please find the attachment in it.
Regards,
Akshath

Re: Lock file or check if file open

_AnonymousUser — Wed, 30 Oct 2013 08:54:12 GMT

Hi,
It's very sad that many of questions are still unanswered in talend forge!!! It is not good for talend as well as for the us (talend user). People will be tired of waiting for answer. I have been expecting answer for AKSHATH question but there is no result here. I think it better, not to ask quetion's from talend forge. If there is no answer then why we have to waste our time in posting questions.

-----------------
From,
Talend User

Re: Lock file or check if file open

Anonymous — Wed, 30 Oct 2013 09:05:36 GMT

Hi Akshath
I have received your email and your job, but we have so many topics/questions to read and answer each day, please notify your reply to me if I miss your topic next time. I will try to take a look at your job and reply you today.
Shong