Moving the files based on file pattern

[Anonymous]

Hi Team,

I want create job below logoc.

FIRST RUN.

1) One common directory(i.e C:/Landing)

files in C:/Landing:

AAA_2017_0610_024629
AAA_2017_0611_024623
AAA_2017_0612_024622
BBB_2017_0612_024300
CCC_2017_0612_024300

2) Move the files to specific directory based on starting pattern. like AAA, BBB and CCC.

Move C:/Landing to C:/AAA
Move C:/Landing to C:/BBB
Move C:/Landing to C:/CCC

Note:
1) Move one file at time to specific directory(Like AAA) for each run.(i will secedule job every 5 mins).
2) Once file is moved to specific direcory delete the files from C:/Landing directory.
3) Move is done based old dates not latest date. ex: move files 10th , 11th and 12th date order.(NOTE: move one file at time for each run).

Second RUN.

same process, only it will move next file if mutiple file exist based on dates.

fisrt run:
move AAA_2017_0610_024629 to c:/AAA
move BBB_2017_0612_024300 to c:/BBB
move CCC_2017_0612_024300 to c:/CCC

second run:
AAA_2017_0611_024623 to C:/AAA

etc.

Kindly help on above scanario.

Thanks

Shree

[Anonymous]

Hi @bshree88,

You need to stop calling your questions "scanario". You need to give a decent description so that other people can make use of the answers you get. You have also asked for someone to tell you how to create the whole job again. If you are new to Talend, that is great. But you (or your company) need to give yourself time to learn the product, since it looks like you are trying to deliver functionality to either your company or a client by asking us to do it for you. That is not really good for you, us or Talend.

However, I have a job that kind of does most of this and I have quickly changed it to cover most of your requirements. You will need to tweak it to cover all of your requirements. This is a very simple job and simply uses the Java substring method to get the folder name and then uses a tFileCopy to copy and delete the file. The tFileList is used to monitor the folder and you can set the tFileList to do this in a particular order.....I have not done that.

You will need Talend v621+ to use this job

FileMove.zip

[Anonymous]

Hi Richard,

Yes, i will take care your inputs.

Yes, I am learning and delivering the functionality. i am trying my side as well. 

Thanks once again for direction with others points.

Regards

shridhar

[Anonymous]

Hi Richard,

I have getting error. Please find the attached file. 

Are you using any routines here.

Thanks

Shree

error.PNG

[Anonymous]

Hi Richard,

I got where is a issue, the issue with jvm and talend version. i have created same job in talend version 6.4.

it's working fine. 

Thanks

shree

[Anonymous]

Hi Richard,

As per jobs. below points are my conversations.

1) It's moving all files for single run if multiple files with same pattern. 

ex: 

AAA_2017_0610_024629
AAA_2017_0611_024623

2) In requirement, creation of destination folders is not based on patter, it could be C:/AAA or C:/test/files etc.

3) Need to move single file for single run if only if multiple file have same patter.(in order)

Kindly give me idea/suggestion. i will work around. 

Thanks for your time and support.

Regards

Shree

[Anonymous]

@bshree88 the job is set to look at the pattern at the start of the filename, create a folder situated on the C:\ drive (if the folder does not exist), and adds the file to that folder. 

Your initial question seemed to suggest that the pattern dictated the folder the file would be in? Has this changed? If you are talking about the path of the folder, you can edit the job using a context variable which can be changed quite easily. But the pattern at the front of the file must match the folder, or how do you know where the file goes?

The job will move every file in the input folder. Why would you want to limit it to one file at a time? This can be done, and would require just taking the first file returned by the tFileList (which can be set to find files in date order). An easy way of doing this is to store the files found n memory and only use the first file. There are other ways.

[Anonymous · Accepted solution]

The reason my job failed is that I have the implicit context load set for my project. There are parameters set which will not work on your machine. Just switch off the implicit context variables in your job settings.

[Anonymous]

Sure Richard.

Thanks

Shree

[Anonymous]

Hi Richard,

Please find the below are my answer.

Your initial question seemed to suggest that the pattern dictated the folder the file would be in? Has this changed? If you are talking about the path of the folder, you can edit the job using a context variable which can be changed quite easily. But the pattern at the front of the file must match the folder, or how do you know where the file goes?

Ans: Destination folder is anything. only i want to move the files based on patter to any destination folder(Not specific).

The job will move every file in the input folder. Why would you want to limit it to one file at a time?

Ans: Reason is i want to process old date file first to new date files one by one(Each run). need to run every 10 min(frequency is depends).

This can be done, and would require just taking the first file returned by the tFileList (which can be set to find files in date order).

Ans: But i want to set based file date and date is coming with file name. i want to substring the date order by date, pick old date and move to destination folder.

An easy way of doing this is to store the files found n memory and only use the first file. There are other ways.

Ans: How to achive 3rd one. storing in meory and move  single old date file to destination folder.

Thaks for your inputs.

PFA my job which created. i am facing how to move single file to destination if mutiple file exist with same pattern based on file dates.

Regards

Shree

Job_design_5.PNG
Job_design_1.PNG
Job_design_2.PNG
Job_design_3_tjava_code.PNG
Job_design_4.PNG