topic Re: Moving files within the bucket after deleting source files in Talend Studio

Moving files within the bucket after deleting source files

Anonymous — Tue, 26 Dec 2017 04:04:02 GMT

scenario : moving the files to destination folder (s3 within the bucket) then removing the source files

job flow

ts3 connection

ts3 list

bucket name :- "bucket_src"

Key prefix :- "Main/Source/"

ts3 copy

source configuration bucket :- "bucket_src"

Key :- ((String)globalMap.get("tS3List_1_CURRENT_KEY"))

Destination configuration bucket :- "bucket_src"

Key :- "Main/Dest/" + ((String)globalMap.get("tS3List_1_CURRENT_KEY")).substring( ((String)globalMap.get("tS3List_1_CURRENT_KEY")).lastIndexOf("/") + 1 )

ts3 Delete

Bucket :- "bucket_src"

Key :- ((String)globalMap.get("tS3List_1_CURRENT_KEY"))

(Deleting entire source folder )

my requirement I want to delete files after loading to destination folder

Please suggest to deleting only files.

Thanks

MRA

Re: Moving files within the bucket after deleting source files

Anonymous — Tue, 26 Dec 2017 08:56:04 GMT

Hello
The key includes the folder path, you need to exclude the the folder path and only iterate the files.
eg:
tS3list--iterate--tjava--runIf--tS3Copy--oncomponent--tS3Delete

on tjava:
String key = ((String)globalMap.get("tS3List_1_CURRENT_KEY"));
int length=key.length();

if(length>12){
globalMap.put("isContinue",true);
}else{
globalMap.put("isContinue",false);
}

Set the condition of runIf link as:
(Boolean)globalMap.get("isContinue")

Hope it helps you!

Regards
Shong

Re: Moving files within the bucket after deleting source files

Anonymous — Tue, 26 Dec 2017 10:14:34 GMT

The Name "(Boolean)globalMap.get("isContinue")" is not valid or already exists.

Re: Moving files within the bucket after deleting source files

Anonymous — Tue, 26 Dec 2017 10:16:36 GMT

Run if condition error

The Name "(Boolean)globalMap.get("isContinue")" is not valid or already exists.

Re: Moving files within the bucket after deleting source files

Anonymous — Tue, 26 Dec 2017 14:46:11 GMT

No double quotes enclose the expression, it is (Boolean)globalMap.get("isContinue").

Re: Moving files within the bucket after deleting source files

Anonymous — Wed, 27 Dec 2017 06:54:37 GMT

Hi Shong,

even without double quotation also getting same error

Thanks,

MRA

Re: Moving files within the bucket after deleting source files

Anonymous — Wed, 27 Dec 2017 07:06:35 GMT

Can you please capture a screenshot to show your job design and where you write the expression?

Re: Moving files within the bucket after deleting source files

Anonymous — Sun, 31 Dec 2017 06:51:37 GMT

Hi Shong,

thank a lot for quick replay

Thanks,

MRA

This is the job flow. I achived

EX:-job flow
ts3 connection---ts3 list----ts3 copy
|----on subjob ok
|
ts3 list----ts3 Delete

ts3 connection

ts3 list

bucket name :- "bucket_src"

Key prefix :- "Main/Source/"

ts3 copy

source configuration bucket :- "bucket_src"

Key :- ((String)globalMap.get("tS3List_1_CURRENT_KEY"))

Destination configuration bucket :- "bucket_src"

Key :- "Main/Dest/" + ((String)globalMap.get("tS3List_1_CURRENT_KEY")).substring( ((String)globalMap.get("tS3List_1_CURRENT_KEY")).lastIndexOf("/") + 1 )

ts3 list

bucket name :- "bucket_src"

Key prefix(multiple key prefix) :- "Main/Source/output" ('output' is the file name which we are getting in source directory)
"Main/Source/temp" ('temp' is the file name which we are getting in source directory)

ts3 Delete

Bucket :- "bucket_src"

Key :- ((String)globalMap.get("tS3List_1_CURRENT_KEY"))