topic Re: S3 bucket naming from talend in Talend Studio

S3 bucket naming from talend

Anonymous — Fri, 23 Nov 2018 11:12:55 GMT

Hi Everyone,

I am using ts3list to get all the files from the s3 bucket.

There are 2 fields : Bucket & Key

i have 2 buckets abc and xyz. The xyz bucket is inside abc.

In Bucket portion-I need to give 2 buckets.

If i give In bucket name Like "abc",It works

but now i need to give "abc/xyz" as bucket name but it is not working.

anyone know any solution for this?

Regards,

Mohit

Re: S3 bucket naming from talend

vapukov — Fri, 23 Nov 2018 11:52:22 GMT

are you sure it is bucket inside the bucket? as I remember - bucket contain folders or files

maybe it is just folder?

as I remember folder could be part of Key prefix

Re: S3 bucket naming from talend

Anonymous — Fri, 23 Nov 2018 12:19:37 GMT

Hi Vapukov,

Thanks,I have tried as a Folder name as you said:

Bucket Name="xyz1234"

Folder Name= abcd_17678xxxxx/

In ts3List i have used

bucket as: "xyz1234"

Key Prefix as:

"abcd_17678xxxxx/"+((String)globalMap.get("tS3List_1_CURRENT_KEY"))

In S3Get i used

Bucket as: "xyz1234"

Key as (String)globalMap.get("bucket"))+"/"+"abcd_17678xxxxx/"+"/"((String)globalMap.get("key"))

where folder name= abcd_17678xxxxx/

Let me know if this approach is correct because i am getting error while giving folder name into Key prefix.

Regards,

Mohit

Re: S3 bucket naming from talend

vapukov — Fri, 23 Nov 2018 12:23:39 GMT

why you use

Key Prefix as:

"abcd_17678xxxxx/"+((String)globalMap.get("tS3List_1_CURRENT_KEY"))?

as I understand - it is tS3List_1 and You try use current filename inside the same component

it always better attach all picture with proper configuration rather then try to guess 🙂

Re: S3 bucket naming from talend

Anonymous — Fri, 23 Nov 2018 13:14:32 GMT

Hi Vapukov,

I used "abcd_17678xxxxx/"+((String)globalMap.get("tS3List_1_CURRENT_KEY"))? in key prefix because it will get folder name first then key items inside that folder.

I attached a screenshot.

Let me know if any solution.

Regards,

Mohit

error_1.docx

Re: S3 bucket naming from talend

vapukov — Fri, 23 Nov 2018 13:19:13 GMT

when you run tS3FileList - YOU DO NOT KNOW filenames 🙂

fot tS3FileList you need only prefix - "abcd_17678xxxxx/"

(foldername)

after - component will return 1 filename for 1 iteration

Re: S3 bucket naming from talend

Anonymous — Fri, 23 Nov 2018 13:34:15 GMT

I am not sure what is happening.It is giving me Null,Null,Null............... inside the folder.

Only folder name i am getting.

I used only folder name in key prefix.

Regards,

Mohit

Re: S3 bucket naming from talend

vapukov — Fri, 23 Nov 2018 13:37:42 GMT

🙂

you are mask half of information, so I can not check - what is wrong what is proper

but it work in my jobs - I delete files in subfolders on bucket (as I remember - job installed on TAC and not opened last year)

Re: S3 bucket naming from talend

Anonymous — Fri, 23 Nov 2018 13:58:04 GMT

It Worked.

Thank You so Much

Regards,

Mohit