topic Re: AWS tS3Put and tS3List in Talend Studio

AWS tS3Put and tS3List

Anonymous — Sat, 16 Nov 2024 03:13:38 GMT

Hi there,

I need to upload files and get the URL of files in SE Bucket. I am able to upload the files but it is triggering S3List component every time it upload file. I want it to be done once the S3Put component task completed.

Senario

First I will upload the multiple files to SE bucket and only once all the files uploaded it will move to next component tS3List and provide me all the URL for files.

Re: AWS tS3Put and tS3List

Anonymous — Wed, 19 Feb 2020 02:42:50 GMT

URL is the file path or the web URL you want? What is the trouble/problem you meet now?

Re: AWS tS3Put and tS3List

Anonymous — Wed, 19 Feb 2020 04:15:37 GMT

I need the link for the files which I have uploaded through tS3Put.

Re: AWS tS3Put and tS3List

Anonymous — Wed, 19 Feb 2020 04:18:25 GMT

Also this job should moved to tS3List only after tS3Put complete the job.

I may need to use IF but could figure it out how to get true value

Re: AWS tS3Put and tS3List

Anonymous — Wed, 19 Feb 2020 04:19:39 GMT

Why you need runIf connector? What's the condition you set now? To me, you just need onSubJobOK to trigger the next subjob.

...tFileList--iterate--tS3Put

|onsubjobok

tS3List--iterate--tFixedFlowInput--tLogRow (or file)

Re: AWS tS3Put and tS3List

Anonymous — Wed, 19 Feb 2020 04:30:12 GMT

This Same job is uploading and getting list of files.

If I run the job and upload is finish only than I should get the Files Link from folder and after the I need to change this to JSON

if you see my logs, everytime it is uploading and its calling tS3List again and again. below is showing from files 3 to 4.

Ideally when all the files uploaded by tS3Put than job should move to tS3List

Re: AWS tS3Put and tS3List

Anonymous — Wed, 19 Feb 2020 08:33:45 GMT

What's the condition expression of runIf connector? As I suggested, use OnSubjobOK to trigger next subjob starts with tS3List component.

Re: AWS tS3Put and tS3List

Anonymous — Wed, 19 Feb 2020 21:58:09 GMT

Hi,

There is no OnSubJobOk for tS3Put. There is only On Component OK. But it still triggering next component every time it upload each file.

Thanks,

Zain

Re: AWS tS3Put and tS3List

Anonymous — Thu, 20 Feb 2020 01:11:50 GMT

OnSubjobOK connector is only avaliable for the first component of a subjob, so your job looks like
...tFileList--iterate--tS3Put
|onsubjobok
tS3List--iterate--tFixedFlowInput--tLogRow (or file)

Re: AWS tS3Put and tS3List

Anonymous — Thu, 20 Feb 2020 02:15:07 GMT

Its not coming , can you please share screen shot for this. Below is my screen shot for reference. I want to connect tS3Put to tS3List and list will only show once all files uploaded to S3.

Re: AWS tS3Put and tS3List

Anonymous — Thu, 20 Feb 2020 04:53:05 GMT

see below.