topic Re: Processing mutliple csv files through tS3Get in Talend Studio

Processing mutliple csv files through tS3Get

Anonymous — Sat, 16 Nov 2024 05:40:10 GMT

DELETED

Re: Processing mutliple csv files through tS3Get

Anonymous — Thu, 06 Jun 2019 20:21:52 GMT

Hi,

You will have to use tS3List and please refer the example below.

In the tS3Get, you will have to specify the current key from previous component (use Control+Space to see the list). In this way, you can move multiple files in iterative fashion.

On a different note, I have seen that you have given On Component Ok to process the downstream flow. I would recommend to use On SubJob OK (from the starting of previous subjob's first component) so that you can allow the entire SubJob to be complete before moving to next activity.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

Re: Processing mutliple csv files through tS3Get

Anonymous — Thu, 06 Jun 2019 22:03:09 GMT

Thank you Nikhil. This helps a ton. I'll give this a shot. I basically just inherited all of our Talend Jobs that were controlling our BI data flow and was told "here it is, have fun, good luck..." So thank you for baring with me on what now seems to be pretty fundamental.

Re: Processing mutliple csv files through tS3Get

Anonymous — Thu, 06 Jun 2019 22:07:28 GMT

Also, I see a ((String)globalMap.get(*component name*)) being used a lot. I take it this is Java code and it's effectively taking the results of that component and formatting them in a string name to be passed into whatever parameters you are using for your components down stream?

Re: Processing mutliple csv files through tS3Get

Anonymous — Thu, 06 Jun 2019 22:18:50 GMT

Hi,

No worries at all. Talend community is quite vibrant and you don't be alone in this journey 🙂

Please also spare a second to mark the topic as resolved if you are happy with the solution.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

Re: Processing mutliple csv files through tS3Get

Anonymous — Fri, 07 Jun 2019 16:34:19 GMT

Is it possible for me to actually go right from the tS3Get to tMap or must there be a physical source that tMap needs to be able to read from? Basically I'm asking is going from tS3Get to tFileInputDelimited the appropriate way to proceed with what I'm trying to accomplish?

Re: Processing mutliple csv files through tS3Get

Anonymous — Fri, 07 Jun 2019 17:34:28 GMT

Hi,

You cannot go directly from tS3Get to tMap. tS3Get is used to bring the files from S3 to local for further processing and its job is transportation of file from one area to another.

So you will need a tFileInputDelimited component to read the file once its available in local area to be processed further using tMap.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂