topic Flow to write to a file in Talend Studio

Flow to write to a file

danblo — Sat, 16 Nov 2024 08:10:08 GMT

I have a job that reads a csv file, retrieves some values and stores them into global variables, then uploads a file to S3. When that it's done, I'd like to write the global variables and the outcome of S3 upload to another file. How do I go from tS3Put to tFileOutputDelimited?

Re: Flow to write to a file

TRF — Fri, 01 Jun 2018 12:54:43 GMT

OnSubjobOK after tSetGlobalVar_1 or tFileInputDelimited_2 should work.

Re: Flow to write to a file

danblo — Fri, 01 Jun 2018 13:14:29 GMT

I need this to happen after tS3Put, so I can get whether the upload completed or not.

Re: Flow to write to a file

TRF — Fri, 01 Jun 2018 13:17:40 GMT

In this case use OnComponentOK/OnComponentError after tS3Put

Re: Flow to write to a file

danblo — Fri, 01 Jun 2018 13:23:57 GMT

It won't let me go from tS3put to tmap.

@TRF wrote:
In this case use OnComponentOK/OnComponentError after tS3Put

Re: Flow to write to a file

TRF — Fri, 01 Jun 2018 13:31:33 GMT

Because tMap expect a flow you cannot use an event trigger to connect it.
Be more explicit about what you expect (maybe with sample).

Re: Flow to write to a file

danblo — Fri, 01 Jun 2018 13:42:15 GMT

loop--> call a python script using tsystem (which generates a log file with value1 and value2) and an output file-->tFileInputDelimiter2 reads the log file-->tjavaRow stores value1 and value2 into global variables-->tS3put uploads the file to S3. That works so far.

Now, if the upload was sucessful (I can use onSubjectOK), I want to write value1, value2 and the message ("OK") to another text file.

Hope that clears it out.

Re: Flow to write to a file

TRF — Fri, 01 Jun 2018 13:54:03 GMT

Right.

So you can replace your tMap by a tFixedFlowInput component with the same schema expected for the output file.

Then you populate each field with the desired values issued from the 2 global variables + "OK" for the message field.

Re: Flow to write to a file

danblo — Fri, 01 Jun 2018 14:13:11 GMT

Bingo. That's what I was looking for. Thank you.

Re: Flow to write to a file

TRF — Fri, 01 Jun 2018 14:28:15 GMT

You're welcome