topic Re: How to get CSV Files from S3 and transform to XML using mediation? in Talend Studio

How to get CSV Files from S3 and transform to XML using mediation?

Anonymous — Sat, 16 Nov 2024 05:23:43 GMT

I am creating a route for the below scenario.

1. Connect to AWSS3

2. Download CSV files from an S3 bucket and save the file to the directory

3. Read the CSV file from a directory and transform CSV rows into XML

I tried to do the first two-point, but unfortunately, there is no component to close the connection. Also not sure on how to pass the CSV file to cTaldnJob for transformation.

Can anyone please help?

Thanks

Re: How to get CSV Files from S3 and transform to XML using mediation?

Anonymous — Wed, 03 Jul 2019 15:30:46 GMT

Hi,

Below is a skeleton flow for the task.

If you are facing any issue, I would recommend to share the job flow screen shot along with error and problematic component screenshots for further analysis.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

Re: How to get CSV Files from S3 and transform to XML using mediation?

Anonymous — Thu, 25 Jul 2019 11:32:39 GMT

Hello Nikhil,

Thank you for your response and sorry delay in my reply. I have managed to replicate the job and successfully retrieved the file.

I am struck with the below scenario can you please shed some light on this?

When any CSV row failed the validation I do not want to process the CSV file further and push the entire input file to a different folder. I've tried but I am getting right, can you please?

Re: How to get CSV Files from S3 and transform to XML using mediation?

Anonymous — Fri, 26 Jul 2019 16:18:40 GMT

Hi,

You can easily do it by adding a flag variable. At the starting of the job, assign assign reject_flag="N" using a tjava in Prejob section.

The next step is to verify whether any records are getting rejected by using tschemacompliancecheck. You can add a tjavarow after that and assign value to a context variable assign reject_flag="Y" . Then use a if condition to verify whether the value is "Y" or "N".

If the value is "Y" send the control to move the input file to reject area and if its "N" send the control to process the record further.

I hope this will resolve your issue. Please spare a second to mark the topic as resolved since your original query has been answered.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

Re: How to get CSV Files from S3 and transform to XML using mediation?

Anonymous — Mon, 29 Jul 2019 09:26:21 GMT

Hi Nikhil,

Thank you for your response. I will try the suggested approach and get back to you soon, please.

Thanks