Migrate data from My SQL to AWS S3 with parquet file

[Sireesha_Chapa]

Hi All,

I have 2 questions which I am looking in multiple documents but could not find concrete answers:

I appreciate your quick response. Based on that I am in need to check about the feasibility of using Talend in our project.

1.If I use Talend Stitch, I can directly insert the data from MySQL to AWS S3 without any file?

2.Even if i use talend studio , can I directly write the data from My SQL to Parquet and then to AWS S3 ? I could see many references where one writes into CSV and from CSV it would be written into S3. So, I should know without using CSV, is there any architecture in Talend that transfers to Parquet from My SQL directly and then from Parquet to S3.

"But the other alternative of using like Spark Job" - Can I get any references for this for inspiration. I could not find it. Could you please show referral architecture to process the data from SQL server to S3 using parquet file.

Also if I use tDBInput_1 -->tFileOutputParquet_1

Would this work? Would it insert the data directly from Server without using any spark jobs?

i attached document for referal architecture. that will work?

[Anonymous]

I recommend you contact Talend Professional Services. It is clear that your use case needs further refinement. First, tFileOutputParquet is only supported for Spark jobs. Second, the recommended solution as I indicated in my reply to your previous post is to use Stitch. I am not clear what you are trying to achieve with a 'no delimited file' type of movement or what your constraints are in this case. This discussion board covers the Talend Move to Cloud topic, which primarily concerns with moving Talend on-premises installations to Talend Cloud. The use case you describe seems more a data-play. You may also consider posting to a different forum that is more relevant to your needs.