topic Re: Batch Data transfer from database in Talend Studio

Batch Data transfer from database

Anonymous — Sat, 16 Nov 2024 11:37:35 GMT

Hi,
What is the best way to transfer data from database to hdfs using Talend? Is there a batch process that transfers the entire data to a single hdfs file, so I could run the job and close the talend application? Or Do I have to split the huge data into several transfers?
Thanks.

Re: Batch Data transfer from database

willm1 — Mon, 02 Jun 2014 15:42:06 GMT

Hi sharidos - take a look at squoop. You can use the Talend Sqoop component to bulk transfer data from an rdbms to hdfs and vice versa
See https://help.talend.com/search/all?query=tSqoopImport&content-lang=en

Re: Batch Data transfer from database

_AnonymousUser — Tue, 10 Jun 2014 19:21:08 GMT

Can Talend's sqoopimport component be used for Synonyms? It works fine with tables.
Is there any specific configuration required for Synonyms?
Thanks.

Re: Batch Data transfer from database

Anonymous — Thu, 12 Jun 2014 19:25:45 GMT

I am using TOS .5.4.1. Is it possible to send the output of tsqoopimport to thbaseoutput? Does it need any other component in between?
I want to transfer data in batches. Like first batch with x number of rows and the second batch with another x number of rows. Is there a way to automate it in single job submission?
Thanks.