topic Re: look for an optimized solution in Talend Studio

look for an optimized solution

Anonymous — Sat, 16 Nov 2024 08:40:28 GMT

Hello,

I am looking for an optimized solution,
the goal is to migrate data from one mysql database to another, my concern is to define a text file that contains the parameters that defines the source database (host, port, databasename ..) and also the file parameters of the output database.
Do you have any ideas how to do this work ?

Thank you

Re: look for an optimized solution

Anonymous — Sat, 17 Feb 2018 23:57:03 GMT

Please check for how to setup contexts in the Talend flow. This way all dB information will be picked up as the job starts.

Re: look for an optimized solution

Anonymous — Sun, 18 Feb 2018 13:49:10 GMT

actually, what I use is the following schema:
tContextLoad_1 -> tFileInputDelimited_1-> tMysqlInput_1 (db_source) -> tMap->
tMysqlOutput_2 (db_destination)

depending on the problem that has occurred is:
1- the tContextLoad_1 text file content change is not supported and the pointing is always on the first database "source database".
2- When I create a second tContextLoad_2 can not be connected by a (but) or (trigger-On subject Ok) with tFileOutputDelimited_1 that linked with tMysqlInput_2 (db_destination), how to define a file that contains the definition parameters of a base output data (host, databasename, identifier, password ...) ??

Re: look for an optimized solution

Anonymous — Mon, 19 Feb 2018 13:00:49 GMT

I have no idea until now, i'm searching how to do it.

Re: look for an optimized solution

Anonymous — Sun, 25 Feb 2018 12:05:19 GMT

I created a job that does the job using its own context, I use export the database parameters as conxtext, the problem is after depolement talend create his own file that contains the settings against my work is to retrieve data from a database that can change its environment, so finally it uses its own default.properties parameter file.

Do you have an idea how on how to represent a database that can change its environment?

Re: look for an optimized solution

TRF — Sun, 25 Feb 2018 12:59:54 GMT

Hi,

Define all the context variables you need for database connection and other as usually but don't set any value for these variables (they will come from an external source).

Define a new context variable (let say ContextFilepath) to define the full pathname of the file in which you will put the values for your context variables.

Go to the Job tab then select the Extra tab.

Untick the option "Use Project Settings".

Tick the options "Implicit tContetxLoad" and "From File".

Enter "context.ContextFilepath" into the "From File" field".

Enter "==" into the "Field Separator" field (or choose an othe separator).

It shoukd look like this:

From outside the Studio, create a file (name and path according to the ContextFilepath context variable.

Enter variable names and values for all your context variables (except ContextFilepath) like this:

variable1 == value1
variable2 == value2
...

Now, each time you'll run your job, context variables will be loaded from your properties file (except ContextFilepath) so you can forget the stardard context files.

If you need different contexts, just create a properties file per context and set the variable ContextFilepath with the correct full pathname, and that's all.

Hope this help.

Re: look for an optimized solution

Anonymous — Sun, 25 Feb 2018 13:16:08 GMT

very well explained Can you give the steps of creating the job exactly, because you mentioned components like tContextFilPath that I never used.

sincerely yours.

Re: look for an optimized solution

TRF — Sun, 25 Feb 2018 14:22:09 GMT

You don't need any additional component for this solution.
Just follow the instructions from my previous answer and that's all.
Don't forget to mark your case as solved as soon as you get the expected result for other guys facing a similar question (Kudos also accepted).