topic Re: Save point in Talend in Talend Studio

Save point in Talend

Anonymous — Thu, 19 Mar 2015 10:04:22 GMT

can Talend provide save point type mechanism or not?

Re: Save point in Talend

Anonymous — Thu, 19 Mar 2015 10:30:44 GMT

Yes! talend has save point but, those are not like database save point. it is used to restart a job from where it failed. It is also called as "Restart Functionality or ability", if job fails in the middle of execution.
this is not available in Open Studio.

Re: Save point in Talend

Anonymous — Fri, 20 Mar 2015 03:15:11 GMT

how we can implement this mechanism of save point.
i want that if any error come my whole jobs effect should be rollback also from database.

Re: Save point in Talend

Anonymous — Fri, 20 Mar 2015 06:08:15 GMT

you are asking about transaction? if yes then you have to use three database components as below.

First Open connection using tOracleConnection ( you can use relevant database)
Then use your oprations like insert, update delete using tOracleInput and output component.
at last on sub job ok use tOracleCommit. to commit the changes. you final job should look like below.

tOracleConnection
|
OnComponentOk
|
tOracleInput---tMap---tOracleOutput ( use connection option instead configuring at component level)
|
OnSubJobOk
|
tOracleCommit

Save point functionality available in enterprise version and you have to configure it using TAC.

Re: Save point in Talend

Anonymous — Fri, 20 Mar 2015 06:37:39 GMT

thanks.
But it slow down my jobs performance.
16 Lac records transfer in 48 minutes which is very slow can u have any idea about it.

Re: Save point in Talend

Anonymous — Fri, 20 Mar 2015 08:05:25 GMT

You can use bulk component, to load the data faster, also you can do some optimisation for input component.

Select only required columns,
If it is oracle then select cursors options in the advance tab of component, it will be faster than current process.
use bulk load component for data load. ( it requires loader to be installed on machine where job runs)
Load data in chunks, by applying filters on select statement.
Add more memory on current hardware so job can utilizes it for better performance.

Re: Save point in Talend

Anonymous — Fri, 20 Mar 2015 09:24:50 GMT

i want to transfer data from oracle input table to oracle output table so i cannot use Bulk load component.
I Implement first 2 points that is given by you.
but still it take 35 to 40 min.

Re: Save point in Talend

Anonymous — Fri, 20 Mar 2015 09:36:46 GMT

Use multi threading option from job tab.

Re: Save point in Talend

Anonymous — Fri, 20 Mar 2015 10:18:46 GMT

how this help to me?

Re: Save point in Talend

Anonymous — Fri, 20 Mar 2015 10:19:53 GMT

how this help to me?
I want to transfer all records.
i don't know how to use it and how it work.
can u please tell me how i can use it

Re: Save point in Talend

Anonymous — Fri, 20 Mar 2015 10:27:06 GMT

it is just a check box on job tab beside the context tab, go there you will see another settings among them click on "extra"-->select "[]multi threading execution" option.
Not required anything else. it will help you to execute your job using threading.

Re: Save point in Talend

Anonymous — Fri, 20 Mar 2015 10:34:05 GMT

but nothing improved in performance by this(Multi Thread Option).
i use bulk component.Job run without error but noting inserted into database table.

Re: Save point in Talend

Anonymous — Fri, 20 Mar 2015 10:47:12 GMT

there are two bulk components which one you are using?
follow the appropriate documentation before using it.
I recommend you, to use tOracleOutputBulkExec.

Re: Save point in Talend

Anonymous — Fri, 20 Mar 2015 10:52:38 GMT

this is the screen short of my job.

Re: Save point in Talend

Anonymous — Fri, 20 Mar 2015 11:14:28 GMT

your flow is right, I am suspecting commit is not happing, can you place commit component after bulkExec and check whether it works or not.
use tOracleOutputBulkExec component. as i told you earlier.

Re: Save point in Talend

Anonymous — Fri, 20 Mar 2015 11:27:07 GMT

Commit is already there after Bulc_exe. It is not worked.

Re: Save point in Talend

Anonymous — Fri, 20 Mar 2015 11:29:39 GMT

check for tOracleOutputBulkExec component in pallet and use that one, but make sure you are running this job on the same machine where sql loader/oracle server installed .

Re: Save point in Talend

Anonymous — Fri, 20 Mar 2015 11:31:18 GMT

in my machine oracle client is there not server.