topic Re: [resolved] tOracleSCD Use Memory Saving Mode in Talend Studio

[resolved] tOracleSCD Use Memory Saving Mode

Anonymous — Thu, 21 Feb 2013 16:01:06 GMT

Hi,
I am trying to use the memory saving mode with a tOracleSCD component as I have a dimension table with over 1 million rows - however I am receiving the error below whenever I check the "Use memory saving mode" box.
Exception in component tOracleSCD_1
java.sql.SQLSyntaxErrorException: ORA-00920: invalid relational operator
Any ideas? Do I need to setup anything special? How does memory saving mode work? I did not see any details in the documentation.
I am using
Talend Enterprise Data Quality Professional edition 5.1.2
Java

Re: [resolved] tOracleSCD Use Memory Saving Mode

Anonymous — Fri, 22 Feb 2013 02:35:00 GMT

Hi,
Here is a component reference about tOracleSCD, in which there is a related scenariotMysqlSCD. Hope it will help you.
Best regards
Sabrina

Re: [resolved] tOracleSCD Use Memory Saving Mode

Anonymous — Fri, 22 Feb 2013 14:59:21 GMT

Hi Sabrina,
Thanks for replying.....I neglected to mention that I'm already familiar with the SCD component and in fact this job works when I don't have the "Use memory saving mode" box checked.
I received the error message after checking the "Use memory saving mode" ....it did not error before however performance was horrible and it was using a lot of memory.
The documentation only has this regarding the "Use memory saving mode" option box:
Select this check box to maximize system performance.
Is there anything else I need to do besides select the check box?
I assume (because of lack of documentation of how this works) memory is saved by writing to disk. Do I need to set a local directory?
Chris

Re: [resolved] tOracleSCD Use Memory Saving Mode

Anonymous — Mon, 25 Feb 2013 05:27:44 GMT

Hi,
Could you upload your job screenshot into the forum so that we can see your work flow and then see if it need be saved in the disk.
Best regards
Sabrina

Re: [resolved] tOracleSCD Use Memory Saving Mode

Anonymous — Mon, 25 Feb 2013 16:12:33 GMT

Here's an upload of my job and the settings for the SCD Component.
Please keep in mind that my target table has over 1.7 million rows and we will update about 5,000 rows each time we execute the job (daily).

Re: [resolved] tOracleSCD Use Memory Saving Mode

Anonymous — Tue, 26 Feb 2013 03:29:13 GMT

Hi,

Exception in component tOracleSCD_1
java.sql.SQLSyntaxErrorException: ORA-00920: invalid relational operator

Here is a bug for your issue https://jira.talendforge.org/browse/TDI-23866. In addition, if you have a large number of dimension table, it suggested that you should checkout "Use Memory Saving Mode".
Uncheck the memory saving mode: Transferring all data which is in the activated State and identified on a Map cache.
If checked: According to the compared Key you wanted to get the query, and then comparing results to see if they are consistent with each other, and if need update the target table.
you can read the generated code from talend studio to know the working principle more clearly.
Best regards
Sabrina

Re: [resolved] tOracleSCD Use Memory Saving Mode

Anonymous — Fri, 01 Mar 2013 21:58:07 GMT

Thanks Sabrina....
So we would need to upgrade to get the fix.
Chris

Re: [resolved] tOracleSCD Use Memory Saving Mode

Anonymous — Sat, 02 Mar 2013 02:12:12 GMT

Hi,
I agree with you that the Production promotion will be a good solution for your issue.
Best regards
Sabrina