topic Re: Comparing CRC values in an expression filter in Talend Studio

Comparing CRC values in an expression filter

Anonymous — Sat, 16 Nov 2024 12:55:20 GMT

Hi,
I need to read records from an Oracle DB(Source) and load it into a SQL server db.(Target)
I am using a tAddCRCRow to generate a CRC column . This column is in the Target Table.
I then lookup the Target table to find out updates vs. inserts.
Here is what I do in the tMap component.
I look up based on primary key using 'Inner Join'
Then I update the table or insert(inner join reject)
In the Update I also use an expression filter to say Source.CRC != Lookup.CRC
But it still updates all the rows , not just the changed rows. I need to retain history as I have a timestamp column to track changes for later.
Is the expression wrong? We are using Java and NOT perl.
New to Talend (& Java) - any help is appreciated.
Thanks

Re: Comparing CRC values in an expression filter

Anonymous — Thu, 19 May 2011 10:15:50 GMT

Hi
What's the data type of this column? Source.CRC?
Best regards
Shong

Re: Comparing CRC values in an expression filter

Anonymous — Tue, 21 Jun 2011 18:48:30 GMT

Hi Shong,
I am sorry, I think I had created this post twice and never saw your reply.
The datatype is Long. This is the default datatype created by the tAddCRCRow Component.
The target.CRC is also long.
When I use (Source.CRC+0) != (Target.CRC+0) , it works.
Another question - is there a possibility using the tAddCRCRow to create the CRC (Using all available fields except the key field) , changed rows will be missed?

Re: Comparing CRC values in an expression filter

_AnonymousUser — Tue, 06 Sep 2011 05:17:17 GMT

Hello
I have the same Issue with excel files as source & loading the data tho target oracle database.Can anyone explain me with screenshots Please.

Re: Comparing CRC values in an expression filter

_AnonymousUser — Tue, 06 Sep 2011 05:24:42 GMT

Hi Gurus -
I have a scenario like I am dealing with XMl files as SRC & want to load into Oracle database(TGT). Here I am dealing with 2 Files,
data from file 1 is a direct pass into DB & based on a primary key file 2 data is present in database neglect it or insert it..
So I have a date coloum for ex : say I got file 1 on july 15th..So database is inserted with July 15th & I got 2nd file on july 27th .My data should get insert file 2 as july 27th..(this is how I'm expecting) . I am new to this tool..Do you have anything like doing lookup on target table or CRC concept or CDC concept?
Can anyone explain me with screenshots Please.
any help would be appreciated.Thank you.

Re: Comparing CRC values in an expression filter

Anonymous — Thu, 12 Dec 2013 22:12:21 GMT

Is it possible to join CRC values in the job and not storing in the table? Also if i want to insert or update based on change in CRC values how do I do it ?