Incremental Load Error

[#1]

Hi All,

I'm trying to do a Incremental Insert and Update Load on Update Date. I'm getting a syntax error. Can someone provide assistance.

Thanks in Advance,

[sunny_talwar]

Why are you doing SQL SELECT * for QVD load?

May be you want this instead

Concatenate

LOAD *

FROM customercuration.qvd (qvd)

Where not Exists (pkdim1customercuration);

[maxgro]

maybe (bold = change)

ODBC connect.....

CustomerCurationIU

SQL Select ........;

concatenate (CustomerCurationIU)

load *

from .......qvd (qvd)

where not exists (pk....);

store.......

drop......

[#4]

Guys I'm new to this so please forebear with me. Your correction works great, With this Insert and Update Load will I lose my old data? I loaded and it showed only 138 rows while I know the DB table has around 200K rows.

Please provide insights.

Thanks

[sunny_talwar]

You should not. The only thing you should loose from the qvd are the new pkdim1customercuration which you are updating with new data from the SQL. So if your qvd had 10,000 rows of data and the  SQL updated 10 of them and 10 were new. You should have 10010 rows of data in the qvd after the reload.

[maxgro]

you have a where updatedate > ......    in the

SQL select ......

maybe it filter the rows from your database

to check how many rows you get from the db, comment the last part of the script and try

ODBC connect.....

CustomerCurationIU

SQL Select ........;

//concatenate (CustomerCurationIU)

//load *

//from .......qvd (qvd)

//where not exists (pk....);

//store.......

//drop......

[#7]

It only shows 138 rows

[#8]

I still get 138 rows. Still don't know the reason.

[sunny_talwar]

As maxgro‌ 1st create a new qvd using

One Time:

ODBC connect.....

CustomerCurationIU

SQL Select ........;

STORE CustomerCurationIU into customercuration.qvd (qvd);

LET vLastRunTime = Date(Today(), 'DD-MMM-YYYY');

This will tell you how many rows of data you have.

Now setup your incremental load

Incremental:

ODBC connect.....

CustomerCurationIU

SQL Select ........

Where updatedate > '$(vLastRunTime)';

Concatenate

LOAD *

FROM CustomerCurationIU

WHERE not Exists(pkdim1customercuration);

STORE CustomerCurationIU into customercuration.qvd (qvd);

LET vLastRunTime = Date(Today(), 'DD-MMM-YYYY');

[#10]

I still don't get the big chunk except for the 138 rows.