Incremental reloading

[#1]

ello,

I am working on Incremental reloading.

The procedure I followed for the same is:

1. 1st time full load and store data in qvd with timestamp field.

2. Next time I get the max value of the timestamp field from the qvd.

3. read new or updated records from database based on the timestamp field.

But here in step 2 I dont get the correct max value.

Please can anyone help me on this issue of how to get the max timestamp value from qvd.

[rwunderlich]

This appears to be a problem of magnitude. When your tiestamp value exceeds the limits of a 32bit positive number 7FFF FFFF = 2,147,483,647.

I don't have any timestamp value that large in my test db. But with a bit of googling I could run your test case as

SQL SELECT *

FROM test.dbo.RevTable

WHERE Revison=convert(timestamp,2209790167)

(Revision is the name of my timestamp column).

So for your case try:

WHERE timestamp=convert(timestamp,$(vMaxTimestamp))

-Rob

[#3]

Rob,

I will try with the solution you provided.

But I also found a solution as

max timestamp in my qvd is 2209790167.

So I will fetch the hex format from the qvd having above value.

like:
LOAD MaxString(VEts) as maxtimestamp FROM $(vQvdPath)$(vQvdName) (qvd);

LET RecordTs = FieldValue('VEts', 1);

LOAD timestamp as maxTS FROM $(vQvdPath)$(vQvdName) (qvd)

where VEts = $(RecordTs)

LET RecordTimeStamp = FieldValue('maxTS', 1);

now this RecordTimeStamp will have hex format and then I can use this in my query as

WHERE timestamp > $(RecordTimeStamp)

I tried this and it is working on the server also.

But need to verify again after sometime if it gives newly records.

What is your comment on above solution?

Also after I try your suggestion and if it also works, then what will you suggest which option shall I go for?

[rwunderlich]

I think either solution is fine.

-Rob

[#5]

Thank you very much Rob for all the support provided.

As of now all is working fine.

If anything will ask you.