Issue with using FileSize for History

[dafnis14]

Hi All,

Working with Ver.11, SR 10, on a virtual machine.

We have  a batch for running a model which prepares QVDs, a few times a day.

Usually, it runs fine, for a few years.

After moving to a virtual machine, we encounter once in a while a strange behavior with retrieving history for various files.

For example, the following  (tweaked) script is used to append current invoices with historical ones:

Invoices:
LOAD *,
resident Invoice_Temp

if (filesize( '$(Path)$(FinalData_QVD)Invoices.qvd' ) >0 or FileTime('$(Path)$(FinalData_QVD)Invoices.qvd') > 0) then
LOAD * FROM $(Path)$(FinalData_QVD)Invoices.qvd(qvd)  where ValueDate < '$(StartDate)' ;
end if
 
store * from Invoices into $(Path)$(FinalData_QVD)Invoices.qvd (qvd);

The issue we encountered is that only the current invoices are saved. Another words, the history table was not loaded. It happens with all history files in that specific model.

Some of the  history files are very big: like 1.5 GB. Could this be the reason for an issue with a virtual machine?

Thanks!

[sushil353]

Hi,

You can check log file of your QVD generator. this will tell you about the no of records fetched from history qvd and the source.

you can enable log file in Setting>document property>General

Check the option for Generate log file

HTH

Sushil

[dafnis14]

Hi,

The problem is that the log  was overriden by the next run.

But this issue doesn't happen often. I recover the files from backup, and everything works fine, for days or weeks.

[marcus_sommer]

I think filetime() and filesize() are based on windows-functions and could imagine that if there are an external access on this file through a security-tool or services like windows shadow-copies the file will be locked and the functions return 0 or false. Maybe you include some kind of variable-logging to be sure, like:

varLog:

Load * From varLog.qvd (qvd);

     concatenate

Load

     filesize( '$(Path)$(FinalData_QVD)Invoices.qvd' ) as FileSize,

     FileTime('$(Path)$(FinalData_QVD)Invoices.qvd') > 0) as FileTime

Autogenerate 1;

store varLog into varLog.qvd (qvd);

drop table varLog;

- Marcus

[dafnis14]

Hi Marcus,

Thanks for the tip.

The task is run a few times a day.

I noticed that when the issue does occur, it's at a certain time when a larger history file is being retrieved.

Another words, is it possible the issue might be related to a file size?

[marcus_sommer]

Maybe the respond-time is too slow so that the function after a certain timespan react with a timeout and FALSE or NULL as result. You could put your file-check in a recursive loop maybe something like this:

for i = 1 to 3

     if yourCheck = true then

          exit for;

     else

          let i = $(i) + 1;

          sleep 2000;

     end if

next

- Marcus

[dafnis14]

Ok, will do that!

Thanks!