topic Issue with using FileSize for History in QlikView

Issue with using FileSize for History

dafnis14 — Thu, 16 Jul 2015 09:28:23 GMT

Hi All,

Working with Ver.11, SR 10, on a virtual machine.

We have a batch for running a model which prepares QVDs, a few times a day.

Usually, it runs fine, for a few years.

After moving to a virtual machine, we encounter once in a while a strange behavior with retrieving history for various files.

For example, the following (tweaked) script is used to append current invoices with historical ones:

Invoices:
LOAD *,
resident Invoice_Temp

if (filesize( '$(Path)$(FinalData_QVD)Invoices.qvd' ) >0 or FileTime('$(Path)$(FinalData_QVD)Invoices.qvd') > 0) then
LOAD * FROM $(Path)$(FinalData_QVD)Invoices.qvd(qvd) where ValueDate < '$(StartDate)' ;
end if

store * from Invoices into $(Path)$(FinalData_QVD)Invoices.qvd (qvd);

The issue we encountered is that only the current invoices are saved. Another words, the history table was not loaded. It happens with all history files in that specific model.

Some of the history files are very big: like 1.5 GB. Could this be the reason for an issue with a virtual machine?

Thanks!

Re: Issue with using FileSize for History

sushil353 — Thu, 16 Jul 2015 09:32:46 GMT

Hi,

You can check log file of your QVD generator. this will tell you about the no of records fetched from history qvd and the source.

you can enable log file in Setting>document property>General

Check the option for Generate log file

HTH

Sushil

Re: Issue with using FileSize for History

dafnis14 — Thu, 16 Jul 2015 09:36:17 GMT

Hi,

The problem is that the log was overriden by the next run.

But this issue doesn't happen often. I recover the files from backup, and everything works fine, for days or weeks.

Re: Issue with using FileSize for History

marcus_sommer — Thu, 16 Jul 2015 10:48:01 GMT

I think filetime() and filesize() are based on windows-functions and could imagine that if there are an external access on this file through a security-tool or services like windows shadow-copies the file will be locked and the functions return 0 or false. Maybe you include some kind of variable-logging to be sure, like:

varLog:

Load * From varLog.qvd (qvd);

concatenate

Load

filesize( '$(Path)$(FinalData_QVD)Invoices.qvd' ) as FileSize,

FileTime('$(Path)$(FinalData_QVD)Invoices.qvd') > 0) as FileTime

Autogenerate 1;

store varLog into varLog.qvd (qvd);

drop table varLog;

- Marcus

Re: Issue with using FileSize for History

dafnis14 — Thu, 16 Jul 2015 11:42:57 GMT

Hi Marcus,

Thanks for the tip.

The task is run a few times a day.

I noticed that when the issue does occur, it's at a certain time when a larger history file is being retrieved.

Another words, is it possible the issue might be related to a file size?

Re: Issue with using FileSize for History

marcus_sommer — Thu, 16 Jul 2015 11:51:51 GMT

Maybe the respond-time is too slow so that the function after a certain timespan react with a timeout and FALSE or NULL as result. You could put your file-check in a recursive loop maybe something like this:

for i = 1 to 3

if yourCheck = true then

exit for;

else

let i = $(i) + 1;

sleep 2000;

end if

- Marcus

Re: Issue with using FileSize for History

dafnis14 — Thu, 16 Jul 2015 13:31:48 GMT

Ok, will do that!

Thanks!