Load script hanging

[#1]

Hi All,

Having trouble with a script I am running. 90% of the time the script completes with no problem. Occasionaly the script gets held up on one table (log has pointed to same one on the same error). The script does not fail, but after 24 hours is just killed. Where it is getting stuck is on a table that is pulling using a wildcard from statement (not sure if this could mean something).

There is a lot of moving parts / refreshing of files over the schedule of this reload, and so my initial thought was that this folder could be getting updated or modified while the load might be going on, but checking our task manager / logs it looks like the dashboard ran as planned.

I know not much information to go off of, but wondering if anyone has gone through anything similar.

Best,

Kevin

[datanibbler]

Hi Kevin,

is that table being pulled from a database via ODBC? Could it be that QlikView cannot access it for some reason, maybe because it is in use and blocked?

Anyway, if the script completes without problems the greater part of times, then the error is with most probability not in your script.

Think logically and list up what external factors there might be influencing the LOAD.

Also, LOADs using wildcards (*) are generally not a good idea because you do load from a nr. of sources - that is the point of it, after all - but in case something goes wrong, the log does not tell you which source it was that QlikView encountered an error on. Try to replace that with a LOOP where the sources are worked on one after another.

(I had just that, a LOAD with a *, loading data from 11 different Excel lists, and we never got detailed debugging info when it failed. So I replaced it with a LOOP, just running 11 times and loading one list at a time. That way, when the LOAD fails, we don't have to open all 11 lists, but only one.

HTH

Best regards,

DataNibbler

[Anonymous]

We had this kind of issue before and problem in the memory utilization (picture below).

Wildcard ("*") in the SQL statement it's not better choice, try to load columns you need to have only in the application.

Also you can load data by years:

Year_2013:

select col1, col2, col(n) from Table where Year = 2013;

Year_2014:

select col1, col2, col(n) from Table where Year = 2014;

And then concatenate tables (or qvd) to the one.

Both methods working for us.

is ki

[rido1421]

Hi Kevin

what database are you connecting to?

I had a similar issue on an Oracle DB , the issue was within my sql script...

I was using sysdate to query and it would run for over 8 hours and sometimes fail as you said

once I specified the dates it would complete within minutes ... not sure if yours is the same issue

Regards,

[#5]

Hey,

Thanks for the responses. This is our front-end dashboard and so we are only using extracts for it (No ODBC connections). To clarify, we are using the * statement to pull in multiple files with same format and path (not in load / SQL statement) and bring them into one table.

Datanibbler (great name), I do agree that the for loop process would be a lot more beneficial solely on the fact that we would be able to decipher which table was giving us the stall in the logs. I think we will be trying that out. But off-hand, would the wildcard have any order in how it brings fields in?

i.e  If we had filesData_01, Data_04, Data_03, Data_02 and used a from Data_* would there be any specific way it would pull fields in. ( I'd guess just on where files are located in folder ?)