Script stopping for no apparent reason

[GLicks_DG]

Hello guys,

Is there any reason for qlik suddently stopping execution? I mean there is no error, the execution just stops as if the script was over. The script itself is doing the same things over and over again just varying the data it loads. All the results are 'saved' in a table which is later (at the actual end of script) in qvd file.

Im still running more tests, but the script stops after almost 3h of execution, so tests are slow...

My guess would be that i'm hitting some internal memory bottleneck or something like that. Is there anywhere i can read more about this possible bottlenecks?

Have anyone had this type of issue and managed to work it out??

Thanks in advance!

[fkeuroglian]

Hi how are you?

Pherphaps in some part of the script you run out of memory because the model and the relation that you are creating.

Do you test in the debug mode with some register(for example 100)? to test  the model and if you have any syntetic ?

Other test could be start to create "exit script" in the part in your model that you consider could be a problem

I would start with the debug mode to be sure that the model with low data is OK.

Hope helps

Fernando

[GLicks_DG]

Sorry for taking this long to get back..

So, I ran some more tests and just cannot explain whats going on. Some more additional info:

• Full script have around 250k lines (i'm calling it from a .qvs);
• It stops just short of 10k lines (2.5h);
• Every line is pretty much the same, changing only the variables it uses. Every line also call other .qvs which does all I need (load the relevant data, do a couple joins, some calculation, generate the results and store it in a .qvd) and ALL tables are dropped! The process then restarts.
• When running it in debug mode, the process gets VERY slower. The 2.5h become more than 2 days of execution (actually it is running since tuesday night and still haven't reached the point where it stops (~10k line)). What a can certanly say is that the scripts is visibly getting slower and slower.
• When running just a few (like 1000) lines, it all works fine; also, if I start the execution on line 5k (for example) it stops around 15k so the problem is not with a particular line or data..

I don't believe it to be any internal memory bottleneck anymore, since i'm basically starting from scratch every line (I start from a blank page, load all i need, do all i need, store all i need and DROP EVERYTHING before going to next line).

Due to security issues I'm not able to share the full code with you guys, but i can provide a sample:

SET vVariable1 = Something; SET vVariable2 = Something; SET vVariable3 = Something; ($Include=Script1.qvs);
SET vVariable1 = SomethingElse; SET vVariable2 = SomethingElse; SET vVariable3 = SomethingElse; ($Include=Script1.qvs);
SET vVariable1 = SomethingElse; SET vVariable2 = SomethingElse; SET vVariable3 = SomethingElse; ($Include=Script1.qvs);
SET vVariable1 = SomethingElse; SET vVariable2 = SomethingElse; SET vVariable3 = SomethingElse; ($Include=Script1.qvs);
...
and so on​

Script1.qvs:

Table1: LOAD * FROM [SOURCE1.QVD];
Table2: LOAD * FROM [SOURCE2.QVD];

Inner Join (Table1) LOAD SomeField Resident Table2 Where Something = Something;
Drop Table Table2;

Results: Load SomeOtherField, SomeCalculations as Calculations Resident Table1 where SomethingElse = SomethingElse;
Drop Table Table1;

Store Results into Results_xxx.qvd (qvd);
Drop Table Results;

 If anyone have any idea of what might be happening... It fells like qlik is just giving up midscript.

Thank again!

[GLicks_DG]

One more quick info: The log file generated only says "script finished"..

[fkeuroglian · Accepted solution]

Hi how are you

As you said:

 When running just a few (like 1000) lines, it all works fine; also, if I start the execution on line 5k (for example) it stops around 15k so the problem is not with a particular line or data.

If this happend, it sound like something in your model (that you are creating) it is not OK, and qlik model braek out! and not finish to create the model

I will be start part to part(divide and conquer) and start just put some EXIT SCRIPT in every join that you have.

In yours example you put some joins, but that is very important because if have more than one field named equal could be a problem(if you have a lot fields equals between tables, you have to create a combinate key and this is the relation)

Why you load the two table(table 1 and table2) and then  you join using a resident to the table 2, why you do not join at fist and not have to do this resident table 

Hope help

Fernando

[GLicks_DG]

Fernando, thanks a lot for the help..

At last I ended up using the 'divide and conquer' strategy and changed the logic (used a few loops) and it finally worked.. Still I don't know exactly what happened, but surely was something in my model.

Thanks again!