QlikView: avoiding a sub() when debugging

[Anonymous]

I have been given a script to debug which loads data from a lot of different files. For each file, there is a FileExists() check before trying to read from the file.

Unfortunately, when I run the debugger, it always goes into this sub and steps through that code. This takes a lot of time and creates a lot of visual switches which is very bothersome, resulting in a debugging which is much more tiresome and timeconsuming than it should be.

In many other debugging systems, it is possible to control whether to dive into a subroutine/function or not. But I cannot find any such way to run debugging in QlikView. Is there some (perhaps undocumented) way to do this, or some other way to get around this issue?

[marcus_sommer]

Personally I doubt that there is any way to skip certain script-parts within the debugger or within a normal load unless it's already included within the script. This meant that the script contained any condition-statements, most commonly per if-loops like:

if '$(var)' = 'run fileexists' then

...

whereby these checks and the call of a sub-routine will be further recorded within the log.

If you really mean a (certain) sub-routine you could just go there and comment the content. An alternatively might be also to evaluate the log-files with a tool like: Search Recipes | Qlikview Cookbook.

Beside this I got the impression that there is put too much within a single qvw (as it is sensible from a developing/maintaining and performance point of view) and it might be useful to split it into multiple tasks.

- Marcus

[Anonymous]

Maybe I was not totally clear, I wan the execution to proceed into the sub() but not shown in the debugger. In some other system this is often called "Step over", i.e the execution proceeds into the function but is not shown, as if it was a system call (which execution are never shown, only done); instead the debugger just moves to the next statement in the executing file.

It seems strange if I should be the only one who is using code where some parts are perfectly valid and validated, and do not need to be debugged any more?

[marcus_sommer]

I agree with you that there could be some improvements on the debugger and also the editor and the script-log. But even without some advanced features you could often debug your code without too much efforts.

I must admit that I use the debugger rather seldom and then usually to check some variable-values. For this moving script-tabs, adding exit script; statements and/or commenting script-parts in combination with appropriate break-points is easily done and not to inconvenient - at least to me. If it's really a more complicated case - this are usually not classical errors else any unexpected results - and the reasons are not quite obvious I add some trace-statements and/or create some dummy-tables with intermediate steps to find the reason.

This meant more debugger-features would be nice but they aren't really necessary to do the debugging with appropriate efforts.

If it's in your case quite painful it looked like that the application is poorly developed for example by including too much tasks within one application or some adverse designed loops. It sounds that this routine is included within a loop which runs through any folders to grab the files but in this case is a check if the file exists not mandatory necessary.

- Marcus

[Anonymous]

The script is built the way it is because it reads daily data from a lot of different "daily" files  (33 files per day) and assembles these into a monthly table. Normally, it only reads the last day of data and then appends that to a larger data table. Since the data collection can go wrong at any time, any of these files can be missing, but we do not want stop the script because of that.

Additionally, if we discover that a file is either missing, or has some or missing data which we need to restore, we need to re-read the whole month (or part of the month if the current month). Sometimes we also need to re-run the script for a longer time period for other reasons, such as changed configuration (interpretation) files.

So, re-running even a few days in the debugger IS quite painful, especially since we also save intermediate results before assembling it all into a few larger tables.

One can of course always have opinions about the architecture of an application, especially one that has been developer and extended over several years, but a good tool is supposed to help and not force you to jump through a lot of hoops, or start to rebuild an application, in order to get your work done. And print-outs, yes, but I thought that was something we tried to get away from when we left the epoch of the punched cards (which I am old enough to have suffered). And the fact that "step over" is a common feature in many debuggers seems to show that my need is not very unique.

BTW, another thing which is much lacking (or very hard to find) in the debugger is a way to look at the current tables during debugging; you can see, albeit a little primitively, the values of variables, but not what is (or is not ) put into a table. That seems hard to understand.

Finally, how the debugger interacts with the editor seems very strange. After I close the debugger, I am not back in the script editor as I would in any other development system, but back in some place where I need to "reload" the app to continue to work with it - and after that, re-open the editor. It seems to me a very strange kind of thinking, that you would like to do something completely different after closing a debugging session?

Maybe there are ways to work more efficient with the debugger, but since the documentation I can find would fit on the back of an envelope with space to spare, I am not sure how I could learn that.

[marcus_sommer]

Beside already above mentioned that there is much place for various improvements and adding new features to the edit script and the debugger there are reasons why no table-data are displayed within the debugger - and this is the way how qlik stored the data and how and when it builds a datamodel.

Qlik stored the data in symbol-tables (one for each field) and data-tables and only if the reload is finished the datamodel is created - there is no intermediate step available. More to this could you find here: Symbol Tables and Bit-Stuffed Pointers - a deeper look behind the scenes. I assume there would be multiple implications if Qlik adds an extra mode to include such temporary table-generating during the load - probably possible but it would be quite expensive ...

Nevertheless there is a workaround available by storing the intermediate data temporary within a qvd - looking into the qvd - and removing them again, see here what is meant: Inspecting QVD files at script runtime with QViewer.

What I don't understand is why using the debugger to just reload some data? Of course there are multiple reasons why a load-script should run incremental, full or in any way partial - and in this also checking if certain files or data are available and/or valid. But after developing the application the debugger would only be needed by inexplicable results - and only then I would go the ways above described.

- Marcus

[Anonymous]

My problem is that I am trying to understand why the data in our QlikView app appears to differ from what I can see when I look at the data directly in our database (actually, by using ODBC in Excel to extract data into into a pivot table and apply the supposedly same filtering as we use in the QV script).

Since I have not written the QV app myself - I am the product owner - I am stuck with trying to understand how the code loads the data using the debugger to see where it possible goes wrong. Since we do a lot of proccessing of the data when reading it - the data is actually saved as a monthly file which is used by the "presenation" QV app - there is a lot of things that can go wrong in a subtle manner.

Anyway, there does not seem to be any easy resolution to this. Please contact me if you ever want to improve the edit and debug process and want some considered input and suggestions.

[marcus_sommer]

I'm not sure f the debugger is the most suitable tool to find the differences between DB, Qlik and Excel because it's mainly designed to find any kind syntax errors and/or wrong variable-values or loop-iterations. Differences in the interpretation of the data or logically mistakes are quite hard to detect in this way.

Usually it's the best way to slice down the task and using a process of elimination to find the source of the differences - and just starting at the beginning by checking if all 3 tools have the same number of records for the same query and then going further.

- Marcus