topic Re: Document Analyzer V1.15 in QlikView

Document Analyzer V1.15

rwunderlich — Thu, 02 May 2013 15:32:58 GMT

I've published an updated version 1.15 of Document Analyzer to

Changes in this update:

- Fix a hang caused by unicode values in variables or expressions. Property files are now written in unicode.

If you have any questions or problems with DocumentAnalyzer, post to this thread or email to the address given on the DocumentAnalyzer "About" sheet.

-Rob

Re: Document Analyzer V1.15

Fri, 24 May 2013 06:44:45 GMT

Hi Rob,

I can't see script running....Do you know possible causes ?

Re: Document Analyzer V1.15

rwunderlich — Fri, 24 May 2013 18:20:35 GMT

Can you post the CollectProperties log and the document log?

-Rob

Re: Document Analyzer V1.15

Wed, 09 Oct 2013 11:52:50 GMT

Hi Rob,

I tried running Doc Analyzer v1.15 and it gave out of memory error. Not sure why.

Could you please help. I am attaching a screenshot and the doc analyzer log file for your reference

Thanks

raghu

Re: Document Analyzer V1.15

Wed, 09 Oct 2013 12:10:37 GMT

Thanks Rob for these useful tools

Have a good day

Chris

Re: Document Analyzer V1.15

jchoucq — Thu, 10 Oct 2013 08:02:22 GMT

Hi Rob and thank you very much for your brillant work !

With Document Anallyzer V1.15 my "Running Script" window stays always empty and the load treatment never ends. Moreover, i don't get any log files. Do you have any ideas ?

Best regards,

Johann

Re: Document Analyzer V1.15

rwunderlich — Mon, 14 Oct 2013 19:50:09 GMT

HI Johann,

Try killing the reload and then loading with a ctl-R. Let me know how that works out.

-Rob

Re: Document Analyzer V1.15

rwunderlich — Mon, 14 Oct 2013 19:52:57 GMT

Hi Rahgu,

Can you zip up and post the directory

C:\Users\rdube1\AppData\Local\Temp\qvwork\Client and Investment Analytics - CopyReduced

-Rob

Re: Document Analyzer V1.15

Tue, 03 Dec 2013 09:44:13 GMT

Hi Rob,

I was having the same issue however reloading and using ctl-R did work.

Thanks,

Jon

Re: Document Analyzer V1.15

Wed, 28 May 2014 20:33:22 GMT

I have 2 issues I need support on. First, here are the environment details:

Qlikview v11.0.11440.0 SR2 64-bit Edition (x64)

Doc Analyzer v1.15

qvw file size: 367MB

approximate number of fields: 450

Issue 1) Cannot Process Doc

I experience an issue that when I click "Process Doc" the actual document which I have browsed is opened and then hangs (goes to a Not Responding state).

Issue 2) Fields "not in use" are actually in use

It seems that since the expressions, dimensions and variables are .csv files the analyzer does not correctly read the values if they contain line breaks. This leads to a field showing up as not in use, but it is actually in use e.g. in a complex set analysis which contains line breaks.

Re: Document Analyzer V1.15

rwunderlich — Thu, 29 May 2014 00:41:21 GMT

There is an issue with processing large docs where the reload phase hangs. I'm addressing this in a new version I'm working on. In the meantime, try this as a workaround.

1 Open the Module (Ctrl-m).

2. Comment out line 107 with a REM statement so it looks like this:

REM If IsInternal() Then ActiveDocument.Reload

That will suppress the Reload phase. After the "Process Doc" completes (hopefully), reload manually using Ctrl-R.

If that doesn't resolve the problem, please post the last several lines of the CollectProperties log which you can find in your

C:\Users\userid\AppData\Local\Temp

folder,

Re the second issue about line breaks. The multi-line expressions should be properly escaped and read as multi-lines. My simple testing works out ok. Can you post the expression you are having issue with so i can test it?

-Rob

Re: Document Analyzer V1.15

Thu, 29 May 2014 14:37:31 GMT

Issue 1) Cannot Process Doc

I've commented out the line you refered to and tried again to "Process Doc". After 21 minutes of working, the application seems to be still spinning with CPU usage spiking all over the place. Is this a normal processing time?

Server details

Processor: Quad-Core AMD Opteron(tm) Processor 8381 HE 2.50 GHz (8 processors)

Memory (RAM): 32.0 GB

System type: 64-bit Operating System

Issue 2) Fields "not in use" are actually in use

At least these 3 fields were identified as not in use but are used int he expression below.

IFS.INTERFACE_CATEGORY

IFS.SPEED_MAX

IFS.CAPACITY_STATUS

Here is the sanitized expression where they are used. I have replaced actual customer values with 'DummyValue' in the Set Analaysis.

sum ({$<LOAD_DATE_IDM={'$(TrendEndPORT)'}, IFS.INTERFACE_CATEGORY={'DummyValue'}, IFS.SPEED_MAX={'DummyValue'}, INVENTORY_DEVICE_MAIN.DEVICE_CLASS = {'DummyValue'},
SC={'DummyValue'}, CONTAINER.CONTAINER_CLASS_DESC = {'DummyValue'},IFS.CAPACITY_STATUS={'DummyValue'}, BRANCH_NAME={'DummyValue'}>} InterfaceCount) -
sum({$<LOAD_DATE_IDM={'$(TrendStartPORT)'}, IFS.INTERFACE_CATEGORY={'DummyValue'}, IFS.SPEED_MAX={'DummyValue'}, INVENTORY_DEVICE_MAIN.DEVICE_CLASS = {'DummyValue'},
SC={'DummyValue'}, CONTAINER.CONTAINER_CLASS_DESC = {'DummyValue'},IFS.CAPACITY_STATUS={'DummyValue'}, BRANCH_NAME={'DummyValue'}>} InterfaceCount)

Re: Document Analyzer V1.15

rwunderlich — Thu, 29 May 2014 14:49:56 GMT

1) No, that is not a normal processing time. Is there anything in the CollectProperties.log?

2) I'll test with that expression and get back to you.

Does DocAnalyzer complete normally on other documents?

-Rob

Re: Document Analyzer V1.15

Thu, 29 May 2014 15:44:32 GMT

1) I reviewed the CollectProperties.log and didn't see anything strange. It looks like it really is just processing the document but this particular document must have so many objects that processing take a very long time. Here is the tail end of the log after I manually killed the process.

...

Writing xml file C:\Users\...\AppData\Local\Temp\4\qvwork\QlikApp\Document\LB1284.xml

Extracting Expressions

Extracting Dimensions

Extracting Fonts

Processing Sheet Objects for Sheet Document\SH16

Writing xml file C:\Users\...\AppData\Local\Temp\4\qvwork\QlikApp\Document\TX6119.xml

Extracting Expressions

Extracting Dimensions

Extracting Fonts

Writing xml file C:\Users\...\AppData\Local\Temp\4\qvwork\QlikApp\Document\TX6210.xml

DocAnalyzer works fine in the same environment on much smaller files around 20MB.

Re: Document Analyzer V1.15

rwunderlich — Fri, 30 May 2014 01:56:34 GMT

I've seen Documents with a 1000 objects take a couple minutes to extract. Can I infer from the objectid numbers you posted above that you have many thousands of objects?

-Rob

Re: Document Analyzer V1.15

rwunderlich — Sat, 31 May 2014 00:59:18 GMT

Hi Philip,

Another thought. Every object is visited, and therefore calculated. Even hidden objects and container objects are calculated. So if it's a big app, that could take a long time.

Make some selections and use File, Reduce Data to make a smaller copy and analyze that.

-Rob

Re: Document Analyzer V1.15

Tue, 03 Jun 2014 21:21:35 GMT

I am out of the customer environment now so I can't check the number of objects, but it it probably true that there are thousands of objects in the qvw. This makes sense that it would take a long time since from the long file I can see that EVERY object is profiled. Although a 400MB file is not huge, if it contains thousands of objects then it would take a long time to process.

Any news on the expression, and my idea that the line-breaks may not be considered properly? (#2 above)?

Re: Document Analyzer V1.15

rwunderlich — Thu, 05 Jun 2014 18:42:54 GMT

I added your expression to a test document and the field references were detected fine. I have not encountered any problems with line breaks in the past. Line breaks in expressions are pretty common. So I don't know why it's not being detected in your case. Could have something to do with the doc size or complexity.

I'll be releasing a major update to Doc Analyzer withing a few days. Try that version when it's available.

-Rob

Re: Document Analyzer V1.15

Thu, 05 Jun 2014 19:12:59 GMT

I will.

Thanks for all the help!

Re: Document Analyzer V1.15

rwunderlich — Mon, 09 Jun 2014 04:29:31 GMT

Note that a new version of Document Analyzer is now available. Read more about it here.

Document Analyzer V2 -- a major update

-Rob