Governance_Dashboard - identify fields again ...

[datanibbler]

Hi,

I am still working on extracting parts of the available data from the „standard GovernanceDashboard“ running once a day and building our own customized version on that basis.

I have just figured out where I can get a lot of data about all the tasks scheduled in the QEMC. That is very good because we really do have a lot of tasks there. There is one info that I cannot get from the GovernanceDashboard, the order in which Jobs in the QEMC must run - simply because we have not set any Tasks to run in sequence, but they all have separate time-dependent Triggers. I am working on putting together that Piece of info in an Excel file which I can then link to what I get from the GovernanceDashboard.

Only one piece of information is missing – the duration of a specific task. That is in the log, visible in the QEMC, so it must be somewhere. But we do have that in a "Monitoring.txt" generated by every app - which simply contains one timestamp at the start of the script and one at the end - which I could also use to decide whether or not a specific app was executed on a given day.

Maybe someone can explain to me what exactly is in the field "StartAt" in the table "PubDocTask"? There seem to be several pieces of Information in there. I will also try to find an answer or to Piece it together looking at the corresp. Tasks in our QEMC.

Thanks a lot!

Best regards,

DataNibbler 

[datanibbler]

Well, the first part is a date, the date that is set as start_date for a schedule, that's clear.

The "T" probably means that the Trigger is time-dependent and the following time is the set execution_time, also clear. Only the last bit, that "+02:00" (or similar) is not yet clear to me ...

[Tyler_Waterfall]

I do not have experience here but will ask around to see if there is some knowledge about this.

[rwunderlich]

You can get a lot of task information (but not duration, sorry) from

Qlikview Cookbook: QVPR Analysis http://qlikviewcookbook.com/recipes/download-info/qvpr-analysis/

Perhaps you can find some info to extract there or something to help your analysis.

-Rob

http://masterssummit.com

http://qlikviewcookbook.com

[datanibbler]

Thanks a lot Rob!

I will have a look.

Meanwhile, I am analyzing the "Filelist" generated by the GovernanceDashboard which Shows me all the qvw- and qvd-files generated.

Am I right in assuming that the fields "LoadDate" and "LoadDateNum" are valid for qvw, but not for qvd files whereas the fields "FileDate" and "FileTime" are valid for qvd files? It seems so ...

[Tyler_Waterfall]

As I recall, LoadDate and LoadDateNum relate to the actual reload of the Governance Dashboard. So if you reload the dashboard each day for 21 days, you would see 21 distinct values of LoadDate. These relate to the file metadata (QVD only I think) over time - allowing you to see QVD filesize over each of the 21 days you reloaded the Governance Dashboard.

FileDate and FileTime are the actual file time (last saved/modified) for the QVWs and QVDs.

[datanibbler]

Thanks a lot Tyler!

I don't really think so, but I will try that out. Most of our apps actually have to run every day, most of them run in the early morning. I will have a Close look at the times. I can tell from the QEMC exactly when the GD is reloaded every day. In the "FileList" (a table), I have one "LoadDate" for every record - those would all be the same if that was always just the ReloadTime of the GovernanceDashboard itself, no? I will see.

I'll take one step at a time. Right now I'm working on gettting the ScriptDuration of every app running on our Server.

Best regards,

DataNibbler

[datanibbler]

Hmmm ... I can see the exact start and end time of a Task in the QEMC. Why does the QVGD not capture this - or does it? I could not find this info anywhere up to now. I did find a table in the QVGD which seems to list all the Jobs we have running in the QEMC, I have a start time of every Task, though only the scheduled one, but I do not have the Duration ...

True, I could deduce the Duration once I have associated a qvw with all the qvd_files it generates, if any, and their filetime - the last filetime is logically the very last Thing a script does, so if I have that and a start_time, I can deduce the Duration - only not all qvw_files generate any qvd_files.

My current Approach is having a few lines of code to capture that in the script of every app itself - it would just be easier and somewhat more elegant if I could get that directly from the Server.

I will have another look at the QVGD and what I can find online and I will have a look at what Rob suggested - though he says that Duration is not in there - but maybe someone has already had the same struggle and figured out how to do it?

Thanks a lot!

Best regards,

DataNibbler

[rwunderlich]

I believe QMC gets it's execution duration and history from the DIstributonService folders:

TaskExecutionHistory

TaskResults

You should be able to get the data you want from there.

-Rob

http://masterssummit.com

http://qlikviewcookbook.com

[datanibbler]

Hi Rob,

thanks a lot!

if that is so, there's no way I can currently get those Infos - the subdirectories of the "DistributionService" Directory are all empty. There are no logs there since we have not turned on the logging functionality in QlikView_Server 😉 The other subdirectories are empty, too, I guess that's for the same reason.

The Thing is a bit tricky because we have been having Trouble with our Server quite frequently in the last few weeks and we are yet unable to exactly identify what is the reason, I guess those few lines of code needed in each app to write that small txt file - implemented as an INCLUDE to Keep it all consistent - is way less resource-heavy than another function to be carried out by the Server,  but well, we are at  loss yet as to what are the reasons - which is not being helped by the fact that the Server itself is located in FFM and administrated by another Company as usual ...

Best regards,

DataNibbler