Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
clarcombefmsb
Contributor III
Contributor III

Use of memory after creating QVDs

I am using the standard technique of two applications ; one to extract data to qvds and the other to load the qvds and display the data. So far, so good.

However, in my extract qvd I have not specifically dropped the tables after use so have (for example)

Inner Join(Dim_Customer)
Load Distinct
    DOC_ACCOUNT 
Resident Fact_Invoicing;

Store Dim_Customer INTO $(Qvd_Location_Invoicing)Dim_Customer.qvd
(qvd);

without a drop table afterwards. I do this in order to be able to see the data model under data model viewer.

Not only that, but in the extractor I also have some sheets to allow me to debug the loaded data.

My concern is that once I have run the extractor and then run the application load that I will have the same data loaded twice in memory. Is this concern justified ?

Questions

1) If I removed the sheets but left the tables resident without dropping them, would this release the memory once the QVDs had been created and the extractor had finished ?

2) Is there any other way that I can see the data model in the data model viewer without keeping the tables loaded in memory

3) Are there any Qlik applications available so I can see what tables are in the server memory at any one time ?

Thanks

 

Labels (2)
1 Solution

Accepted Solutions
MartW
Partner - Specialist
Partner - Specialist

Hi @clarcombefmsb, Great questions. 

I'll try to answer this to the best of my knowledge.

So first I'll just answer the questions. after that i will explain this a bit more.

 

1) If I removed the sheets but left the tables resident without dropping them, would this release the memory once the QVDs had been created and the extractor had finished ?

Answer: No. when the data is loaded this will stay in memory until the app is closed. (better explanation below)

2) Is there any other way that I can see the data model in the data model viewer without keeping the tables loaded in memory 

Answer: No

3) Are there any Qlik applications available so I can see what tables are in the server memory at any one time ?

Answer: yes and no. there are some Qlik products that Qlik have released them self. those are called the monitoring apps.

but the community also have created some monitoring apps that are also very good for example. you can try the QSDA app from Rob Wunderlich (an Qlik master).

 

Explanation:

So when loading an app what happens is that the data is loaded into the app. Also the data is loaded into memory. when opening an app that got data init it loads the data into memory. after closing the app this memory will be cleared after x amount of time.

 

because you got  2 apps(what is a good thing) ; an extinction app and an dashboard app when loading both the apps both the data is loaded into memory. When you drop the tables in the extraction app then in the data-model viewer there is nothing to show.   

so what you can do are a couple of things:

1. within the extraction app:

load the data do extraction things and store and drop the tables.

after that within the app  load the qvd files but load only the first 100 records if you don't do anything with joins or apply maps this give you the chance to look at the data but not load all the data within memory.

2. create an loop that delete all tables that are currently loaded within the app.

to only trigger this part of the script put this in a subroutine (sub) and only call this sub when you want to drop the tables after extracting data.

3. an other option is to only extract the data store and drop the tables. after that you can create a new app where you can debug the tables.

this looks counter intutive but if you extract the data first. close that app wait a bit the data is out of memory.

after that load the debug app into memory to start debugging (if necessary) close this app when done and load the dashboard app.

 

 

hope this helps.

if this helped please like and mark this as the answer

View solution in original post

1 Reply
MartW
Partner - Specialist
Partner - Specialist

Hi @clarcombefmsb, Great questions. 

I'll try to answer this to the best of my knowledge.

So first I'll just answer the questions. after that i will explain this a bit more.

 

1) If I removed the sheets but left the tables resident without dropping them, would this release the memory once the QVDs had been created and the extractor had finished ?

Answer: No. when the data is loaded this will stay in memory until the app is closed. (better explanation below)

2) Is there any other way that I can see the data model in the data model viewer without keeping the tables loaded in memory 

Answer: No

3) Are there any Qlik applications available so I can see what tables are in the server memory at any one time ?

Answer: yes and no. there are some Qlik products that Qlik have released them self. those are called the monitoring apps.

but the community also have created some monitoring apps that are also very good for example. you can try the QSDA app from Rob Wunderlich (an Qlik master).

 

Explanation:

So when loading an app what happens is that the data is loaded into the app. Also the data is loaded into memory. when opening an app that got data init it loads the data into memory. after closing the app this memory will be cleared after x amount of time.

 

because you got  2 apps(what is a good thing) ; an extinction app and an dashboard app when loading both the apps both the data is loaded into memory. When you drop the tables in the extraction app then in the data-model viewer there is nothing to show.   

so what you can do are a couple of things:

1. within the extraction app:

load the data do extraction things and store and drop the tables.

after that within the app  load the qvd files but load only the first 100 records if you don't do anything with joins or apply maps this give you the chance to look at the data but not load all the data within memory.

2. create an loop that delete all tables that are currently loaded within the app.

to only trigger this part of the script put this in a subroutine (sub) and only call this sub when you want to drop the tables after extracting data.

3. an other option is to only extract the data store and drop the tables. after that you can create a new app where you can debug the tables.

this looks counter intutive but if you extract the data first. close that app wait a bit the data is out of memory.

after that load the debug app into memory to start debugging (if necessary) close this app when done and load the dashboard app.

 

 

hope this helps.

if this helped please like and mark this as the answer