Skip to main content
Announcements
Have questions about Qlik Connect? Join us live on April 10th, at 11 AM ET: SIGN UP NOW
cancel
Showing results for 
Search instead for 
Did you mean: 
vishalgoud
Creator III
Creator III

App reloading from the last 20 hours ?? Need help

Our final application with UI is relaoding from 19-dec-2017  13:59:32 to till.  almost 20 hours and still going on.

app details : earlier app size is 13 GB.

now few new columns are added to few tables so not sure about the size but definately more than 13 GB.

Prod server ram : 354 GB. 4 nodes.

For the first time facning this kind of issues Please help me.

Just explored more into log got to know that 7 left joins and 7 resident loads and 1 outer join and 1 concatination is taking more time than usual...

12 Replies
sushil353
Master II
Master II

Cross check if there is any synthetic key or circular reference has formed due to the new colomns added.

vishalgoud
Creator III
Creator III
Author

was verified couple of times in debug mode in my dev server no issues DM is good.

Just explored more into log got to know that 7 left joins and 7 resident loads and 1 outer join and 1 concatination is taking more time than usual...

marcus_sommer

If you could ensure that there are no issues with other tools/processes on the machine and that maybe nor enough RAM is available or with the OS I would assume that there is an issue with the datamodel.

I think I would run this app within the dev-environment and trace the number of records from each reload - then the same on the prod-environment and I assume that the number of records will be different probably because one of the join-statements worked properly and creates a lot of additionally records.

- Marcus

vishalgoud
Creator III
Creator III
Author

Hi Marcus,

i can not run in dev env due to storage issue, i used to debug every time and it was fine with extractor and final app.

we want to test it in test env but due to version issue we can not as we have Qv 12 in test and Qv 11 in dev and prod.

we have many inline loads as mapping tables in the applymap. is inline load itself is a issue in Qv 12 or inline table which is used as a mapping table is a problem ??  Bit confused here.

Lot of strange issues after deployed in prod

1. fact table exctractor converted all the required qvds but not exit the script on its own, we manually killed it.

2. now the same with final UI app its reloaded for 20 hours and still going on, they killed it again.

3. now am suscpecting the generated qvds as well because of the final app behaviour.

these are giving me ache please suggest good method to deploy this in prod, Thank you in advance,

pls let me know if any info is not clear.

marcus_sommer

It's difficult to say how you could check for the problem. Normally the various environments should be so similar as possible regarding to the storages and the access on the data otherwise testings would have only a limited value and a move of an application between the environments will need various adjustments.

Beside this I would like above mentioned use trace and the log-files to see what's happing during the load - important are until which point runs the script normally before it stuck and how many records are loaded in which table. This should give some valuable hints which loads didn't work like expected and prevent a proper joining / (auto) concatenating with a likely resulting of huge tables and/or synthetic keys.

- Marcus

vishalgoud
Creator III
Creator III
Author

Thanks for you response,

yeah not having same versions is across diff env is a serious problem for developers like me.

Running more time is okay we can see that in log file that it took 3 -5 hours for variuous joins but taking almost 3 hours time just before the exit script is not clear to me - is that time it is taking to save the app.

and another things is Fact extractor not ended even after generating the all the required qvds. this is an another big question for my self.

it would be great if you provide some pointers to the above, Thanks Again

sunny_talwar

We have seen large delays after the dashboard has finished reloading and these delays are attributed to publishing of the application. Once the dashboard finishes reloading, it is then published to another folder. This new location is what is used by Access Point to display to your users. So, based on what I told the publishing time not only includes the time to save the document, but also time to move it (and for some reason the time to move is more than the time it would take you to move it outside of publisher).

The work around we have used for a huge dashboard is to not use publisher at all and have the source location as the final location for the qvw to be used on Access Point.

Few of the issues/problems

1) The problem is that we have to be extra careful when making updates. As this is the sole copy, any changes that you save in this dashboard will straight away shown to users on access point. If the file go bad and you don't have back up, you will be in trouble.

2) Any security restriction will have to be done outside of QMC (right click into properties and security to give user access to the dashboard)

Knowing the restriction, it still might make sense to do this for one or two big dashboards as it saves on the overall time it takes to update the application with the new data.

Best,

Sunny

vishalgoud
Creator III
Creator III
Author

Thank you sunny for such clear explanation. but 2-3 hours fine for the application which reloaded for 20 hours.

But coming to fact qvds extractor it reloaded for 6 hours and generated all the qvds but not shown successful even after 8 more hours finally we killed it manually.

as you know we were run the final UI app based on the success of Qvd extractors...any idea on why that extractor is not ending. Thanks again

My seniors are not considering all the concerns they are simply saying that some thing wrong with the script that why it is taking this much time and killed the job...

sunny_talwar

facts qvds extractor may not have finished because of couple of reasons (I can only think of two right now, but there might be others)

1) It is still reloading new data (doing joins, group bys etc) and there is nothing we can do about it

2) Server has gone into a weird state which require you to restart server/restart machine. This can be checked by trying to kill the reload. If you are unable to kill the reload, then it is probably in a weird state. Restarting server/restarting machine and then trying to reload again might help.

One way to get around this problem is to break down the qvd generation process into multiple qvd generators which run for smaller duration.

Fact Generator Task1

     Fact Generator Task2

          Fact Generator Task3

               ....

                    FinalDashboard

This will help you determine which of the fact generator qvw file is failing/not working/hanging. Additionally, you can parallel load some/all of them in order to save on some reload time if they are not dependent on the previous task.