Unlock a world of possibilities! Login now and discover the exclusive benefits awaiting you.
Hi,
I have been trying to find any documentation, website or anything that can give me advice on what the best practices or best ways are to go about data modelling in QlikView, but I haven;t found anything.
Can someone please provide links or info regarding this matter.
I would like to know what is the best way to develop in QlikView:
- What do I do when I have more than one Fact table? Do I concatenate them?
- Do I always concatenate codes and descriptions?
- What should I be weary of?
- What will cause performance issues, not just in my script, but also stuff like charts?
- What is the best way to go about Security, especially multi-tiered?
- Why do I get 'object exceeded allocated memory'?
I need answers and tips on how to develop, script and deploy the optimum QlikView solution. Please do not refer me to the Reference Manual, because I have looked in there for Data Modelling tips, but there aren't any.
Please advise.
Thanx!
To make it easy.
Data model advised is Star schema. If you have different fact tables, you can concatenate them, but it is not mandatory... Depend on the structure and complexity.
To create keys, you can use feature Autonumberhash128(key1,key2) and then keep the description or the data in the normal table.
Performance issues can be various, bad links, tables with no links, synthetic keys, not optimized expressions in charts, etc..)
Object exceeded allocated memory can be cause by a memory overflow ( expression really too complicated) or by an error in your chart expression.
For security, everything is described in the user guide (sorry !) but it consists of creating an access section.
If you need mode information on some subject, I guess there is a lot already developped nin the forum. But of course don't hesitate
To make it easy.
Data model advised is Star schema. If you have different fact tables, you can concatenate them, but it is not mandatory... Depend on the structure and complexity.
To create keys, you can use feature Autonumberhash128(key1,key2) and then keep the description or the data in the normal table.
Performance issues can be various, bad links, tables with no links, synthetic keys, not optimized expressions in charts, etc..)
Object exceeded allocated memory can be cause by a memory overflow ( expression really too complicated) or by an error in your chart expression.
For security, everything is described in the user guide (sorry !) but it consists of creating an access section.
If you need mode information on some subject, I guess there is a lot already developped nin the forum. But of course don't hesitate
hesten wrote:- What do I do when I have more than one Fact table? Do I concatenate them?
hesten wrote:- Do I always concatenate codes and descriptions?
Or are you asking a data modeling question? I suppose that's much more likely given the subject. So, say, you have an invoice table with customer IDs but not names, and then a customers table with the customer ID and the name of the customer. Are you asking if you should denormalize the name of the customer onto the invoice table? If so, again, my answer is sometimes. If all I'm pulling from the customer table is the name, it seems wasteful to me to clutter the layout with an additional table for one extra field. So I would probably just put the customer name on the invoice table. As mentioned above, denormalization like this simply isn't an issue in QlikView. On the other hand, if I wanted several fields from the customer table, or if other tables references the customer ID as well, I would likely build a separate customer table.
hesten wrote:- What should I be weary of?
hesten wrote:- What will cause performance issues, not just in my script, but also stuff like charts?
hesten wrote:- What is the best way to go about Security, especially multi-tiered?
hesten wrote:- Why do I get 'object exceeded allocated memory'?
hesten wrote:I need answers and tips on how to develop, script and deploy the optimum QlikView solution.
hesten wrote:Please do not refer me to the Reference Manual, because I have looked in there for Data Modelling tips, but there aren't any.
Thanks for your suggestions and insights!
I am familiar with Star Schemas, but if you do it as you would in a Data Wrehouse with more than one Fact Table, you create loops! That is a bit frustrating, but I am getting there.
With the 'Exceeded Allocated Memory' error: It isn't consistent. It happens to different charts at different times... usually they work fine but now and then it just freaks out and gives me that error. The only solution that I have found is to clone the original one and then delete it. For some reason the clone is fine.
Thanks again guys. I guess I haven't fully made the mind shift from Data Warehousing to QV
Hello hesten
Re the Exceeded Allocated Memory error: it happens to one of my applications every other day, After 2-3 days, the chart which worked previously suddenly cannot show anything anymore. I did the same thing you did and deleted the chart and put in a new chart which is exactly the same as the previous one. DUH....
I was told that I may be using too many checks or set analysis so I need to use flags and sum them up instead. This means changing the application. I am just confused as to why this could be happening if QlikView is supposed to handle fairly complex and dynamic selections in-memory.
Should I jazz up the in-memory RAM in the server OR increase allocated memory handling in the Server Console ?
Anyone in the forum has an opinion here?
I can see how set analysis might cause memory problems. My current understanding is that is uses the same set engine as normal selections (there's a thread where senior tech support states otherwise, but I later got an email from the same person saying that it basically DOES use the same engine). So if you use set analysis, I can see that QlikView would likely be keeping at least two sets active in memory - the set analysis set, and the set corresponding to the selections. Multiple sets might be handled in parallel or sequentially, and this could even depend on available memory. Flags and IF statements I could see using less memory because they process using the current set, and likely operate on one record at a time. So they probably tend to be slower, but I could see them using less memory. Most of this is just guesswork.
Assuming you're using a 64-bit server, I think it is definitely worth seeing what happens when you throw RAM at the problem. I would definitely try that before recoding everything, particularly since I suspect the recode will execute more slowly than the original code. Well, flags may improve performance a little if used with set analysis instead of with IFs or multiplication.
On behalf od the different certifications courses delivered by Qliktech, there is a lot of things to improve the speed of expressions ( using flags, avoid count features and try to replace them by sums, try to compare data with integers instead of strings,etc..).
Regarding the set analysis, it seems that their advise is... to test between set analysis and if...
As for the IF functions, set analysis seems to be more powerful with integers conditions.
Now, in my opinion, i think that set analysis is really more powerful than if functions. Especially when you want to summarize different kind of data in one table ( for example instead of doing something like IF (A and not B, Sum(xxxx),IF(B and not A),sum(YYY),sum(zzz)).
In this case ou just to have to do an expression like sum({$<A=1>} xxxx) + sum({$<B=1>} YYYY) + sum({$<C=1>} zzzz), etc..
So it really depends on the complexity of your data.
Now regarding the preferred data model, qliktech advise a star or snowflakes shema in their certification courses.
But this is more to avoid synthetic keys. So if your model is quite simple and you don't have the need to create complex links, then, why don't use a classic model ?
Finally regarding the error message on the table it can be a lot of things, memory exceeded, maybe, errors in expression, soometimes also, it happens when you expand a node in a pivot table that contains a blank or a null value. More memory you have on your server, easier it wil be for you. In some cases (in some applications i mean) memory grw up and fast, it is important to keep an eye on it specially during development and test of the application.
I hope this will be useful for you.
Rgds,
Sébastien
Hi John and Spastor
Thank you for your thoughts. I like set analysis but sometimes it takes so much brain power that I use IF....THEN statements. Re my applications, I try to keep them simple with one or two tables and a calendar. The Allocated Memory Exceeded problem I have tried to solve by bumping up the allowed memory in the Advanced tab in the Server Mgt Console to 2048 and am crossing my fingers from here on.