A perfect datamodel?

    A perfect datamodel?

     

    Quite often arise the question how to get / to develop a perfect datamodel. Like by many other things there is no perfectionism possible and therefore also no perfect datamodel achievable - a scheme generally applied to various kinds of data and very different requirements to generate insights couldn't work. Especially if you need to handle large amounts of data or you hit some limitations in your environment.

     

    If you handle only small datasets like those which are usually included in a single excel file (even it's a big one) you didn't need to worry very much about if you have a well-built datamodel or to the performance of your application - nearly each (valid) kind of datamodel will work fast enough. Are the amounts of the data are bigger and/or if you need to handle several applications or a whole environment it's recommended to apply some of the best practice approaches - for which you could get here some starting points:

     

    Get started with developing qlik datamodels

    Advanced topics for creating a qlik datamodel

    More advanced topics of qlik datamodels

     

    This meant the data-architecture and the datamodels should be relative easy to understand and to develop, the design of scripts/datamodels/gui should be quite consistent and it should be have some degree of extensibility and scalability and of course a good maintainability is a very important point.

     

    The main focus is in to find a good compromise from costs and efforts to the requirements of the business - for running a stable system in the long term.

     

    If you have to handle huge amounts of data and/or you hit some limitations regarding to your resources you might need to think something different. Then the above mentioned best practices don't neglect a performance point of view but they aren't mainly designed to optimize a datamodel respectively an application to the highest possible level.

     

    You might be forced to sacrifice some of your well-working logics, for example the quite popular link-table datamodels against a single fact-table approach or some usability, for example the use of container, dimension-groups, calculated dimensions and complex expressions against quite simple charts (one or two dimensions + simple expressions).

     

    This meant to get a maximum of performance you will need a well-planned and checked datamodel which considered how the data will be stored and processed in QlikView and you will need quite a lot of time to inquiry your data and their quality and further some time to check and verify that your assumptions and measures are right and the application worked like expected.

     

    Here you will find some postings which will help you in understanding how the data are processed and how an application could be optimized:

     

    Symbol Tables and Bit-Stuffed Pointers - a deeper look behind the scenes

    ByteSizes from Values in QlikView

     

    Happy Qliking!

    Marcus Sommer