Your experience in using QlikSense with data warehouse

[markhavi]

We are going to use a data warehouse in order to aggregate huge amount of data from multiple sources. What are your experience in working with DWH from QlikSense? Are there certain "more preferable" DHW solutions for QlikSense? Have you ever faced a certain obstacles causing you risks/problems when integrating QlikSense with data warehouse solutions?

[Michael_Tarallo]

Hi Marks,

Thanks for your question.

I expect the more experienced and those that have implemented this in the field with Qlik Sense to chime in as well, but allow me to add my 2 cents.

Technically your "data warehouse" - is just another data source to Qlik. Qlik Sense and QlikView - take the data from the source and load it into its in-memory indexing engine - which compresses the data and only stores the unique values. It also associates common elements of the data based on common field names. By default - you can directly load the source(s) into the app itself, but as an implementation grows it is common to create this in-memory representation stored on disk in a .qvd file - and then load the app directly from that file. Note that you can perform CDC-like operations and procedures and update the QVD as needed. These apps loaded from the .QVD file can be refreshed on  a scheduled basis as needed.

For more detail - you may want to look at this Technical Brief it was originally designed with the QlikView product in mind but Qlik Sense and QlikView use the same engine so it is applicable - you can start around page 5.

Please let us know how you do and if we can be of further assistance.

Please mark the appropriate replies as CORRECT / HELPFUL so our team and other members know that your question(s) has been answered to your satisfaction.

Regards,

Mike Tarallo

Qlik

[markhavi]

And if we talk about huge amount of data (hundreds of GB or several TB), I believe Qlik qvd files will not be the most relevant solution in this situation, right? Can I find somewhere best practices tutorial for Qlik to work with pretty large data sets?

[vlad_komarov]

The problem with "huge amount of data" is not with QVDs, but with your applications. Data (QVDs) will be compressed and stored, the problem will be to optimize your model / UI to handle/display that information.

Try to google "qlik handling huge data" - you will see tons of posts with suggestions/issues/recommendations. 

Specifically: Large amount of data (120 million rows) ... anyone got experience?

There are few documents attached there...

Regards,

Vladimir