Skip to main content
Announcements
Qlik Connect 2024! Seize endless possibilities! LEARN MORE
cancel
Showing results for 
Search instead for 
Did you mean: 
lena2a2a2
Partner - Contributor III
Partner - Contributor III

Is There Any Limits In Term Of Numbers Of ROWS In Qlik Sense Straight Table?

HI, there!!

Guys, I faced with an error.  Look, please,  attached images.

The problem is i've got message "too much raw for visualization"

I use only one dimension (amount of distinct values this dimension  is about 3000 ). But i use Direct query.

Main table in direct query has  8 mil rows.

Is There Any Limits In  Term Of Numbers Of ROWS In Qlik Sense Straight Table?

What could be cause such error for direct query?

Labels (3)
1 Solution

Accepted Solutions
marcus_sommer

AFAIK there are no limits in regard to the number of rows but restrictions in regard to the max. consumption of RAM. It's a protection against crashes of the application or even from the server. I don't know if there are any direct configurations in Sense to customize it but I wouldn't recommend such changes else taking it as a serious hint to change the datamodel and/or the dimensions and calculations within the object.

Your issue might be directly related to the used direct query feature which is IMO not designed to pull large datasets into fact-tables and therefore probably not very suitable for your task.

- Marcus

View solution in original post

4 Replies
marcus_sommer

AFAIK there are no limits in regard to the number of rows but restrictions in regard to the max. consumption of RAM. It's a protection against crashes of the application or even from the server. I don't know if there are any direct configurations in Sense to customize it but I wouldn't recommend such changes else taking it as a serious hint to change the datamodel and/or the dimensions and calculations within the object.

Your issue might be directly related to the used direct query feature which is IMO not designed to pull large datasets into fact-tables and therefore probably not very suitable for your task.

- Marcus

lena2a2a2
Partner - Contributor III
Partner - Contributor III
Author

direct query  is very interesting feature of QLik products. really i can  not realize properly how does it work.

are  8 mil rows in fact table   too many? is it can be reason of restriction in viz?

acording to documentation of  Direct query  all calculations are made on dwh server but not QlikSEnse server, why can be RAM overloaded?

 

 

marcus_sommer

Just to make sure that there is no misunderstanding I meant RAM restrictions on the single object-level and not in regard to the server.

I don't know how direct query tables are loaded and associated with the datamodel but I have some doubts that they are loaded like normal Qlik data (each field with only distinct values within an own system-table which is linked with a bit-stuffed pointer to the real data-tables) because it would in general require to refresh those system-tables and the pointers. Therefore I assume they are more or less loaded like they are in sql within an additionally layer in the datamodel.

This may cause a significantly increase of the RAM consumption while creating the virtual table which is underneath your visible table in the UI. Virtual table means to combine all required dimensional information on which then the applied measures are performed (they need always a dimensional context). Such virtual tables could be several orders of magnitude bigger as the visible one in the UI. Especially if the dimensions/expressions use nested if-loops, aggr-constructs and/or interrecord-functions it will require a lot of resources particularly if the included fields didn't come from one table else multiple ones and in the worst case associated through link-tables.

AFAIK direct query is aimed to pull small datasets like a few real-time currencies or similar stuff into an application but not to pull large fact-tables which will take often too much time - the response times from the queried database + network is usually much higher as just keeping these data already within the normal loaded datamodel. Therefore this would be my first suggestion - just load them normal.

- Marcus

lena2a2a2
Partner - Contributor III
Partner - Contributor III
Author

Thanks a lot!!