best approach for large data table

[#1]

Really need help and clarification please.

We have been working on a document where we are laoding all our customers data and associated information (inc arrears, internat access, preferences) - all data we have on our housing management system.  The key thing with this app is we want it all to show in one table so among several things we can do in the app we want to also export it all into excel as one table.

We have about 60 columns and then up 28,000 rows of records so not massive by any stretch. We started doing this as a table box and added all the columns to be displayed. This worked in that it shows data but then we kept getting 'inconsistency type D - ran out of memory 2MB'. In response to this a post on the community suggested doing it as a straight table instead. This has overcome the out of memory issue; however, unsure if its laoding all the data.

Is there a defintive answer over whether a straight table or table box is the best approach?

Can anyone explain why when using a table box (which is much easier as it eliminates needing to develop dimensions and is our preference) it can cause memory issues?

Chris

[dominicmander]

Hi Chris,

A Table Box is not really meant as an object for reporting. A table box will display every unique combination of values in the fields added. So, if you have two records in your data with the same values (or you leave out some columns that would make the records unique) they would be de-duplicated in a table box. Furthermore, no aggregation is possible in a table box.

A Straight table is definitely the right object in your scenario. I'm curious to understand why you are unsure if the straight table is returning all the rows? The only case where it might not return a record is if all the measures are zero, but there is a setting on the presentation tab of the chart properties to enable/disable supression of zero values.

[dominicmander]

One further thought ...

Often, big flat tables to export to excel are included so that users can perform further analysis in Excel of the data (as appears to be the case here).

It would be worth thinking about two things here ...

1 - What are the users doing in Excel with this data that isn't catered for in the other parts of the QlikView report? If 20 users are each spending an hour doing the same analysis independently should it be build in to the reporting to save them all the time and effort?

2 - Do they need all the data in one table to answer the questions they need to answer in Excel, or could the data be split up into "subject areas" ... smaller tables that could be more easily exported and analysed, with users choosing which table to turn to depending on the question they wish to answer.