Performance of a straight_table chart

[datanibbler]

Hi,

I am facing the following issue:

- I have a rather large straight_table chart in my app (showing a lot of data, that is, the chart itself is not too big 😉

- When I have no selections at all, it all shows up although some fields are empty in a number of rows - I compare two large sets of data and of course there are items just in the one or just in the other, so there are empty fields.

<=> The issue is, as soon as I select "both" to have only the items that match between both sets of data - that should be somewhat less and some fields with comparisons are only displayed when I select that because they only make sense for the matched items - the chart is not available anymore,  I just get the message "maximum memory exceeded".

I wonder, why would that be when the data has actually become less with the selection? Is it because of the additional fields that should  be displayed now?

Thanks a lot!

Best regards,

DataNibbler

[jonathandienst]

Hard to say without details

How have loaded the data?

Is it in two tables or concatenated into a single table?

How are the two sets associated?

What are you selecting when you "select both"

And so on...

Jonathan

[datanibbler]

Hi John,

I load both sets of data from Excel - for now, in a second step I will load the data that is in the one from our database myself - the data in that Excel_file comes from our database anyway by another tool. That is irritating because I don't know for sure what logic that other tool is using.

That is in another app, however, because it takes a while. I load that BINARY into this one then.

=> Then I OUTER JOIN both tables, so I have only one big table in the datamodel where all my data comes from (the key is an alphanumeric ID and I have empty rows in some columns now because of unmatched items).

=> I have a field "ID_present_in" that uses the LEN() function to determine whether the ID is in both sets of data or in just one - that is calculated in the script, however. I try to have as few calculations as possible on the GUI.

=> I select the value "both" in that field to have only those items show up that can be matched.

<=> That selection also causes some additional fields in the table (comparisons that only make sense with the matched items) to be displayed which wouldn't be displayed otherwise. That may cause some more load on the GUI, I don't know - because of that I have buttons for m.o.l. all the fields causing them to show or not show.

[jonathandienst · Accepted solution]

DataNibbler wrote:

That may cause some more load on the GUI, I don't know - because of that I have buttons for m.o.l. all the fields causing them to show or not show.

Are these in conditionally displayed columns? If displaying these columns is causing the out of memory condition, then you will need to investigate which expression is causing the problem. Disable them all, the enable them one by one to see when the calculation time and memory increase (use task manger | performance to watch for a memory spike).

HTH

Jonathan

[datanibbler]

Hi,

every column in that straight table is governed by two things: A v_show variable that is governed by any of the buttons and an expression_in_variable that queries which dataset has been selected - "A, B or Both" - to have only the columns displayed that make sense acc. to that.

I am just trying to take as many expressions as possible out of the chart and do them in the script instead.

I will also have a look at the Task_manager.

Thank you!

[datanibbler]

I do not know if it makes a difference performance-wise, but currently I have those two variables that govern the display of every individual column both as conditions (on the formula_tab of the dialog).

<=> Would it make the whole thing slimmer if I moved them to the "presentation" tab instead? There is another edit_field there to govern the visibility of a column.

Or - I don't know if the type of dimension I have makes any difference? The dimension in my case is just that alphanumeric ID that forms the key - but that column is deactivated (on the "presentation" tab) and it is there once more as an expression so that I can have the COUNT of all items on top - that is not possible with a dimension, but in a way it's all about knowing how many items of a certain kind there are ...

[jonathandienst]

My understanding is that the conditional on the epxpression tab conditionally disables the expression, so it is not calculated when the condition is false.

The condition on the presentation tab hides the expression, but it is still calculated when hidden.

[datanibbler]

Hi John,

yes, that is my understanding as well - in that case, conditions on the presentation_tab would not reduce the load at all since the expressions would still be calculated.