Qlik Community

Qlik Sense App Development

Discussion board where members can learn more about Qlik Sense App Development and Usage.

Re: Derived fields not working in Set Analysis

Sorry, haven't had a chance to research it. If anyone has any input, please share.

plexpro52
Contributor

Re: Derived fields not working in Set Analysis

Hello Ralf,

>> ...these fields are calculated fields, not real!  ...nobody has showed me this working with 500 M rows.

Hmm, I was considering changing my set-up for calendars...  Better not to change from a canonical calendar, I guess?

René

Luminary
Luminary

Re: Derived fields not working in Set Analysis

I tested it on 100M rows, works pretty well! And its much faster during load.

Highlighted
MVP
MVP

Re: Derived fields not working in Set Analysis

Why this should be faster in LOAD than a Master Calendar?

Employee
Employee

Re: Derived fields not working in Set Analysis

Ralf

Calculated dimensions have been changed: When possible, a calculated dimension will use the code for derived fields. Which means that you CAN have selections in them. See picture below where I have a selection in a calculated dimension.

Derived field.png

However, if a calculated dimension is based on two or more real fields, the old code for calculated fields will be used. (There may be other cases also. I haven't investigated this yet...)

Further, if the derived field is defined in the script, like in a calendar, it should be possible (and fast) to use it in Set Analysis.

HIC

plexpro52
Contributor

Re: Derived fields not working in Set Analysis

Hello Henric,

Do you know whether any examples of integrating the derived field approach with a canonical calendar, or am I misunderstanding the approach?

Thanks,

René

MVP
MVP

Re: Derived fields not working in Set Analysis

Hi Henric,

thanks, I'm aware of that. In situations with large applications and many rows (500 M+) I have seen some unexpected single threaded processing of associations (filter on dimension table), charts with measures on two tables or calulated dimensions. Also, set analysis on high volume seems to slow down compared to a use of a 2nd dimension. Very often the only acceptable solution (from point of user experience) is having everything joined and pre-calculated in a huge single table and charts with few dimensions and measures.

- Ralf

Employee
Employee

Re: Derived fields not working in Set Analysis

If you want a canonical date, you will need to create the bridge table manually. Then you can use a derived field for the date in the bridge table.

HIC

Employee
Employee

Re: Derived fields not working in Set Analysis

If you have very large applications, the single threaded phase of a chart evaluation can indeed take a considerable time. And then, a solution is to join everything into one fact table.

However, this problem exists irrespective of whether you use set analysis (or derived fields) or not. So, I don't think that it gets worse because you use derived fields.

HIC

Re: Derived fields not working in Set Analysis

I finally got around to testing the Derived Fields... Some interesting results:

- Comparing an app with a derived calendar, with a "conventional" app with a regular Master Calendar - the two applications ended up with precisely the same size on disk, which tells me that Derived fields are stored in memory just like the regular fields (originally, I thought that they are "virtual", not stored)

- Using Derived calendar fields compared to "conventional" calendar fields - getting the same performance for the most part.

- However, in many instances, Qlik Sense seems to "freeze" and enter into a long (possibly infinite) calculation phase when trying to build lists of derived values. It happens in several situations:

     - Adding a derived field into a Filter Pane

     - Adding a derived field as a Chart Dimension

     - Opening a "Selection Tool" and scrolling to the right to show all derived Calendar fields

After this problem happens for the first time, several random things stop working, and I usually have to close the App and start all over again. I suspect it's a bug that needs to be looked into.

Tested in Qlik Sense Desktop, ver. 3.1 SR3.

cheers everyone!

Oleg Troyansky

Did you know that Masters Summit for Qlik is coming soon to Munich, Germany?

Community Browser