Bug report: Outer set expression is not inherited if 1) the outset set expression selects an inexistant value, 2) there is an inner set expression

[sergiogiogio]

I would like to report what I believe is a bug in inner and outer set expressions feature (May 2023 patch 1), not sure if this forum can be a suitable location.

Given the the script, 

Fruits:
LOAD * Inline [
Type,Color,Quantity
apple,green,1
apple,green,1
apple,green,1
orange,orange,1
orange,orange,1
];

Consider the below formulae outputs, where we select apples (which exist in the data set) and kiwis (which do not exist):

Sum({<Type={'apple'}, Color={'green'}>} Quantity)	3	OK
{<Type={'apple'}>} Sum({<Color={'green'}>} Quantity)	3	OK
{<Type={'apple'}, Color={'green'}>} Sum(Quantity)	3	OK
Sum({<Type={'kiwi'}, Color={'green'}>} Quantity)	0	OK
{<Type={'kiwi'}>} Sum( {<Color={'green'}>} Quantity)	3	WRONG!! The outer set analysis is not inherited!
{<Type={'kiwi'}, Color={'green'}>} Sum(Quantity)	0	OK
Sum({<Type={'orange'}, Color={'green'}>} Quantity)	0	OK
{<Type={'orange'}>} Sum( {<Color={'green'}>} Quantity)	0	OK
{<Type={'orange'}, Color={'green'}>} Sum(Quantity)	0	OK
Sum({<Type={'kiwi'}>} Quantity)	0	OK
{<Type={'kiwi'}>} Sum( Quantity)	0	OK

As per the results above: outer set expression is not inherited if

1. the outset set expression selects an inexistant value,
2. there is an inner set expression

Could someone confirm this is indeed a bug and whether it has already been fixed in the recent version? Thanks!

[marcus_sommer]

Like hinted above I do also think that it worked as designed because from a technically point of view nothing else made sense. A set analysis is a selection which is in general an one-level-logic which just selects/excludes the chosen fields/values if possible without any extra logic to check if they exists or any kind of exception-handling.

This principle isn't changed if n selection-states are on the inner/outer-side combined/nested else they are simply performed in a certain order. Implementing any on-top logic would cost valuable performance and even more important disimproving the usability. The simplicity of the associated data-model and the selections of the chosen fields/values leads to a natively understanding colored selection-state is the biggest strengths of Qlik.

Nevertheless it's a big mistake from Qlik to announce the outer set-statement as simplification-feature and not exactly describing the functionality within the documentation. I assume it comes not directly from the developer else from more marketing-oriented people who didn't really comprehend this feature and/or wanted to give the impression that's a simple and general to use approach. IMO it's the opposite and it increased the complexity but it could be very valuable for specific use-cases.

[stijn_verhoeff]

Update from my side on this topic:

We have received a response from Qlik on the issue. While they still phrase it as 'working as designed', they have found a way to resolve the issue by introducing new syntax to Set Analysis. 

We have tested this new syntax on Qlik Cloud and it does work as expected. I suspect this will also be included in the November 2024 release, but cannot confirm that at this point. 

Introducing new syntax to Set Analysis (which new users often already find daunting), is likely not done lightly and, to me, signals that they to recognize the importance of this issue. I do still feel that this should be the default engine behaviour. 

This issue was previously reported and investigated by our R&D team with reference QB-23718. 

Our R&D team concluded this is working as designed: 

When multiple set expressions exist (no matter outer or inner), the engine evaluates the selections for each and tries to merge the results. 
If in any step the result is empty, the engine considers that the whole dataset is selected then continue with the subsequent evaluations. 

A workaround is to put the outer set (that returns empty) in the inner set. 

However, a new feature is currently being developed to address this, and is already available in Qlik Cloud. 

Here is is how the feature works, reusing the expression in your example: 

{&<Source = {'C'}>} [Total Sales (CY)] 

The only thing changed is the '&' symbol added inside the outer set expression. 

This symbol forces inner set expressions to inherit the result of the outer set expression even when the result is empty. 
Without this symbol, the empty set is not inherited but instead ignored. 

Kindly note this feature is not yet officially documented. 

[hadrienlp]

Thanks a lot for the update! That is a very good news! 

Looking forward to see this live.