How does Qlik's handle single quotes in variables?

[mmarchese]

Can someone please explain how Qlik handles single quotes with regard to variables and dollar sign expansion? I tried the test cases below, and the results are bizarre. There seems to be no consistent pattern.

My immediate goal was to be able to do this:

Num(..., $(format));

instead of having to do this:

Num(..., '$(format)');

But I want to understand this is general.

And I want to know why Qlik's auto-generated boilerplate looks like this:

SET ThousandSep=',';
...

instead of this:

SET ThousandSep=,;
...

Ok, here are the test cases:

	Declaration	Result	Comment
1.0	Set v_f1 = '#,##0';	#,##0	So Qlik automatically strips outer single quotes with Set?
1.1	Set v_f1_1 = Num(Sum(m_qty), '$(v_f1)');	Num(Sum(m_qty), '#,##0')	Consistent with result 1.0
1.2	Set v_f1_2 = Num(Sum(m_qty),  $(v_f1));	Num(Sum(m_qty), #,##0)	Consistent with result 1.0
2.0	Let v_f2 = '#,##0';	#,##0	This makes sense since I used Let.
2.1	Set v_f2_1 = Num(Sum(m_qty), '$(v_f2)');	Num(Sum(m_qty), '#,##0')	Makes sense, consistent with result 2.0
2.2	Set v_f2_2 = Num(Sum(m_qty),  $(v_f2));	Num(Sum(m_qty), #,##0)	Makes sense, consistent with result 2.0
3.0	Set v_f3 = ''#,##0'';	''#,##0''	So Qlik DOESN'T strip outer single quotes with Set?  And it doesn't escape outer double single quotes either?
3.1	Set v_f3_1 = Num(Sum(m_qty), '$(v_f3)');	Num(Sum(m_qty), '''''#,##0''''')	...huh?  What is this monstrosity?
3.2	Set v_f3_2 = Num(Sum(m_qty),  $(v_f3));	Num(Sum(m_qty), ''#,##0'')	Consistent with result 3.0
4.0	Let v_f4 = ''#,##0'';		This just triggers an error.
5.0	Set v_f5 = '''#,##0''';	'#,##0'	So Qlik strips outer single quotes and escapes inner double single quotes with Set?
5.1	Set v_f5_1 = Num(Sum(m_qty), '$(v_f5)');	Num(Sum(m_qty), '''#,##0''')	...huh?
5.2	Set v_f5_2 = Num(Sum(m_qty),  $(v_f5));	Num(Sum(m_qty), '#,##0')	Consistent with result 5.0
6.0	Let v_f6 = '''#,##0''';	'#,##0'	This makes sense since I used Let.
6.1	Set v_f6_1 = Num(Sum(m_qty), '$(v_f6)');	Num(Sum(m_qty), '''#,##0''')	...huh?
6.2	Set v_f6_2 = Num(Sum(m_qty),  $(v_f6));	Num(Sum(m_qty), '#,##0')	Makes sense, consistent with result 6.0

Edit:

I did some cases involving double quotes too (bolded to distinguish them from double single quotes):

	Declaration	Result	Comment
7.0	Set v_f7 = "'#,##0'";	'#,##0'	Makes sense
7.1	Set v_f7_1 = Num(Sum(m_qty), '$(v_f7)');	Num(Sum(m_qty), '''#,##0''')	...huh?
7.2	Set v_f7_2 = Num(Sum(m_qty),  $(v_f7));	Num(Sum(m_qty), '#,##0')	Makes sense, consistent with 7.0
8.0	Let v_f8 = "'#,##0'";		No error but the variable is missing

[marcus_sommer]

It's not only a query how the variable is created else also how the variable is called and where - all three parts matters if a variable has a valid (and the expected) content within the used context. This means either that there isn't a single way to create and use a variable as well as that this variety gives a lot of flexibility to cover a wide area of use-cases.

Nevertheless it's usually not so complex as it look like and a few rules of thumbs are enough to use variables successfully, like:

- never use LET if there isn't the need to calculate the variable(parts)
- is the content a number, don't use quotes by creating and calling the variable
- is the content a string use single-quotes around the variable-call, like: '$(var)'
- try to avoid quotes within the variable and if they are necessary wrap the variable with other quotes, like:
  set v1 = "F = {'a', 'b'}";     or       set v2 = '"Field with Spaces"';

Further and very important is not to forget the aim of the variables which is to simplify the things, for example to store larger content into a variable to keep the calculations clear or to avoid too much redundancy. The attempt to create for nearly each field/calculation/formatting/… a variable and furthermore to combine and even nest them is usually the opposite and could easily take much longer to develop a stable solution. Just keep it simple.

- Marcus

[mmarchese]

Marcus_sommer:

1) Thanks for your reply, but the main point of my question was to understand why Qlik is doing what it is.  If you look at my test cases, the results seem very inconsistent.  I want to understand what rules the Qlik parser is following to produce these strange results.  For instance, I challenge you to come up with a coherent explanation for case 3.1 above, where Qlik spawns 4 extra quotes.  The only way I can make sense of it in my head is to assume that Qlik is like JavaScript in that it was designed with fault tolerance in mind, which leads to lots of confusing and dangerous behavior.

2) "- is the content a string use single-quotes around the variable-call, like: '$(var)'"

Well, I could do that.  But it's cumbersome, likely to causing mistakes, and unlike any programming language I've ever seen, so I'd rather not.

3) "if [quotes] are necessary wrap the variable with other [types of] quotes"

Yes, I suppose I will take this route.

4) "Don't make a lot of variables.  Don't nest them.  Keep it simple." (paraphrased)

I disagree that avoiding variables, constants, or nesting is simpler or worthwhile over the long term.  But I'm not here to advocate for software engineering fundamentals like abstraction, do-not-repeat-yourself, avoiding magic numbers, etc.  I'm here to understand Qlik's seemingly incomprehensible behavior.

[marcus_sommer]

I fear that I'm not be able to explain the Qlik logic about the variable-handling sufficient to you. Many of the internal processes aren't official documented - of course there is the help with some descriptions and a few examples but usually it doesn't explained the underneath (nearly none tool does).

Nevertheless I will try to deduce some parts of the logic from the behaviour. Saying this I think that most of the noticed behaviour isn't really directly regarded to the variable-handling else to the way how the target-place (where the variable is called) handled the input - because the variable is just a replace for a certain content.

This means the way how Qlik handled in general the call of fields, expressions and numbers/strings is important - and of course there are various rules for it, especially if the content contained spaces or any kind of special chars. Relevant is also the context in which the content should be performed - this means expect the target-place a number or a string or a field or an expression? Also an explicit/implicit set formatting (if it contained quotes or commas) may have an effect and if the target is evaluated or not (means if an equal-sign is mandatory for an evaluation or not).

I think quite relevant to all of it is that there are no data-types in Qlik else the content is always interpreted within the used context. Surely this design-decision has some disadvantages but there are also benefits (especially to the performance) and in the end I'm not so sure that's more complex as the rather strict rules within other tools / programming languages.

Here some postings which explain the backgrounds much better as my attempt:

QlikView-Quoteology 
Escape-sequences 
Data-Types-in-QlikView 

In regard to the duplication of quotes within your case 3.1 it's an attempt of the parser to cure the invalid statement to avoid run-time errors - whereby the results are seldom useful. I don't know this kind of correction is nowadays necessary anymore.

- Marcus