I think there is no general and therefore satisfying solution possible. The comma will be always treated as parameter-delimiter for the variable. There is no possibility to change the behaviour or to mask the comma.
This means you will need workarounds to get such logics to work. This may be to replace the comma with another char by calling the variable and the variable itself contained also a logic to revert it again. This could become rather tedious ... maybe suitable for certain use-cases but rather not for general approaches.
Depending on the intended scenarios it's also thinkable to structure the variables different especially splitting them into several parts to avoid the comma-issue and other difficulties which could occur by nesting variables. Again not really recommended for a general use.
More practically but not without challenges is to avoid the commas by using different syntaxes and/or functions, for example:
One intend would be to show numbers in a correctly rounded format with exactly one decimal and K M B ... afterwards for respective blocks of thousands.
currently we applied the following formatting varibale to almost all metrics in the apps:
SET vNumericalAbbreviation = 'dual(if(fabs($1)>=1000, if(fabs($1)>=1000000, round($1/1000000,0.1)&chr(77),round($1/1000,0.1)&chr(107)),round($1,0.1)),$1)';
Moving the set expressions itself into script wouldn't be a viable option because then users can't do ad-hoc changes. Replacing the escaped character might work, though it will increase complexity even further.
Up until now I tried to separate the , parts e.g by using * but we recently stumbled across one expression which i didn't know how to rewrite. Splitting the whole expression into multiple blocks wouldn't work because the variable function needs to wrap the whole expression.
it is important that some dimensions are always filtered the same way while others should be filtered depening on the current analysis state. (dimensions not listed in the expression) In order to exclude "Sonderfall" and "Kostenart ..." dimensions from the filter we had to add them with , a simple | would not work because then there would also be entrys which should have been filtered by the other set modifiers... (unless you could use logical brackets in set expressions - which to my knowledge you can't