topic Re: Regarding new_sub_data_type setting in task exported (json) file in Qlik Replicate

Regarding new_sub_data_type setting in task exported (json) file

iti-attunity-sup — Mon, 10 Jun 2024 08:10:22 GMT

Hello Experts

I have questions regarding the output of exported (json) file.

The customer edited expression of transformations for some tables from web console.

They exported the task and compared the exported (json) files before and after the change to see if there were any problems with the modification.

Then they found that the following settings had been added in the "transform_columns" section automatically.

"new_sub_data_type": "KAR_SUB_DATA_TYPE_UNSPECIFIED"

e.g.

before:
"transform_columns": [{
"column_name": "XXX",
"action": "KEEP",
"computation_expression": "insight_data_masking($XXX,\"text-mask\")"
},

after:
"transform_columns": [{
"column_name": "XXX",
"action": "KEEP",
"computation_expression": "insight_data_masking($XXX,\"account-number-mask\")",
"new_sub_data_type": "KAR_SUB_DATA_TYPE_UNSPECIFIED" <--- 
}, {

I have not been able to find any documentation describing this setting.

Questions:
1. What does the setting mean and when would it be added?
2. Are there any difference with or without the settings?
--> if there are any problems in the current settings how do I correct the settings.

Regards,
Kyoko Tajima

Re: Regarding new_sub_data_type setting in task exported (json) file

john_wang — Mon, 10 Jun 2024 09:18:34 GMT

Hello Tajima-san, @iti-attunity-sup

Thanks for reaching out to Qlik Community!

When the CLOB, NCLOB, STRING, or WSTRING data types are selected, you can also set a data subtype. Select either JSON or XML from the drop-down list n the Subtype column.
Make sure that the new data in the column will be compatible with the selected subtype. The
default is Regular, In JSON file it's KAR_SUB_DATA_TYPE_UNSPECIFIED which means that the selected data type will be used without a subtype.

In below example, the added column subtype in JSON format is KAR_SUB_DATA_TYPE_UNSPECIFIED .

Hope this helps.

John.

Re: Regarding new_sub_data_type setting in task exported (json) file

Heinvandenheuvel — Mon, 10 Jun 2024 13:45:20 GMT

@iti-attunity-sup - good question. Well documented. Thanks.

A couple of remarks, in addition to @john_wang's great answer.

First of all, none to the JSON Replicate exports is documented. Much of it is intuitive, but some is confusing notably in the transformation area - for starters it's name: manipulations. Much, nearly all, of it is simple JSON structured where element order and whitespace is ignored. But there are exceptions notably "databases": [{ "name": "..", "db_settings": { "$type": "... ", That $type line must be the first under db_settings.

Best you can do is what you did. Export some basis - make changes with the GUI - re-export changes and run a DIFFERENCE tool. (diff, windiff, Notepad++ compare,...)

To automate further changes clone the manual change and blindly copy the structure and syntax first and foremost. This will for example likely protect you from defaults changing in some future version.

Next you can (and should!) ask yourself (as you did) what surprise components might mean/do and who they perhaps can be exploited for further gain. That's all bonus work, time and interest permitting. Just stick to what Replicate tossed out to beging with and make sure that works after doing the additional (scripted/automated) changes.

bonus hint: sometimes it is tricky to find the json parameter names. What I do is to give it a funky value like "12345" then export, the search for that: "max_transaction_size": 12345. Now you know where to make your changes.

Hein.