Netezza Target endpoint - Allow to specify control characters ("0x1e") as column delimiter

RajT — Tue, 21 Sep 2021 17:34:58 GMT

Business Justification

The main reason we adopted the Qlik replicate solution is to replace custom Mainframe data pipelines with a sound replication solution. Qlik delimiter limitation and not having an official support around using control characters as delimiter is limiting the adoption.

Why use Control characters like “0x1e” as column delimiter?

Over the years of managing mainframe data we found that the delimiter “0x1e” is a sound delimiter that does not show up within character dataset and does not cause load jobs to fail on distributed side.

What is the request?

Phase1: Asking Qlik to officially support use of control characters using method mentioned below.
Phase2: Enhance the Qlik product to use decimal number for the control character with replicate UI.

Specifying Control characters as column delimiter to csv load files.

Currently, there are two steps to specify a delimiter when loading into Netezza. 1) set the delimiter at $info.query_syntax.csv_delimiter internal parameter 2) specify the delimiter in provider syntax file.
For 1) above, replicate do not allow to specify “control” characters like 0x1E via GUI. We found a crude way to do it, but Qlik SME/CF/CSE team confirmed this is unsupported and should go via ideation. Below details on the workaround (but unsupported) we are currently using
Add internal parameter $info.query_syntax.csv_delimiter in Netezza target endpoint and set the value to desired delimiter created using Notepad++
Control character creation using Notepad++
- Open a new notepad++ document, hold “Alt” key and enter “030” using Numeric keypad and release “Alt” key to see control character in the document.
- Copy control character from notepad++ document and past into $info.query_syntax.csv_delimter value.
Update Provider syntax file override for Netezza load statement.

"load_data_exe_params": "-quotedValue DOUBLE -host ${SERVER} -db ${DATABASE} -nullvalue attN -u ${USER_NAME} -pw ${PASSWORD} -delim 030 -t \\\"${TABLE_NAME}\\\" -df \"${FILENAME}\" -escapeChar \\ -outputDir \"${OUTPUT_DIR}\" ",

Re: Netezza Target endpoint - Allow to specify control characters ("0x1e") as column delimiter - Status changed to: Closed - Declined

Nulee_Massaro — Tue, 04 Apr 2023 18:36:35 GMT

Re: Netezza Target endpoint - Allow to specify control characters ("0x1e") as column delimiter - Status changed to: Closed - Declined

RajT — Wed, 05 Apr 2023 21:14:42 GMT

I see it is declined, what is the solution.
Thanks

Re: Netezza Target endpoint - Allow to specify control characters ("0x1e") as column delimiter

Nulee_Massaro — Thu, 06 Apr 2023 12:15:35 GMT

@RajT We are currently in the process of moving all ideations to a new system. In an effort to make this migration easier, we are closing older ideations that did not see a lot of activity (likes/comments). If you are still interested in this enhancement, please open a new ideation so it repopulates as "new" in our system.

Thank you. Let me know if you have any questions!

idea Netezza Target endpoint - Allow to specify control characters ("0x1e") as column delimiter in Suggest an Idea

Netezza Target endpoint - Allow to specify control characters ("0x1e") as column delimiter

Re: Netezza Target endpoint - Allow to specify control characters ("0x1e") as column delimiter - Status changed to: Closed - Declined

Re: Netezza Target endpoint - Allow to specify control characters ("0x1e") as column delimiter - Status changed to: Closed - Declined

Re: Netezza Target endpoint - Allow to specify control characters ("0x1e") as column delimiter