Non-ASCII table names corrupted in DESCRIBE TABLE on Databricks Lakehouse (Delta) target

[aleksabl]

Hi,

We are hitting an issue where table names containing non-ASCII characters (Norwegian Ø) fail against a Databricks Lakehouse (Delta) target. The character survives DDL generation but is corrupted in the metadata lookup, so Replicate cannot find the table it just created.

Qlik Replicate: 2026.5.0.169, Windows
Source: IBM DB2 for LUW via Log Stream
Target endpoint: Databricks Lakehouse (Delta), Unity Catalog enabled, staging type Databricks Volume
Databricks ODBC driver: 2.10.0.1002 (64-bit)

DETAILS:

CREATE TABLE sends the correct UTF-8 name:
CREATE TABLE `[REDACTED]`.`transportør` ( `transportør` DECIMAL(6,0) NOT NULL, ... ) USING DELTA

DESCRIBE TABLE sends a corrupted name:
09:12:34 Failed to DESCRIBE TABLE
SqlState: 42P01 NativeError: 80
[TABLE_OR_VIEW_NOT_FOUND] The table or view `[REDACTED]`.`transport?r` cannot be found.
Failed to get table transport?r definition
Table '[REDACTED]'.'transport?r' does not exist, insufficient privileges or no supported columns

Because the lookup fails, Replicate treats the table as new, skips the drop, and issues CREATE, which then fails:
SqlState: 42P07 NativeError: 35
[TABLE_OR_VIEW_ALREADY_EXISTS] Cannot create table or view `[REDACTED]`.`transportør` because it already exists.

The table is suspended and never loads. Tables without Ø in the name work normally in the same task.

VERIFIED
- The character is a normal U+00F8. Verified in Databricks: hex(column_name) returns C3B8, identical in both schemas.
- Column names with æ/ø/å work fine. For example, `transportør` are created and populated correctly as columns.
- The Replicate user has full privileges on the schema. Manual DROP TABLE from the Databricks SQL editor succeeds.
- The target tables were dropped manually before retrying, so TABLE_OR_VIEW_ALREADY_EXISTS is not caused by leftover objects.

ALREADY TESTED, NO EFFECT
1. Global rule RENAME_TABLE with expression:
replace(replace(replace(lower($AR_M_SOURCE_TABLE_NAME),'Æ','æ'),'Ø','ø'),'Å','å')
This produces the correct lowercase name in CREATE TABLE, but DESCRIBE TABLE still sends transport?r.
2. Server setting "Support special characters in column names used in expressions".
3. Databricks ODBC DSN option "Unicode SQL character types" (UseUnicodeSqlCharacterTypes), with a restart of the Replicate service.

QUESTIONS
1. Is there an internal parameter on the Databricks Lakehouse (Delta) endpoint that controls character encoding for identifiers in metadata queries, so non-ASCII table names survive the DESCRIBE TABLE lookup?
2. If not, can you confirm this is the documented ASCII limitation for the Databricks target, and whether support for non-ASCII table names is on the roadmap?
3. Is the inconsistency itself considered a defect? The same task encodes the same table name correctly in DDL and incorrectly in metadata lookups.

We are applying an ASCII transliteration of the table names as a workaround so the tables load, but we would prefer to keep the original names.

[john_wang]

Hello ,

Thanks for posting this here.

I’m unable to reproduce the behavior in my lab. I’m running Qlik Replicate 2025.11 on Windows, replicating data from DB2 LUW 11.1 to Databricks 15.4 via a direct connection, without using LogStream.

Could you please confirm whether the issue occurs only when LogStream is involved? This would help us determine whether LogStream is a contributing factor.

Screen copy of both Full Load and CDC work fine for me:

Best Regards,

John.

[aleksabl]

Hi John,

Thanks for testing this.

We have not yet run it without Log Stream, so I can't confirm or rule that out.

A few differences from your lab that may matter:
- Qlik Replicate 2026.5 (you are on 2025.11)
- Databricks ODBC driver 2.10.0.1002
- Target: Databricks Lakehouse (Delta), Unity Catalog enabled, staging in a Databricks Volume

What we see (screenshots attached):

Screenshot 1: full load fails on Ø in the table name (LEVERANDØR). 

Screenshot 2: the full error text. CREATE TABLE is sent with the correct UTF-8 name `LEVERANDØR`, and the server replies TABLE_OR_VIEW_ALREADY_EXISTS.

The second symptom is probably the more interesting one. In the same task, DESCRIBE TABLE is sent with a corrupted name:

[TABLE_OR_VIEW_NOT_FOUND] The table or view `<schema>`.`transport?r` cannot be found

while CREATE TABLE minutes earlier used `transportør` correctly. Because the lookup fails, Replicate treats the table as new, skips the drop and issues CREATE, which then fails with "already exists".