Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.

Qlik Replicate

Announcements
Share your agentic AI experience, learn from others, and earn a new badge: Put Agentic AI to Work

Who Me Too'd this topic

aleksabl
Contributor
Contributor

Non-ASCII table names corrupted in DESCRIBE TABLE on Databricks Lakehouse (Delta) target

Hi,

We are hitting an issue where table names containing non-ASCII characters (Norwegian Ø) fail against a Databricks Lakehouse (Delta) target. The character survives DDL generation but is corrupted in the metadata lookup, so Replicate cannot find the table it just created.

Qlik Replicate: 2026.5.0.169, Windows
Source: IBM DB2 for LUW via Log Stream
Target endpoint: Databricks Lakehouse (Delta), Unity Catalog enabled, staging type Databricks Volume
Databricks ODBC driver: 2.10.0.1002 (64-bit)

DETAILS:

CREATE TABLE sends the correct UTF-8 name:
CREATE TABLE `[REDACTED]`.`transportør` ( `transportør` DECIMAL(6,0) NOT NULL, ... ) USING DELTA

DESCRIBE TABLE sends a corrupted name:
09:12:34 Failed to DESCRIBE TABLE
SqlState: 42P01 NativeError: 80
[TABLE_OR_VIEW_NOT_FOUND] The table or view `[REDACTED]`.`transport?r` cannot be found.
Failed to get table transport?r definition
Table '[REDACTED]'.'transport?r' does not exist, insufficient privileges or no supported columns

Because the lookup fails, Replicate treats the table as new, skips the drop, and issues CREATE, which then fails:
SqlState: 42P07 NativeError: 35
[TABLE_OR_VIEW_ALREADY_EXISTS] Cannot create table or view `[REDACTED]`.`transportør` because it already exists.

The table is suspended and never loads. Tables without Ø in the name work normally in the same task.

VERIFIED
- The character is a normal U+00F8. Verified in Databricks: hex(column_name) returns C3B8, identical in both schemas.
- Column names with æ/ø/å work fine. For example, `transportør` are created and populated correctly as columns.
- The Replicate user has full privileges on the schema. Manual DROP TABLE from the Databricks SQL editor succeeds.
- The target tables were dropped manually before retrying, so TABLE_OR_VIEW_ALREADY_EXISTS is not caused by leftover objects.

ALREADY TESTED, NO EFFECT
1. Global rule RENAME_TABLE with expression:
replace(replace(replace(lower($AR_M_SOURCE_TABLE_NAME),'Æ','æ'),'Ø','ø'),'Å','å')
This produces the correct lowercase name in CREATE TABLE, but DESCRIBE TABLE still sends transport?r.
2. Server setting "Support special characters in column names used in expressions".
3. Databricks ODBC DSN option "Unicode SQL character types" (UseUnicodeSqlCharacterTypes), with a restart of the Replicate service.

QUESTIONS
1. Is there an internal parameter on the Databricks Lakehouse (Delta) endpoint that controls character encoding for identifiers in metadata queries, so non-ASCII table names survive the DESCRIBE TABLE lookup?
2. If not, can you confirm this is the documented ASCII limitation for the Databricks target, and whether support for non-ASCII table names is on the roadmap?
3. Is the inconsistency itself considered a defect? The same task encodes the same table name correctly in DDL and incorrectly in metadata lookups.

We are applying an ASCII transliteration of the table names as a workaround so the tables load, but we would prefer to keep the original names.

Labels (1)
Who Me Too'd this topic