topic Re: Mapping expression results not pushing through to data warehouse in Qlik Compose

Mapping expression results not pushing through to data warehouse

JacobTews — Fri, 14 Mar 2025 20:49:07 GMT

I have updated the mapping for a particular field but can't figure out how to convince Compose to make the changes on previously written records. New records to the table are correct.

Testing the expression in the GUI gives the expected result:

However, after loading, only newly-written values are correct in the table in the data warehouse:

I have tried running an incremental load as well as a full load on the table, have cleared the metadata cache and the landing cache. It seems like I'm missing something obvious...any ideas?

Re: Mapping expression results not pushing through to data warehouse

barbm — Fri, 15 Mar 2024 17:51:55 GMT

Jacob - It is the full load that will make the changes to the previously written records. Just make sure that mapping is included in the full load ETL set and that you've generated etl for that full load after making any changes.

Re: Mapping expression results not pushing through to data warehouse

JacobTews — Fri, 15 Mar 2024 18:10:13 GMT

That's definitely what I expected, too.

I checked the task statements, and the function is called correctly when populating the staging table:

Not sure where to turn next...

Re: Mapping expression results not pushing through to data warehouse

JacobTews — Fri, 15 Mar 2024 18:18:53 GMT

For the good of those following along at home, I also just tried the fake "Drop and Recreate" which is sometimes needed to fully clear the cache: in the project settings, select "Generate DDL scripts but do not run them," drop and recreate the data warehouse (which executes nothing but tells Compose that it did) to clear cache, then deselect the "Generate DDL..." option.

Alas, this was unsuccessful in solving the problem.

Re: Mapping expression results not pushing through to data warehouse

JacobTews — Mon, 18 Mar 2024 15:46:16 GMT

An update today:

I went through the full load ETL tasks one SQL statement at a time, and the behavior still doesn't make sense to me. I would expect that running a full load would result in 'RED WILLOW' being updated with 'Red Willow', but this is not what is occurring. The vast majority (93%) of the records in the table remain as originally written.

@Dana_Baldwin any ideas what I'm doing wrong here? I would like to understand how to do this correctly, because there are already additional updates that I need to make to other tables and columns, and almost certainly more in the future. Thanks!

Re: Mapping expression results not pushing through to data warehouse

JacobTews — Tue, 26 Mar 2024 15:49:59 GMT

@shashi_holla any thoughts here?

Re: Mapping expression results not pushing through to data warehouse

JacobTews — Fri, 29 Mar 2024 18:38:50 GMT

Well, everyone, with the help of @Suresh_Kumar I now know what the problem is:

The function I am using transforms a string into Proper Case, but SQL Server is case-insensitive, so Compose does not detect the casing as a change and thus does not update any values.

Haven't decided how to solve it yet, but that's the issue, for my legions of fans following this thread! 😆

Re: Mapping expression results not pushing through to data warehouse

sureshkumar — Tue, 02 Apr 2024 09:18:55 GMT

Hello @JacobTews
Thank you, one of the customers had similar issue and which was addressed in a below community post

https://community.qlik.com/t5/Qlik-Compose-for-Data-Warehouses/Case-Insensitivity-in-SQL-Server/m-p/2122446#M1129

Regards,

Suresh

Re: Mapping expression results not pushing through to data warehouse

JacobTews — Tue, 02 Apr 2024 14:51:06 GMT

That was helpful; thanks for the tip, @sureshkumar.

In my case, the source for the fields in question is always uppercase, so I manually updated the existing records (in SSMS) and now new records flow through using the ToProperCase UDF.

Re: Mapping expression results not pushing through to data warehouse

sureshkumar — Wed, 03 Apr 2024 02:31:26 GMT

Hello @JacobTews

If my previous post helps, then kindly Accept it as Solution.

Regards,

Suresh